ibicus.debias module
The debias
module provides the necessary functionality to bias correct climate models.
The foundation of the module is the Debiaser
class from which every debiaser inherits and which provides a unified interface to instantiate and apply debiasers.
The following bias correction methodologies are currently implemented in the package, each based on the respective publication cited.

A generic debiaser meant for subclassing. 
















Methodology
For a brief introduction to bias correction, and some issues to pay attention to when applying a bias correction method, have a look at the ‘What is bias adjustment?’ page.
The general idea behind bias correction is to calibrate an empirical transfer function between simulated and observed distributional parameters that bias adjust the climate model output. This can be done in a number of different ways. This table in ‘Overview’ provides an overview of the different methodological choices made by the bias correction methods implemented in the package. For a detailed description of their methodology, we refer you to the class descriptions and the cited publications.
Usage
Three types of data are required in order to conduct bias correction for a given climatic variable:
Observations / reanalysis data for given historical period:
obs
.Climate model simulation for same historical period as observations:
cm_hist
.Climate model simulation for the period that is to be bias corrected, often a future period:
cm_future
.
Let’s generate some pseudo climate data:
>>> import numpy as np
>>> np.random.seed(12345)
>>> obs, cm_hist, cm_future = np.random.normal(loc = 3, size = 40000).reshape((10000, 2, 2)), np.random.normal(loc = 5, size = 40000).reshape((10000, 2, 2)), np.random.normal(loc = 7, size = 40000).reshape((10000, 2, 2))
Every debiaser can be instatiated using from_variable()
and a standard abbrevation for a meteorological variable following the CMIPconvention:
>>> debiaser = CDFt.from_variable("tas")
This instantiates a debiaser with default settings for "tas"
(daily mean 2m air surface temperature (K)).
The following code the applies this debiaser, given the data obs
, cm_hist
and cm_future
.
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)
Variable support
Variables currently supported across debiasers include:
["hurs", "pr", "prsnratio", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax", "tasrange", "tasskew"]
However, whereas some bias correction methods such as ISIMIP have explicitly been published and implemented for all these variables (tasmin and tasmax are not explicitely debiased but can be calculated from tas, tasrange and taskew), other methods have been published only for specific variables such as precipitation or temperature. Where possible, the authors have introduced informed choices for the default settings of other variables as well.
The following table provides an overview of which debiasers currently have which default settings for which variables. Crosses in brackets signify socalled ‘experimental default settings’ that have been chosen by the creators of this package and may not have been evaluated by the peer reviewed literature. It is advised to evaluate those carefully.
Variable 


hurs 
(x) 
(x) 
(x) 
(x) 
(x) 
(x) 
x 

pr 
x 
x 
x 
x 
x 
x 
x 
x 
prsnratio 
x 

psl 
(x) 
(x) 
(x) 
(x) 
(x) 
(x) 
x 

rlds 
(x) 
(x) 
(x) 
(x) 
(x) 
(x) 
x 

rsds 
(x) 
(x) 
(x) 
x 

sfcWind 
(x) 
(x) 
(x) 
(x) 
(x) 
(x) 
x 

tas 
x 
x 
x 
x 
x 
x 
x 
x 
tasmin 
x 
x 
(x) 
(x) 
x 
(x) 
(x) 

tasmax 
x 
x 
(x) 
(x) 
x 
(x) 
(x) 

tasrange 
(x) 
x 

tasskew 
(x) 
x 
Note
A warning message is shown for variabledebiaser combinations that are still experimental.
Setting and Modifying parameters
In addition to these default setting settings, the user can also modify the settings and parameters of each debiaser.
In particular for those default settings that are still experimental, it is highly recommended to try out some alternatives.
This is possible either by setting alternative settings in from_variable()
or modifying the object attribute:
>>> debiaser = CDFt.from_variable("tas", delta_shift = "no_shift")
or:
>>> debiaser = CDFt.from_variable("tas")
>>> debiaser.delta_shift = "no_shift"
It is also possible to instantiate debiasers by directly setting the necessary parameters, bypassing from_variable()
:
>>> debiaser1 = CDFt()
>>> from scipy.stats import norm
>>> debiaser2 = QuantileMapping(distribution = norm, detrending = "none")
Some debiasers additionally provide a for_precipitation()
classmethod to help you initialise the debiaser for precipitation. Debiasing precipitation often requires setting additional arguments like a threshold under which precipitation is assumed zero and the for_precipitation()
method helps with that:
>>> debiaser = QuantileMapping.for_precipitation(model_type = "hurdle")
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)
The documentation of the individual debiasers provides some information on this.
Some debiasers eg. ISIMIP require date information to be applied in a running window. Dates should then be passed as 1d numpy arrays as keyword arguments:
>>> debiaser = ISIMIP.from_variable("tas")
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)
Whenever date information is needed this is indicated in the debiaser documentation.
When applying the debiaser we can parallelise the execution by setting parallel = True
and setting the nr_processes
(default: 4). It is also possible to activate/deactive the progressbar using the progressbar
argument and to activate a failsafe mode failsafe = True
, continuing execution when encoutnering errors:
>>> debiased_cm_future1 = debiaser1.apply(obs, cm_hist, cm_future, parallel = True, failsafe = True)
ibicus.debias Debiaser abstractclass
 class ibicus.debias.Debiaser(*, variable='unknown', reasonable_physical_range=None)
A generic debiaser meant for subclassing. Provides functionality for individual debiasers and a unified interface to apply bias adjustment.
The debiaser abstract class provides a unified interface to call the debiaser, as well as a vaeriety of setup tasks and inputchecks. In order to subclass the
Debiaser
class, the proposed debiaser needs to implement thefrom_variable()
andapply_location()
functions:apply_location()
: applies an initialised debiaser at one location. Arguments are 1dvectors of obs, cm_hist, and cm_future representing observations, and climate model values during the reference (cm_hist) and future period (cm_future). Additionallykwargs
passed to the debiaserapply()
function are passed down to theapply_location()
function.from_variable()
: initialises a debiaser with default arguments given a climatic variable either asstr
or member of theVariable
class.kwargs
are meant to overwrite default arguments for this variable. Given a dict of default arguments: with variables of theVariable
class as keys and dict of default arguments as values the_from_variable()
function can be used.
The
apply()
function, maps the debiaserapply_location()
over locations. This allows to always initialise and apply debiasers follows:>>> debiaser = LinearScaling.from_variable("tas") # LinearScaling is a childclass of Debiaser >>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)
or:
>>> debiaser = CDFt.from_variable("pr", delta_shift="no_shift") # CDFt is a childclass of Debiaser >>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)
 Attributes:
 variablestr
Variable that is meant to be debiased, by an initialisation of the debiaser. Default:
"unknown"
. reasonable_physical_rangeOptional[list]
Reasonable physical range of the variable to debias in the form
[lower_bound, upper_bound]
. It is checked against and warnings are raise if values fall outside the range. Default:None
.
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
apply_location
(obs, cm_hist, cm_future, **kwargs)Applies the debiaser at one location.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. abstract classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 abstract apply_location(obs, cm_hist, cm_future, **kwargs)
Applies the debiaser at one location.
 Parameters:
 obsnp.ndarray
1dimensional numpy array of observations of the meteorological variable at one location.
 cm_histnp.ndarray
1dimensional numpy array of values of the historical climate model run (run during the same or a similar period as observations) at one location.
 cm_futurenp.ndarray
1dimensional numpy array of values of a climate model to debias (future run) at one location.
 Returns:
 np.ndarray
1dimensional numpy array containing the debiased climate model values for the future run (cm_future).
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias RunningWindowDebiaser abstractclass
ibicus.debias LinearScaling class
 class ibicus.debias.LinearScaling(delta_type, *, variable='unknown', reasonable_physical_range=None, running_window_mode=False, running_window_length=31, running_window_step_length=1)
Implements debiasing via linear scaling based on Maraun 2016.Linear scaling corrects a climate model by the difference in the mean of observations and the mean of the climate model on the reference period, either additively or multiplicatively.
The present day model bias is calculated and then either subtracted or divided from the future climate model data.
Let \(x_{\text{obs}}\) be the observed timeseries \(x_{\text{cm_hist}}\) the simulated historical one and \(x_{\text{cm_fut}}\) the simulated future one (climate model historical and future run). Then additive linear scaling adjusts \(x_{\text{cm_fut}}\) as follows:
\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}}  (\bar x_{\text{cm_hist}}  \bar x_{\text{obs}})\]and multiplicative scaling:
\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} \cdot \frac{\bar x_{\text{obs}}}{\bar x_{\text{cm_hist}}}.\]Here \(\bar x\) stands for the mean over all xvalues.
Multiplicative scaling is classically used for precipitation (pr) and additive scaling for temperature (tas). Additive scaling amounts to a simple mean bias correction, whilst multiplicative one adjusts both mean and variance, but keeps their ration constant (Maraun 2016).
References:
Maraun, D. Bias Correcting Climate Change Simulations  a Critical Review. Curr Clim Change Rep 2, 211–220 (2016). https://doi.org/10.1007/s406410160050x
Usage information:Default settings exist for:
["hurs", "pr", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax"]
.apply()
requires: no additional arguments exceptobs
,cm_hist
,cm_future
.The debiaser works with data in any time specification (daily, monthly, etc.).
Examples:>>> debiaser = DeltaChange.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
 Attributes:
 delta_typestr
One of
["additive", "multiplicative"]
. Determines whether additive or multiplicative scaling is used. running_window_modebool
Whether LinearScaling is used in running window over the year to account for seasonality. If
running_window_mode = False
then LinearScaling is applied on the whole period. Default:False
. running_window_lengthint
Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. Default:31
. running_window_step_lengthint
Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if
running_window_mode = True
. Default:1
. variablestr
Variable for which the debiasing is used
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias DeltaChange class
 class ibicus.debias.DeltaChange(delta_type, running_window_mode=False, running_window_length=31, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)
Implements ‘delta change’ method based on Maraun 2016 as reference.This is technically not a bias correction method because the future climate model output is not transformed. Instead, the delta change method applies the climate change trend from the model to historical observations, therefore generating modified observations rather than a modified model output. So the output by
apply()
from this method has the same number of timesteps as the obs data, and not the same number as cm_fut like other debiasers.Let \(x_{\text{obs}}\) be the observed timeseries \(x_{\text{cm_hist}}\) the simulated historical one and \(x_{\text{cm_fut}}\) the simulated future one (climate model historical and future run). For an additive change a future timeseries is generated as:
\[x_{\text{obs}} + (\bar x_{\text{cm_fut}}  \bar x_{\text{cm_hist}})\]and for multiplicative change:
\[x_{\text{obs}} \cdot \frac{\bar x_{\text{cm_fut}}}{\bar x_{\text{cm_hist}}}.\]Here \(\bar x\) stands for the mean over all xvalues.
Multiplicative change is typically used for precipitation and additive scaling for temperature.
References:
Maraun, D. Bias Correcting Climate Change Simulations  a Critical Review. Curr Clim Change Rep 2, 211–220 (2016). https://doi.org/10.1007/s406410160050x
Usage information:Default settings exist for:
["hurs", "pr", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax"]
.apply()
requires: no additional arguments exceptobs
,cm_hist
,cm_future
.The debiaser works with data in any time specification (daily, monthly, etc.).
Examples:>>> debiaser = DeltaChange.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
 Attributes:
 delta_typestr
One of
["additive", "multiplicative"]
. Determines whether additive or multiplicative scaling is used. running_window_modebool
Whether DeltaChange is used in running window over the year to account for seasonality. If
running_window_mode = False
then DeltaChange is applied on the whole period. Default:False
. running_window_lengthint
Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. Default:31
. running_window_step_lengthint
Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if
running_window_mode = True
. Default:1
. variablestr
Variable for which the debiasing is used
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias QuantileMapping class
 class ibicus.debias.QuantileMapping(distribution=None, mapping_type='nonparametric', detrending='no_detrending', cdf_threshold=1e10, *, variable='unknown', reasonable_physical_range=None, running_window_mode=False, running_window_length=31, running_window_step_length=1)
Implements (detrended) Quantile Mapping based on Cannon et al. 2015 and Maraun 2016.(Parametric) quantile mapping maps every quantile of the climate model distribution to the corresponding quantile in observations during the reference period. Optionally, additive or multiplicative detrending of the mean can be applied to make the method trend preserving in the mean. Most methods build on quantile mapping.
Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. Let \(F\) be a CDF. The future climate projections \(x_{\text{cm_fut}}\) are then mapped using a QQmapping between \(F_{\text{cm_hist}}\) and \(F_{\text{obs}}\), so:
\[x_{\text{cm_fut}} \rightarrow F^{1}_{\text{obs}}(F_{\text{cm_hist}}(x_{\text{cm_fut}}))\]If detrended quantile mapping is used then \(x_{\text{cm_fut}}\) is first rescaled and then the mapped value is scaled back either additively or multiplicatively. That means for additive detrending:
\[x_{\text{cm_fut}} \rightarrow F^{1}_{\text{obs}}(F_{\text{cm_hist}}(x_{\text{cm_fut}} + \bar x_{\text{cm_hist}}  \bar x_{\text{cm_fut}})) + \bar x_{\text{cm_fut}}  \bar x_{\text{cm_hist}}\]and for multiplicative detrending.
\[x_{\text{cm_fut}} \rightarrow F^{1}_{\text{obs}}\left(F_{\text{cm_hist}}\left(x_{\text{cm_fut}} \cdot \frac{\bar x_{\text{cm_hist}}}{\bar x_{\text{cm_fut}}}\right)\right) \cdot \frac{\bar x_{\text{cm_fut}}}{\bar x_{\text{cm_hist}}}\]Here \(\bar x_{\text{cm_fut}}\) designs the mean of \(x_{\text{cm_fut}}\) and similar for \(x_{\text{cm_hist}}\). Detrended Quantile Mapping accounts for changes in the projected values and is thus trendpreserving in the mean.
For precipitation a distribution or model is needed that accounts for the mixed zero and positive value character. Default is a precipitation hurdle model (see
ibicus.utils.gen_PrecipitationHurdleModel
). However, other models are also possible,for_precipitation()
helps with the initialisation of different precipitation methods.References:
Cannon, A. J., Sobie, S. R., & Murdock, T. Q. (2015). Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? In Journal of Climate (Vol. 28, Issue 17, pp. 6938–6959). American Meteorological Society. https://doi.org/10.1175/jclid1400754.1
Maraun, D. (2016). Bias Correcting Climate Change Simulations  a Critical Review. In Current Climate Change Reports (Vol. 2, Issue 4, pp. 211–220). Springer Science and Business Media LLC. https://doi.org/10.1007/s406410160050x
Usage information:Default settings exist for:
["hurs", "pr", "psl", "rlds", "sfcWind", "tas", "tasmin", "tasmax"]
.apply()
requires: no additional arguments exceptobs
,cm_hist
,cm_future
.Next to
from_variable()
afor_precipitation()
method exists to help you initialise the debiaser forpr
.The debiaser works with data in any time specification (daily, monthly, etc.), although some of the default distributions have the best fit to daily data.
Examples:Initialising using
from_variable
:>>> debiaser = QuantileMapping.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
Initialising using
for_precipitation
:>>> debiaser = QuantileMapping.for_precipitation(model_type = "hurdle") >>> debiaser.apply(obs, cm_hist, cm_future)
 Attributes:
 distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel, None]
Distribution or statistical model used to compute the CDFs F. Default:
None
. Usually a distribution inscipy.stats.rv_continuous
, but can also be an empirical distribution as given byscipy.stats.rv_histogram
or a more complex statistical model as wrapped by theibicus.utils.StatisticalModel
class. mapping_typestr
One of
["parametric", "nonparametric"]
. Whether quantile mapping is done using parametric CDFs or using nonparametric density estimation (“empirical quantile mapping”). Default:nonparametric
. detrendingstr
One of
["additive", "multiplicative", "no_detrending"]
. What kind of scaling is applied to the future climate model data before quantile mapping. Default:"no_detrending"
. cdf_thresholdfloat
Threshold to round CDFvalues away from zero and one. Default:
1e10
. running_window_modebool
Whether QuantileMapping is used in running window over the year to account for seasonality. If
running_window_mode = False
then QuantileMapping is applied on the whole period. Default:False
. running_window_lengthint
Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. Default:31
. running_window_step_lengthint
Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if
running_window_mode = True
. Default:1
. variablestr
Variable for which the debiasing is done. Default:
"unknown"
.
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
for_precipitation
([mapping_type, ...])Instanciates the class to a precipitationdebiaser.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 classmethod for_precipitation(mapping_type='parametric', model_type='hurdle', amounts_distribution=<scipy.stats._continuous_distns.gamma_gen object>, censoring_threshold=1.1574074074074074e06, hurdle_model_randomization=True, hurdle_model_kwds_for_distribution_fit={'floc': 0, 'fscale': None}, **kwargs)
Instanciates the class to a precipitationdebiaser. This allows granular setting of available precipitation models without needing to explicitly specify the precipitation censored model for example.
 Parameters:
 model_typestr
One of
["censored", "hurdle", "ignore_zeros"]
. Model type to be used. Seeibicus.utils.gen_PrecipitationGammaLeftCensoredModel
,ibicus.utils.gen_PrecipitationHurdleModel
andibicus.utils.gen_PrecipitationIgnoreZeroValuesModel
for more details. amounts_distributionscipy.stats.rv_continuous
Distribution used for precipitation amounts. For the censored model only
scipy.stats.gamma
is possible. censoring_thresholdfloat
The censoringvalue if a censored precipitation model is used.
 hurdle_model_randomizationbool
Whether when computing the cdfvalues for a hurdle model randomization shall be used. See
ibicus.utils.gen_PrecipitationHurdleModel
for more details. hurdle_model_kwds_for_distribution_fitdict
Dict of parameters used for the distribution fit inside a hurdle model. Default: location of distribution is fixed at zero (
floc = 0
) to stabilise Gamma distribution fits in scipy. **kwargs:
All other class attributes that shall be set and where the standard values shall be overwritten.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias ScaledDistributionMapping class
 class ibicus.debias.ScaledDistributionMapping(distribution, mapping_type, pr_lower_threshold=1.1574074074074074e06, distribution_fit_kwargs={}, cdf_threshold=1e10, running_window_mode=False, running_window_length=91, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)
Implements Scaled Distribution Matching (SDM) based on Switanek et al. 2017.SDM is conceptually similar to QDM, and in the same ‘family’ as CDFt and ECDFM. It is a parametric quantile mapping approach that also attempts to be trend preserving in all quantiles. In addition to the quantile mapping the method also contains an event likelihood adjustment.
SDM scales the observed distribution by changes in magnitude and additionally likelihood of events – either multiplicatively (for precipitation) or additively (for temperature).
Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. Let \(F\) be a parametric CDF.
Temperature (
tas
): absolute scaled distribution mapping
First CDFs are fitted to both historical and future climate model values as well as observations. The default settings for
tas
use a normal distribution. Then the scaling is calculated as:\[\text{scaling} = [F^{1}_{\text{cm_fut}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))  F^{1}_{\text{cm_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))] * \frac{\sigma_\text{obs}}{\sigma_\text{cm_hist}}\]where \(\sigma_\text{obs}\) and \(\sigma_\text{cm_hist}\) refers to the standard deviation of a normal distribution fitted to obs and cm_hist. Given the CDFs for obs, cm_hist, cm_fut then recurrence intervals for all three are calculated as:
\[\text{RI} = \frac{1}{0.5  \CDF  0.5\}\]and a scaled recurrence interval (RI) and CDF as:
\[\text{RI}_{\text{scaled}} = \text{max}\left(1, \frac{\text{RI}_{\text{obs}} \cdot \text{RI}_{\text{cm_fut}}}{\text{RI}_{\text{cm_hist}}}\right)\]\[\text{CDF}_{\text{scaled}} = 0.5 + \text{sgn}(\text{CDF}_{\text{obs}}  0.5) \cdot \left 0.5  \frac{1}{\text{RI}_{\text{scaled}}} \right.\]Then the adjusted values are given as follows:
\[F^{1}_{\text{obs}}(\text{CDF}_{\text{scaled}}) + \text{scaling}\]Precipitation (
pr
): relative scaled distribution mapping
For precipitation first in obs, cm_hist and cm_fut all values below a given threshold are set to zero rain. Let \(\text{# rain}\) design the number of rain days and \(\text{# total}\) the total number of days. The bias corrected number of rain days is calculated as:
\[\text{# rain}_{\text{bc}} = \text{# rain}_{\text{cm_fut}} \cdot \frac{\text{# rain}_{\text{obs}} / \text{# total}_{\text{obs}}}{\text{# rain}_{\text{cm_hist}} / \text{# rain}_{\text{cm_hist}}}\]Using all values bigger than the threshold a scaling is calculated as:
\[\text{scaling} = \frac{F^{1}_{\text{cm_fut}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))}{F^{1}_{\text{cm_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))}\]\(\text{RI}_{\text{scaled}}\) and \(\text{CDF}_{\text{scaled}}\) are calculated similarly as for temperature. The final bias corrected precipitation values on rainy days are then given as:
\[F^{1}_{\text{obs}}(\text{CDF}_{\text{scaled}}) \cdot \text{scaling}\]The \(\text{# total}_{\text{bc}}  \text{# rain}_{\text{bc}}\) days with the smallest precipitation values in cm_fut are then set to zero and all bias corrected rain values are inserted at the correct locations, starting with the biggest one.
Warning
The relative SDM method does not currently allow correcting the number of precipitation days in cm_fut upwards, so to convert dry into rainy days. Should the calculated expected number of rainy days be higher than what is given inside the future climate model then the number of rainy days is left unadjusted. The method focuses on the biggest precipitation values, so this should not be an issue for most applications. However if such a correction is required this method might not be appropriate.
Reference:
Switanek, M. B., Troch, P. A., Castro, C. L., Leuprecht, A., Chang, H.I., Mukherjee, R., & Demaria, E. M. C. (2017). Scaled distribution mapping: a bias correction method that preserves raw climate model projected changes. In Hydrology and Earth System Sciences (Vol. 21, Issue 6, pp. 2649–2666). Copernicus GmbH. https://doi.org/10.5194/hess2126492017.
Usage information:Default settings exist for:
["pr", "tas", "tasmin", "tasmax"]
.apply()
requires: no additional arguments exceptobs
,cm_hist
,cm_future
.Next to
from_variable()
afor_precipitation()
method exists to help you initialise the debiaser forpr
.The method has been developed for daily data, however application on data in other time specifications (monthly etc.) is possible.
Examples:>>> debiaser = ScaledDistributionMapping.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
 Attributes:
 distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]
Method used for the fit to the historical and future climate model outputs as well as the observations. Usually a distribution in
scipy.stats.rv_continuous
, but can also be an empirical distribution as given byscipy.stats.rv_histogram
or a more complex statistical model as wrapped by theibicus.utils.StatisticalModel
class. mapping_typestr
One of
["absolute", "relative"]
. Type of SDM used. Default are “absolute” fortas
and"relative"
forpr
. pr_lower_thresholdfloat
Lower threshold used for setting precipitation values to zero in relative SDM. Only used if
mapping_type = "relative"
. distribution_fit_kwargsdict
Dict of additional arguments passed to the
distribution.fit
method. Useful for fixing certain parameters of a distribution. Default:{}
(empty dict). cdf_thresholdfloat
Threshold to round CDFvalues away from zero and one. Default:
1e10
. running_window_modebool
Whether ScaledDistributionMapping is used in running window over the year to account for seasonality. If
running_window_mode = False
then ScaledDistributionMapping is applied on the whole period. Default:False
. running_window_lengthint
Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. Default:91
. running_window_step_lengthint
Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if
running_window_mode = True
. Default:1
. variablestr
Variable for which the debiasing is done. Default:
"unknown"
.
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias CDFt class
 class ibicus.debias.CDFt(SSR=False, delta_shift='additive', apply_by_month=True, running_window_mode=True, running_window_length=31, running_window_step_length=31, running_window_mode_over_years_of_cm_future=True, running_window_over_years_of_cm_future_length=17, running_window_over_years_of_cm_future_step_length=9, ecdf_method='linear_interpolation', iecdf_method='linear', *, variable='unknown', reasonable_physical_range=None)
Implements CDFt based on Michelangeli et al. 2009, Vrac et al. 2012 and Famien et al. 2018, as well as Vrac et al. 2016 for precipitation.CDFt is a nonparametric quantile mapping method that attempts to be trendpreserving in all quantiles. CDFt applies a concatenation between a quantile mapping of future and historical climate model data and a quantile mapping of the future climate model with historical observations. It also includes a running window over the future period to account for changes in the simulated trend.
Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. In this methodology, all cdfs are estimated empirically. Let \(F\) therefore be an empirical cdf. The future climate projections \(x_{\text{cm_fut}}\) are then mapped using a QQmapping between \(F_{\text{cm_fut}}\) and \(F_{\text{obs_fut}}\), with:
\[F_{\text{obs_fut}} := F_{\text{obs_hist}}(F^{1}_{\text{cm_hist}}(F_{\text{cm_fut}})).\]This means that \(x_{\text{cm_fut}}\) is mapped using the following formula:
\[x_{\text{cm_fut}} \rightarrow F^{1}_{\text{obs_fut}}(F_{\text{cm_fut}}(x_{\text{cm_fut}})) = F^{1}_{\text{cm_fut}}(F_{\text{cm_hist}}(F^{1}_{\text{obs_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))))\]Because an empirical CDF will not be able to map values outside its fitted range, a delta shift is applied to the future and historical climate model prior to fitting empirical CDFs. This ensures that the data is approximately in the same range. This delta shift can be additive:
\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} + \bar x_{\text{obs}}  \bar x_{\text{cm_hist}}\]\[x_{\text{cm_hist}} \rightarrow x_{\text{cm_hist}} + \bar x_{\text{obs}}  \bar x_{\text{cm_hist}}\]or multiplicative:
\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} \cdot \frac{\bar x_{\text{obs}}}{\bar x_{\text{cm_hist}}}\]\[x_{\text{cm_hist}} \rightarrow x_{\text{cm_hist}} \cdot \frac{\bar x_{\text{obs}}}{\bar x_{\text{cm_hist}}}\]Here \(\bar x\) stands for the mean over all xvalues.
After this shift by the absolute or relative mean bias between cm_hist and obs are applied to both cm_fut and cm_hist, the cm_fut values are mapped as shown above using the QQmapping between \(F_{\text{cm_fut}}\) and \(F_{\text{obs_fut}}\).
If
SSR = True
then Stochastic Singularity Removal (SSR) based on Vrac et al. 2016 is used to correct the precipitation occurrence in addition to amounts (default setting forpr
). All zero values are first replaced by uniform draws between 0 and a small threshold (the minimum positive value of observation and model data). Then CDFtmapping is used and afterwards all observations under the threshold are set to zero again.If
running_window_mode_over_years_of_cm_future = True
(default) then the method is used in a running window mode, running over the values of the future climate model. This helps to smooth discontinuities.If
running_window_mode_within_year = True
(default) then the method is used in a running window mode, running over the year to account for seasonalities.If
apply_by_month = True
(default: False) then CDFt uses a running window within the year (running_window_mode_within_year) with a window length and step length of 31, so broadly applies it by month following Famien et al. 2018 to take seasonality into account. Ifapply_by_month = False
andrunning_window_mode_within_year = False
the method is applied to the whole year.
Warning
Currently only uneven sizes are allowed for window length and window step length. This allows symmetrical windows of the form [window_center  window length//2, window_center + window length//2] given an arbitrary window center.
References:
Michelangeli, P.A., Vrac, M., & Loukos, H. (2009). Probabilistic downscaling approaches: Application to wind cumulative distribution functions. In Geophysical Research Letters (Vol. 36, Issue 11). American Geophysical Union (AGU). https://doi.org/10.1029/2009gl038401
Famien, A. M., Janicot, S., Ochou, A. D., Vrac, M., Defrance, D., Sultan, B., & Noël, T. (2018). A biascorrected CMIP5 dataset for Africa using the CDFt method – a contribution to agricultural impact studies. In Earth System Dynamics (Vol. 9, Issue 1, pp. 313–338). Copernicus GmbH. https://doi.org/10.5194/esd93132018
Vrac, M., Drobinski, P., Merlo, A., Herrmann, M., Lavaysse, C., Li, L., & Somot, S. (2012). Dynamical and statistical downscaling of the French Mediterranean climate: uncertainty assessment. In Natural Hazards and Earth System Sciences (Vol. 12, Issue 9, pp. 2769–2784). Copernicus GmbH. https://doi.org/10.5194/nhess1227692012
Vrac, M., Noël, T., & Vautard, R. (2016). Bias correction of precipitation through Singularity Stochastic Removal: Because occurrences matter. In Journal of Geophysical Research: Atmospheres (Vol. 121, Issue 10, pp. 5237–5258). American Geophysical Union (AGU). https://doi.org/10.1002/2015jd024511
Usage information:Default settings exist for:
["hurs", "pr", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax", "tasrange", "tasskew"]
.apply()
requires: time argumentstime_obs
,time_cm_hist
, andtime_cm_future
next toobs
,cm_hist
andcm_future
. These are just 1d numpyarrays of dates (multiple formats are possible as long as they as convertible to numpy or datetime dates) specifying the date for each value/timestep inobs
,cm_hist
andcm_future
. If they are not specified they are inferred, assuming the first observation in all three observation/climate value arrays is on a 1st of January.The debiaser has been developed for and assumes daily data, however application on data using other time specifications (monthly etc.) is possible by setting
apply_by_month = False
, modifying the running window arguments and specifying the time arguments inapply()
.
Examples:Running without dates (they are inferred assuming the first value in
obs
,cm_hist
andcm_future
always corresponds to a January 1st):>>> debiaser = CDFt.from_variable("pr") >>> debiaser.apply(obs, cm_hist, cm_future)
Running with dates:
>>> debiaser = CDFt.from_variable("pr") >>> debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)
 Attributes:
 SSRbool
If Stochastic Singularity Removal (SSR) following Vrac et al. 2016 is applied to adjust the number of zero values (only relevant for
pr
). delta_shiftstr
One of
["additive", "multiplicative", "no_shift"]
. Type of shift applied to the data prior to fitting empirical distributions. running_window_modebool
Controls whether CDFt is applied in a running window over the year to account for seasonality. Default:
True
. running_window_lengthint
Length of the running window over the year in days (default: 31 days): the amount of days used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. running_window_step_lengthint
Step length of the running window over the year in days (default 31 days): the amount of days that are bias adjusted/how far the running window is moved. Only relevant if
running_window_mode = True
.
 running_window_mode_over_years_of_cm_futurebool
Whether CDFt is used in running window mode, running over the values of the future climate model to help smooth discontinuities. Default:
True
. running_window_over_years_of_cm_future_lengthint
Length of the running window in years: how many values are used to calculate the empirical CDF. Only relevant if
running_window_mode_over_years_of_cm_future = True
. Default:17
. running_window_over_years_of_cm_future_step_lengthint
Step length of the running window in years: how many values are bias adjusted inside the running window. Only relevant if
running_window_mode_over_years_of_cm_future = True
. Default:9
. apply_by_monthbool
Whether CDFt is applied month by month (default) to account for seasonality. This is equivalent to a running window within the year with length 31 and step length 31. Default:
Faöse
. variablestr
Variable for which the debiasing is done. Default:
"unknown"
. ecdf_methodstr
One of
["kernel_density", "linear_interpolation", "step_function"]
. Method to calculate the empirical CDF. Default:"linear_interpolation"
. iecdf_methodstr
One of
["inverted_cdf", "averaged_inverted_cdf", "closest_observation", "interpolated_inverted_cdf", "hazen", "weibull", "linear", "median_unbiased", "normal_unbiased"]
. Method to calculate the inverse empirical CDF (empirical quantile function). Default:"linear"
.
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias ECDFM class
 class ibicus.debias.ECDFM(distribution, cdf_threshold=1e10, running_window_mode=False, running_window_length=91, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)
Implements Equidistant CDF Matching (ECDFM) based on Li et al. 2010.ECDFM is a parametric quantile mapping method that attempts to be trendpreserving in all quantiles. ECDFM applies quantilewise correction by adding the difference between a quantile mapping of observations and future values and a quantile mapping of historical climate model values to the future climate model ones.
Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. Let \(F_{\text{cm_hist}}\) be the cdf fitted as a parametric distribution to climate model output data in the reference period. The future climate projections \(x_{\text{cm_fut}}\) are then mapped to:
\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}}  F^{1}_{\text{cm_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}})) + F^{1}_{\text{obs}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))\]The difference between the future climate model data and the future climate model data quantile mapped to historical climate model data (de facto future model data bias corrected to historical model data) is added to a quantile mapping bias correction between observations and future climate model data. In essence, this method says that future climate model data can be bias corrected directly with reference period observations, if the quantile specific difference between presentday and future climate model simulations is taken into account. This allows for changes in higher moments in the climate model, compared to standard Quantile Mapping where just the mean is assumed to change in future climate.
The method was originally developed with monthly data in view, however the authors of this package think that there is no reason for the method not to be applicable to daily data.
Note
As opposed to most other publications, Li et al. use a 4parameter beta distribution (
scipy.stats.beta
) fortas
instead of a normal distribution. This can be slow for the fit at times. Consider modifying thedistribution
parameter fortas
.For precipitation a distribution or model is needed that accounts for mixed zero and positive value character. Default is a precipitation hurdle model (see
ibicus.utils.gen_PrecipitationHurdleModel
). However also different ones are possible.for_precipitation()
helps with the initialisation of different precipitation methods.Reference:
Li, H., Sheffield, J., and Wood, E. F. (2010), Bias correction of monthly precipitation and temperature fields from Intergovernmental Panel on Climate Change AR4 models using equidistant quantile matching, J. Geophys. Res., 115, D10101, doi:10.1029/2009JD012882.
Usage information:Default settings exist for:
["hurs", "pr", "psl", "rlds", "sfcWind", "tas", "tasmin", "tasmax"]
.apply()
requires: no additional arguments exceptobs
,cm_hist
,cm_future
.Next to
from_variable()
afor_precipitation()
method exists to help you initialise the debiaser forpr
.The debiaser has been developed for monthly data, however it works with data in any time specification (daily, monthly, etc.).
Examples:Initialising using
from_variable
:>>> debiaser = ECDFM.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
Initialising using
for_precipitation
:>>> debiaser = ECDFM.for_precipitation(model_type = "hurdle") >>> debiaser.apply(obs, cm_hist, cm_future)
 Attributes:
 distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]
Method used for the fit to the historical and future climate model outputs as well as the observations. Usually a distribution in
scipy.stats.rv_continuous
, but can also be an empirical distribution as given byscipy.stats.rv_histogram
or a more complex statistical model as wrapped by theibicus.utils.StatisticalModel
class. cdf_thresholdfloat
Threshold to round CDFvalues away from zero and one. Default:
1e10
. running_window_modebool
Whether DeltaChange is used in running window over the year to account for seasonality. If
running_window_mode = False
then DeltaChange is applied on the whole period. Default:False
. running_window_lengthint
Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. Default:91
. running_window_step_lengthint
Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if
running_window_mode = True
. Default:1
. variablestr
Variable for which the debiasing is done. Default: “unknown”.
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
for_precipitation
([model_type, ...])Instanciates the class to a precipitationdebiaser.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 classmethod for_precipitation(model_type='hurdle', amounts_distribution=<scipy.stats._continuous_distns.gamma_gen object>, censoring_threshold=0.1, hurdle_model_randomization=True, hurdle_model_kwds_for_distribution_fit={'floc': 0, 'fscale': None}, **kwargs)
Instanciates the class to a precipitationdebiaser. This allows granular setting of available precipitation models without needing to explicitly specify the precipitation censored model for example.
 Parameters:
 model_typestr
One of
["censored", "hurdle", "ignore_zeros"]
. Model type to be used. Seeibicus.utils.gen_PrecipitationGammaLeftCensoredModel
,ibicus.utils.gen_PrecipitationHurdleModel
andibicus.utils.gen_PrecipitationIgnoreZeroValuesModel
for more details. amounts_distributionscipy.stats.rv_continuous
Distribution used for precipitation amounts. For the censored model only
scipy.stats.gamma
is possible. censoring_thresholdfloat
The censoringvalue if a censored precipitation model is used.
 hurdle_model_randomizationbool
Whether when computing the cdfvalues for a hurdle model randomization shall be used. See
ibicus.utils.gen_PrecipitationHurdleModel
for more details. hurdle_model_kwds_for_distribution_fitdict
Dict of parameters used for the distribution fit inside a hurdle model. Default:
{"floc": 0, "fscale": None} location of distribution is fixed at zero (``floc = 0
) to stabilise Gamma distribution fits in scipy. **kwargs:
All other class attributes that shall be set and where the standard values shall be overwritten.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias QuantileDeltaMapping class
 class ibicus.debias.QuantileDeltaMapping(distribution, trend_preservation, censor_values_to_zero=False, censoring_threshold=5.787037037037037e07, running_window_mode=True, running_window_length=91, running_window_step_length=31, running_window_mode_over_years_of_cm_future=True, running_window_over_years_of_cm_future_length=31, running_window_over_years_of_cm_future_step_length=1, ecdf_method='linear_interpolation', cdf_threshold=None, *, variable='unknown', reasonable_physical_range=None)
Implements Quantile Delta Mapping based on Cannon et al. 2015.QDM is a parametric quantile mapping method that also attempts to be trendpreserving. It extends ECDFM such that the two quantile mappings defined there are not only added but also divided by each other to create multiplicative correction. Furthermore it includes both a running window over the year: to account for seasonality, as well as one over the future period to account for changes in trends.
Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections.
Similar to the Equidistant CDF Matching Method based on Li et al. 2010 and implemented in
ECDFM
, this method bias corrects the future climate model data directly using reference period observations. This is then multiplied by the quotient of future climate model data and a quantile mapping between past and future climate model data to account for the change from past to future period in all quantiles.In mathematical terms, the transformation can be written as:
\[x_{\text{cm_fut, bc}} (t) = x_{\text{cm_fut}} (t) \cdot \frac{F^{1}_{\text{obs}}(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}(t)))}{F^{1}_{\text{cm_hist}}(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}))}\]This variation preserves relative trends. If absolute trends are to be preserved, the transformation is equivalent to the transformation introduced in Li et al 2010:
\[x_{\text{cm_fut, bc}} (t) = x_{\text{cm_fut}} (t) + F^{1}_{\text{obs}}(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}(t)))  F^{1}_{\text{cm_hist}}(\hat F_{\text{cm_fut}}^{(t)} ( x_{\text{cm_fut}})).\]Hereby \(\hat F_{\text{cm_fut}}^{(t)}\) is the empirical CDF of future climate model values in a window around t. \(F^{1}_{\text{obs}}\) is the inverse CDF estimated by fitting a distribution to observations. \(F^{1}_{\text{cm_hist}}\) is the inverse CDF estimated by fitting a distribution to the historical climate model run.
Delta Quantile Mapping is approximately trend preserving in all quantiles because the absolute \(\Delta_{\text{cm}}^{\text{abs}}\) or relative change \(\Delta_{\text{cm}}^{\text{rel}}\) is calculated and applied for each quantile individually.
Running window:
running_window_over_year
: controls whether the methodology and mapping is applied on a running window over the year to account for seasonality.running_window_over_year_length
andrunning_window_over_year_step_length
control the length (how many days are included in the running window) and step length (by how far the window is shifted and how many days inside are debiased) respectively.
running_window_mode_over_years_of_cm_future
controls whether a running window is used to estimate the empirical CDF \(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}(t))\) and timedependent quantiles or if this is done statically on the whole future climate model run.running_window_over_years_of_cm_future_length
andrunning_window_over_years_of_cm_future_step_length control
the length and step length of this window respectively. Estimating this information in a running window has the advantage of accounting for changes in trends.
If both running windows are active then first the running window inside the year is used to account for seasonality. Values are chunked according to this one. Afterwards the running window over years is used and values are further split up. This is just a choice made for computational efficiency and the order of running window application/chunking does not matter.
Warning
Currently only uneven sizes are allowed for window length and window step length. This allows symmetrical windows of the form [window_center  window length//2, window_center + window length//2`] given an arbitrary window center. This affects both within year and over year window.
References:
Cannon, A. J., Sobie, S. R., & Murdock, T. Q. (2015). Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? In Journal of Climate (Vol. 28, Issue 17, pp. 6938–6959). American Meteorological Society. https://doi.org/10.1175/jclid1400754.1
Usage information:Default settings exist for:
["hurs", "pr", "psl", "rlds", "sfcWind", "tas", "tasmin", "tasmax"]
.apply()
requires: time argumentstime_obs
,time_cm_hist
, andtime_cm_future
next toobs
,cm_hist
andcm_future
. These are just 1d numpyarrays of dates (multiple formats are possible as long as they as convertible to numpy or datetime dates) specifying the date for each value/timestep inobs
,cm_hist
andcm_future
. If they are not specified they are inferred, assuming the first observation in all three observation/climate value arrays is on a 1st of January.Next to
from_variable()
afor_precipitation()
method exists to help you initialise the debiaser forpr
.The debiaser has been developed for and assumes daily data, although application on data using other time specifications (monthly etc.) is possible by setting
running_window_mode_within_year = False
, modifying the running window arguments for the running window over the years of cm_future, and specifying the time arguments inapply()
.
Examples:Running without dates (they are inferred assuming the first value in
obs
,cm_hist
andcm_future
always corresponds to a January 1st):>>> debiaser = QuantileDeltaMapping.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
Running with dates and initialising using
for_precipitation()
:>>> debiaser = QuantileDeltaMapping.for_precipitation(censoring_threshold = 0.1/86400) >>> debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)
 Attributes:
 distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]
Distribution or statistical model used to compute the CDF \(F\) of observations and historical climate model values.
Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by theibicus.utils.StatisticalModel
class. trend_preservationstr
One of
["absolute", "relative"]
. If"absolute"
then absolute trend preservation is used, if"relative"
then relative trend preservation is used.
 censor_values_to_zerobool
Whether values below a censoring threshold shall be censored to zero. Only relevant for precipitation. Default:
False
. censoring_thresholdfloat
Threshold below which values shall be censored to zero if
censor_values_to_zero = True
. Relevant mainly for precipitation.
If it is used (socensor_values_to_zero = True
) one needs to make sure that the distribution fits to censored data, knows the correctcensoring_threshold
and assumes all observations under the specified censoring_threshold are zero/censored.
If the standard for_precipitation and from_variable methods are used to construct the class this is ensured by default. However if this parameter is changed manually or own distributions for precipitation are specified problems can arise.
 running_window_modebool
Whether QuantileDeltaMapping is used in running window over the year to account for seasonality. If
running_window_mode = False
then QuantileDeltaMapping is applied on the whole period. Default:True
. running_window_lengthint
Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if
running_window_mode = True
. Default:91
. running_window_step_lengthint
Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if
running_window_mode = True
. Default:31
. running_window_mode_over_years_of_cm_futurebool
Controls whether the methodology is applied on a running time window, running over the years of cm_fut to calculate time dependent quantiles in future climate model values.
 running_window_over_years_of_cm_future_lengthint
Length of the time window centered around t to calculate time dependent quantiles in future climate model values (default: 31 years). Only relevant if
running_window_mode_over_years_of_cm_future = True
. running_window_over_years_of_cm_future_step_lengthint
Step length of the time window centered around t to calculate time dependent quantiles in future climate model values (default: 1 year). Only relevant if
running_window_mode_over_years_of_cm_future = True
.
 variablestr
Variable for which the debiasing is done. Default:
"unknown"
.
 ecdf_methodstr
One of
["kernel_density", "linear_interpolation", "step_function"]
. Method used to calculate the empirical CDF. Default:"linear_interpolation"
. cdf_thresholdOptional[float]
Threshold for the CDFvalues to round away from 0 and 1. Default: None. It is then set to
1 / (self.running_window_within_year_length * self.running_window_over_years_of_cm_future_length + 1)
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
for_precipitation
([censoring_threshold])Instanciates the class to a precipitationdebiaser.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 classmethod for_precipitation(censoring_threshold=5.787037037037037e07, **kwargs)
Instanciates the class to a precipitationdebiaser.
 Parameters:
 censoring_threshold: float
The censoringvalue under which precipitation amounts are assumed zero/censored.
 **kwargs:
All other class attributes that shall be set and where the standard values shall be overwritten.
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.
ibicus.debias ISIMIP class
 class ibicus.debias.ISIMIP(trend_preservation_method, distribution, nonparametric_qm, detrending, lower_bound=inf, lower_threshold=inf, upper_bound=inf, upper_threshold=inf, scale_by_annual_cycle_of_upper_bounds=False, window_length_annual_cycle_of_upper_bounds=31, impute_missing_values=False, detrending_with_significance_test=True, trend_transfer_only_for_values_within_threshold=True, bias_correct_frequencies_of_values_beyond_thresholds=True, event_likelihood_adjustment=False, ks_test_for_goodness_of_cdf_fit=True, ecdf_method='linear_interpolation', iecdf_method='linear', mode_non_parametric_qm='normal', running_window_mode=True, running_window_length=31, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)
Implements the ISIMIP3b and ISIMIP3BASD bias adjustment methodology based on Lange 2019 and Lange 2021.ISIMIP is a semiparametric quantile mapping method that attempts to be trendpreserving in all quantiles by generating ‘pseudo future observations’ and executing the quantile mapping between the future climate model and the pseudo future observations. ISIMIP includes special cases for several of the variables, and for a complete description of the methodology we refer to the ISIMIP documentation.
This notebook demonstrates that the results obtained with this implementation of ISIMIP are consistent with the reference implementation.
ISIMIP covers the following variables:
["hurs", "pr", "prsnratio", "psl", "rsds", "rlds", "sfcwind", "tas", "tasrange", "taskew"]
The variables tasmin, tasmax and prsn can be derived as:
tasrange = tasmin  tasmax, tasskew = (tastasmin)/(tasmaxtasmin)
andprsnratio = prsn/pr
. The ISIMIP methodology consists of 8 steps that cover the special cases and particularities for the different variables. We refer to Lange 2019 for a complete description. In the following the steps shall only be sketched. Let obs refer to the observation dataset and cm_hist and cm_fut to historical and future climate model values.Steps:
step1()
: (For rsds only). Ifscale_by_annual_cycle_of_upper_bounds = True
then an annual cycle of upper bounds is calculated to scale the values in obs, cm_hist and cm_fut onto the interval [0,1]. This is to account physical upper bounds that vary over the year, while other variables have static upper and lower bounds.
step2()
: (For prsnratio only). Ifimpute_missing_values = True
missing values (passed in either as masked array or array with np.nan values) in obs, cm_hist and cm_fut are imputed via a somewhat trend preserving sampling from all other nonmissing values within a given time window.
step3()
: (For tas, psl and rlds). Ifdetrending = True
the timeseries are detrended to account for trends within historical and future period (not for trends between historical and future one as this is done in step 5). Trends are estimated via linear regression on the annual means and then removed from the daily observations. Ifdetrending_with_significance_test = True
then a significance test is first used to determine whether the trend is to be removed.
step4()
: If the variable has either a lower or upper bound and threshold, values beyond the threshold are randomized within between the respective bound and threshold. This allows to quantile map these values in step 6. Randomization is done uniformly (different than from Lange 2019 to reflect a change in ISIMIP3b v2.2), however ranks are preserved.
step5()
: Here pseudo future observations are generated by transferring a climate change signal onto the observations. This is done differently depending on the trend_preservation_method: one of["additive", "multiplicative", "mixed", "bounded"]
to reflect differences in the trends of different variables. For bounded variables iftrend_transfer_only_for_values_within_threshold = True
then trend transfer only happens for the values within thresholds, else for all values.
step6()
: Given the pseudo future observations from step 5 a quantile mapping is used mapping the values of cm_fut to the pseudo future observations to generate debiased future climate model values.If
event_likelihood_adjustment = True
also an event likelihood adjustment is done, following Switanek et al. 2017 and as originally described in Lange 2019, but removed in the ISIMIP3b run.By default the quantile mapping is done parametically using the distribution in self.distribution, however if
nonparametric_qm = True
orks_test_for_goodness_of_cdf_fit = True
and a Kolmogorov Smirnov statistic indicats a bigger misfit a nonparametric quantile mapping is used instead (default for hurs, psrnratio, rsds, tasskew).If
bias_correct_frequencies_of_values_beyond_thresholds = True
(default) also the frequencies of values beyond thresholds are adjusted to. This adjusts eg. the number of precipitation dry days.
step7()
: Ifdetrending = True
then the trend removed in step 3 is restored onto the debiased cm_fut.
step8()
: (For rsds only). Ifscale_by_annual_cycle_of_upper_bounds = True
then a debiased annual cycle of upper bounds is calculated to rescale the debiased climate model values from [0,1] back to the original scale.
As for all other debiasers all steps are applied locationwise and if
running_window_mode = True
(default) in a running window with step sizerunning_window_step_length
and lengthrunning_window_length
. Otherwise steps are applied on a month by month basis to account for seasonality.Warning
Currently only uneven sizes are allowed for window length and window step length. This allows symmetrical windows of the form [window_center  window length//2, window_center + window length//2] given an arbitrary window center.
All variables have different default settings and using the
ISIMIP.from_variable(variable, **kwargs)
method it is possible to initialize a debiaser with these settings eg.>>> debiaser = ISIMIP.from_variable("tas")
However depending on the debiasing region and usecase at hand it might be important to modify some of the default settings, such as for example:
>>> debiaser = ISIMIP.from_variable("tas", detrending = False)
or
>>> debiaser = ISIMIP.from_variable("tas") >>> debiaser.detrending = False
If run with the default options, the results generated in this package are exactly identical to v3.0.1 (20220627) of the ISIMIP3BASD reference implementation, as shown in the notebook 05 ISIMIP Consistency. By setting parameters appropriately reproducing lower versions is possible and also consistency with future versions is considered.
In contrast to the reference implementation implementation of ISIMIP this code includes further options for customization of the ISIMIP behavior, allows more flexible usage given that it operates on a numerical basis and handles a variety of cases that might lead to bugs in the reference implementation.
References:
Lange, S. (2019). Trendpreserving bias adjustment and statistical downscaling with ISIMIP3BASD (v1.0). In Geoscientific Model Development (Vol. 12, Issue 7, pp. 3055–3070). Copernicus GmbH. https://doi.org/10.5194/gmd1230552019
Lange, S. (2022). ISIMIP3BASD (3.0.1) [Computer software]. Zenodo. https://doi.org/10.5281/ZENODO.6758997
Switanek, M. B., Troch, P. A., Castro, C. L., Leuprecht, A., Chang, H.I., Mukherjee, R., & Demaria, E. M. C. (2017). Scaled distribution mapping: a bias correction method that preserves raw climate model projected changes. In Hydrology and Earth System Sciences (Vol. 21, Issue 6, pp. 2649–2666). Copernicus GmbH. https://doi.org/10.5194/hess2126492017.
Usage information:Default settings exist for:
["hurs", "pr", "prsnratio", "psl", "rlds", "rsds", "sfcWind", "tas", "tasrange", "tasskew"]
.apply()
requires: time argumentstime_obs
,time_cm_hist
, andtime_cm_future
next toobs
,cm_hist
andcm_future
. These are just 1d numpyarrays of dates (multiple formats are possible as long as they as convertible to numpy or datetime dates) specifying the date for each value/timestep inobs
,cm_hist
andcm_future
. If they are not specified they are inferred, assuming the first observation in all three observation/climate value arrays is on a 1st of January.The debiaser assumes daily data.
Examples:Running without dates (they are inferred assuming the first value in
obs
,cm_hist
andcm_future
always corresponds to a January 1st):>>> debiaser = ISIMIP.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future)
Running with dates:
>>> debiaser = ISIMIP.from_variable("tas") >>> debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)
Core Parameters: Parameters:
 trend_preservation_methodstr
One of
["additive", "multiplicative", "mixed", "bounded"]
. Method of how the climate change trend is transferred to observations to generate the pseudo future observations. Depends on the kind of climate change trend expected in the variable. distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]
Distribution or statistical model used for the quantile mapping in step 6 if
nonparametric_qm = False
.
Usually a distribution inscipy.stats.rv_continuous
, but can also be an empirical distribution as given byscipy.stats.rv_histogram
or a more complex statistical model as wrapped by theibicus.utils.StatisticalModel
class. nonparametric_qmbool
Whether nonparametric quantile mapping is used for debiasing in step 6. If
nonparametric_qm = False
then quantile mapping is done using the distribution in self.distribution. detrendingbool
Whether detrending is applied in step4 to account for within obs, cm_hist and cm_fut trends.
Full list of attributes:
 Attributes:
 lower_boundfloat
Variable specific: Lower bound of a variable. Default:
np.inf
. lower_thresholdfloat
Variable specific: Lower threshold of a variable. Default:
np.inf
. upper_boundfloat
Variable specific: Upper bound of a variable. Default:
np.inf
. upper_thresholdfloat
Variable specific: Upper threshold of a variable. Default:
np.inf
. scale_by_annual_cycle_of_upper_bounds**bool
Step 1: Whether in step 1 a scaling by an estimated annual cycle of upper bounds is done to scale obs, cm_hist and cm_fut values onto [0,1]. Default:
False
. window_length_annual_cycle_of_upper_bounds**int
Step 1: Window length to estimate an annual cycle of upper bounds to rescale. Only relevant if
scale_by_annual_cycle_of_upper_bounds = True
. Default:31
. impute_missing_valuesbool
Step 2: Whether in step 2 missing values (indicated either by a mask of
np.nan
values) are imputed using the other values within the window. detrendingbool
Step 3: Whether detrending is applied in step4 to account for within obs, cm_hist and cm_fut trends.
 detrending_with_significance_testbool
Step 3: Whether detrending is applied conditional on a significance test for the trend. Only relevant if
detrending = True
. Default:True
. trend_preservation_methodstr
Step 5: One of
["additive", "multiplicative", "mixed", "bounded"]
. Method of how the climate change trend is transferred to observations to generate pseudo future ones. trend_transfer_only_for_values_within_thresholdbool
Step 5: Whether the trend transfer is done only for the values within thresholds. Default:
True
. distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]
Step 6: Distribution or statistical model used for the quantile mapping in step 6 if
nonparametric_qm = False
.
Usually a distribution inscipy.stats.rv_continuous
, but can also be an empirical distribution as given byscipy.stats.rv_histogram
or a more complex statistical model as wrapped by theibicus.utils.StatisticalModel
class. nonparametric_qmbool
Step 6: Whether nonparametric quantile mapping is used for debiasing in step 6. If
nonparametric_qm = False
then quantile mapping is done using the distribution indistribution
. bias_correct_frequencies_of_values_beyond_thresholdsbool
Step 6: Whether frequencies of values beyond thresholds are adjusted/bias corrected. Default:
True
. event_likelihood_adjustmentbool
Step 6: Whether in conjuction with quantile mapping an event likelihood adjustment is used inspired by Switanek et al. 2017. Default:
False
. ks_test_for_goodness_of_cdf_fitbool
Step 6: Whether a Kolmogorov Smirnov statistic is used to diagnose misfit of the parametric CDF prior to quantile mapping. If such misfit is diagnosed nonparametric quantile mapping is used. Only relevant if
nonparametric_qm = False
. Default:True
. ecdf_methodstr
Computation: One of
["kernel_density", "linear_interpolation", "step_function"]
. Method to calculate the empirical CDF. Default:"linear_interpolation"
. iecdf_methodstr
Computation: One of
["inverted_cdf", "averaged_inverted_cdf", closest_observation", "interpolated_inverted_cdf", "hazen", "weibull", "linear", "median_unbiased", "normal_unbiased"]
. Method to calculate the inverse empirical CDF (empirical quantile function). Default:"linear"
. mode_non_parametric_qmstr
Computation: One of
["normal", "isimipv3.0"]
. Computation method used for nonparametric quantile mapping. If"normal"
this is based on estimated empirical CDFs and inverse empirical CDFs. If"isimipv3.0"
this is based on estimated ranks usingscipy.stats.rankdata
which can lead to inconsistencies. Default:"normal"
. running_window_modebool
Iteration: Whether ISIMIP is used in running window mode to account for seasonalities. If
running_window_mode = False
then ISIMIP is applied by months. Default:True
. running_window_lengthint
Iteration: Length of the running window in days: how many values are used to the debiased climate model values. Only relevant if
running_window_mode = True
. Default:31
. running_window_step_lengthint
Iteration: Step length of the running window in days: how many values are debiased inside the running window. Only relevant if
running_window_mode = True
. Default:1
. variablestr
Variable for which the debiasing is done. Default:
"unknown"
.
Methods
apply
(obs, cm_hist, cm_future[, ...])Applies the debiaser onto given data.
from_variable
(variable, **kwargs)Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object.step1
(obs_hist, cm_hist, cm_future, ...)Step 1: if
scale_by_annual_cycle_of_upper_bounds = True
(rsds
only).step2
(obs_hist, cm_hist, cm_future)Step 2: impute values for
prsnratio
which are missing on days where there is no precipitation.step3
(obs_hist, cm_hist, cm_future, ...)Step 3: Linear trend removal if
detrending = True
.step4
(obs_hist, cm_hist, cm_future)Step4: If the variable is bounded then values between the threshold and corresponding bound (including values equal to the bound) are randomized uniformly between the threshold and bound and resorted according to the order in which they were before.
step5
(obs_hist, cm_hist, cm_future)Step 5: generates pseudofuture observations by transfering simulated trends to historical recorded observations.
step6
(obs_hist, obs_future, cm_hist, cm_future)Step 6: parametric quantile mapping between cm_future and obs_future (the pseudofuture observations) to debias the former.
step7
(cm_future, trend_cm_future)Step 7: If
detrending = True
adds the trend removed in step 3 back to the debiased cm_future values.step8
(cm_future, debiased_annual_cycle, ...)Step 8: if
scale_by_annual_cycle_of_upper_bounds = True
(rsds only). classmethod from_variable(variable, **kwargs)
Instanciates the class from a variable: either a string referring to a standard variable name or a
Variable
object. Parameters:
 variableUnion[str, Variable]
String or Variable object referring to standard meteorological variable for which default settings can be used.
 **kwargs:
All other class attributes that shall be set and where the standard values for variable shall be overwritten.
 Returns:
 Debiaser
Instance of the class for the given variable.
 step1(obs_hist, cm_hist, cm_future, time_obs_hist, time_cm_hist, time_cm_future)
Step 1: if
scale_by_annual_cycle_of_upper_bounds = True
(rsds
only). Scales obs, cm_hist and cm_future values to [0, 1] by computing an annual cycle of upper bounds as “running mean values of running maximum values of multiyear daily maximum values.” (Lange 2019). Furthermore return a debiased annual cycle of upper bounds to rescale cm_future values in step8.
 step2(obs_hist, cm_hist, cm_future)
Step 2: impute values for
prsnratio
which are missing on days where there is no precipitation. They are imputed by effectively sampling the iecdf (see Lange 2019 and ISIMIP3b factsheet for the method).
 step3(obs_hist, cm_hist, cm_future, years_obs_hist, years_cm_hist, years_cm_future)
Step 3: Linear trend removal if
detrending = True
. This is because certain variables (eg. temp) can have substantial trends also within training and application period (not only between). These trends are removed to “prevent a confusion of these trends with interannual variability during quantile mapping (steps 5 and6)” (Lange 2019). The trend for cm_future is subsequently added again in step7. Trends are calculated by linearly regressing the yearly mean values y_i against years t_i. From each observation in year t_i then the slope * (t_i  mean(t_i)) is removed (normalising the trend such that the sum over all t_i is zero). See Lange 2019 for the exact method.If
detrending_with_significance_test = True
(default in ISIMIP v2.5) then linear trends are only removed subject it being significant (pvalue < 0.05).
 step4(obs_hist, cm_hist, cm_future)
Step4: If the variable is bounded then values between the threshold and corresponding bound (including values equal to the bound) are randomized uniformly between the threshold and bound and resorted according to the order in which they were before.
 step5(obs_hist, cm_hist, cm_future)
Step 5: generates pseudofuture observations by transfering simulated trends to historical recorded observations. This makes the ISIMIPmethod trendpreserving.
 step6(obs_hist, obs_future, cm_hist, cm_future)
Step 6: parametric quantile mapping between cm_future and obs_future (the pseudofuture observations) to debias the former. Core of the bias adjustment method. For (partly) bounded climate variables additionally the frequency of values is biasadjusted before the other observations are quantile mapped. If
event_likelihood_adjustment = True
then additionally to “normal quantile mapping” the likelihood of individual events is adjusted (see Lange 2019 for the method which is based on Switanek 2017).
 step7(cm_future, trend_cm_future)
Step 7: If
detrending = True
adds the trend removed in step 3 back to the debiased cm_future values.
 step8(cm_future, debiased_annual_cycle, time_cm_future)
Step 8: if
scale_by_annual_cycle_of_upper_bounds = True
(rsds only). Scales debiased cm_future on [0, 1] back to the usual scale using the debiased annual cycle
 apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)
Applies the debiaser onto given data.
 Parameters:
 obsnp.ndarray
3dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.
 cm_histnp.ndarray
3dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 cm_futurenp.ndarray
3dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.
 progressbarbool = False
Whether a progressbar shall be shown to indicate the debiaser status. Default:
True
. parallelbool = False
Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default:
False
. nr_processesint = 4
Number of processes for parallel code execution. Default: 4.
 failsafebool = False
Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns
np.nan
at this location. Default:False
.
 Returns:
 np.ndarray
3dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.