ibicus.debias module

The debias-module provides the necessary functionality to bias correct climate models.

The foundation of the module is the Debiaser class from which every debiaser inherits and which provides a unified interface to instantiate and apply debiasers.

The following bias correction methodologies are currently implemented in the package, each based on the respective publication cited.

Debiaser(*[, variable, ...])	A generic debiaser meant for subclassing.
LinearScaling(delta_type, *[, variable, ...])	Implements debiasing via linear scaling based on Maraun 2016.
DeltaChange(delta_type[, ...])	Implements 'delta change' method based on Maraun 2016 as reference.
QuantileMapping([distribution, ...])	Implements (detrended) Quantile Mapping based on Cannon et al. 2015 and Maraun 2016.
ScaledDistributionMapping(distribution, ...)	Implements Scaled Distribution Matching (SDM) based on Switanek et al. 2017.
CDFt([SSR, delta_shift, apply_by_month, ...])	Implements CDF-t based on Michelangeli et al. 2009, Vrac et al. 2012 and Famien et al. 2018, as well as Vrac et al. 2016 for precipitation.
ECDFM(distribution[, cdf_threshold, ...])	Implements Equidistant CDF Matching (ECDFM) based on Li et al. 2010.
QuantileDeltaMapping(distribution, ...[, ...])	Implements Quantile Delta Mapping based on Cannon et al. 2015.
ISIMIP(trend_preservation_method, ...[, ...])	Implements the ISIMIP3b and ISIMIP3BASD bias adjustment methodology based on Lange 2019 and Lange 2021.

Methodology

For a brief introduction to bias correction, and some issues to pay attention to when applying a bias correction method, have a look at the ‘What is bias adjustment?’ page.

The general idea behind bias correction is to calibrate an empirical transfer function between simulated and observed distributional parameters that bias adjust the climate model output. This can be done in a number of different ways. This table in ‘Overview’ provides an overview of the different methodological choices made by the bias correction methods implemented in the package. For a detailed description of their methodology, we refer you to the class descriptions and the cited publications.

Usage

Three types of data are required in order to conduct bias correction for a given climatic variable:

1. Observations / reanalysis data for given historical period: obs.

2. Climate model simulation for same historical period as observations: cm_hist.

3. Climate model simulation for the period that is to be bias corrected, often a future period: cm_future.

Let’s generate some pseudo climate data:

>>> import numpy as np
>>> np.random.seed(12345)
>>> obs, cm_hist, cm_future = np.random.normal(loc = 3, size = 40000).reshape((10000, 2, 2)), np.random.normal(loc = 5, size = 40000).reshape((10000, 2, 2)), np.random.normal(loc = 7, size = 40000).reshape((10000, 2, 2))

Every debiaser can be instatiated using from_variable() and a standard abbrevation for a meteorological variable following the CMIP-convention:

>>> debiaser = CDFt.from_variable("tas")

This instantiates a debiaser with default settings for "tas" (daily mean 2m air surface temperature (K)).

The following code the applies this debiaser, given the data obs, cm_hist and cm_future.

>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)

Variable support

Variables currently supported across debiasers include:

["hurs", "pr", "prsnratio", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax", "tasrange", "tasskew"]

However, whereas some bias correction methods such as ISIMIP have explicitly been published and implemented for all these variables (tasmin and tasmax are not explicitely debiased but can be calculated from tas, tasrange and taskew), other methods have been published only for specific variables such as precipitation or temperature. Where possible, the authors have introduced informed choices for the default settings of other variables as well.

The following table provides an overview of which debiasers currently have which default settings for which variables. Crosses in brackets signify so-called ‘experimental default settings’ that have been chosen by the creators of this package and may not have been evaluated by the peer reviewed literature. It is advised to evaluate those carefully.

Variable	LinearScaling	DeltaChange	QuantileMapping	ScaledDistributionMapping	CDFt	ECDFM	QuantileDeltaMapping	ISIMIP
hurs	(x)	(x)	(x)		(x)	(x)	(x)	x
pr	x	x	x	x	x	x	x	x
prsnratio								x
psl	(x)	(x)	(x)		(x)	(x)	(x)	x
rlds	(x)	(x)	(x)		(x)	(x)	(x)	x
rsds	(x)	(x)			(x)			x
sfcWind	(x)	(x)	(x)		(x)	(x)	(x)	x
tas	x	x	x	x	x	x	x	x
tasmin	x	x	(x)	(x)	x	(x)	(x)
tasmax	x	x	(x)	(x)	x	(x)	(x)
tasrange					(x)			x
tasskew					(x)			x

Note

A warning message is shown for variable-debiaser combinations that are still experimental.

Setting and Modifying parameters

In addition to these default setting settings, the user can also modify the settings and parameters of each debiaser. In particular for those default settings that are still experimental, it is highly recommended to try out some alternatives. This is possible either by setting alternative settings in from_variable() or modifying the object attribute:

>>> debiaser = CDFt.from_variable("tas", delta_shift = "no_shift")

or:

>>> debiaser = CDFt.from_variable("tas")
>>> debiaser.delta_shift = "no_shift"

It is also possible to instantiate debiasers by directly setting the necessary parameters, bypassing from_variable():

>>> debiaser1 = CDFt()
>>> from scipy.stats import norm
>>> debiaser2 = QuantileMapping(distribution = norm, detrending = "none")

Some debiasers additionally provide a for_precipitation() classmethod to help you initialise the debiaser for precipitation. Debiasing precipitation often requires setting additional arguments like a threshold under which precipitation is assumed zero and the for_precipitation() method helps with that:

>>> debiaser = QuantileMapping.for_precipitation(model_type = "hurdle")
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)

The documentation of the individual debiasers provides some information on this.

Some debiasers eg. ISIMIP require date information to be applied in a running window. Dates should then be passed as 1d numpy arrays as keyword arguments:

>>> debiaser = ISIMIP.from_variable("tas")
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)

Whenever date information is needed this is indicated in the debiaser documentation.

When applying the debiaser we can parallelise the execution by setting parallel = True and setting the nr_processes (default: 4). It is also possible to activate/deactive the progressbar using the progressbar argument and to activate a failsafe mode failsafe = True, continuing execution when encoutnering errors:

>>> debiased_cm_future1 = debiaser1.apply(obs, cm_hist, cm_future, parallel = True, failsafe = True)

---

ibicus.debias Debiaser abstract-class

class ibicus.debias.Debiaser(*, variable='unknown', reasonable_physical_range=None)

A generic debiaser meant for subclassing. Provides functionality for individual debiasers and a unified interface to apply bias adjustment.

The debiaser abstract class provides a unified interface to call the debiaser, as well as a vaeriety of setup tasks and input-checks. In order to subclass the Debiaser-class, the proposed debiaser needs to implement the from_variable() and apply_location() functions:

• apply_location(): applies an initialised debiaser at one location. Arguments are 1d-vectors of obs, cm_hist, and cm_future representing observations, and climate model values during the reference (cm_hist) and future period (cm_future). Additionally kwargs passed to the debiaser apply()-function are passed down to the apply_location()-function.

• from_variable(): initialises a debiaser with default arguments given a climatic variable either as str or member of the Variable-class. kwargs are meant to overwrite default arguments for this variable. Given a dict of default arguments: with variables of the Variable class as keys and dict of default arguments as values the _from_variable()-function can be used.

The apply() function, maps the debiaser apply_location() over locations. This allows to always initialise and apply debiasers follows:

>>> debiaser = LinearScaling.from_variable("tas") # LinearScaling is a child-class of Debiaser
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)

or:

>>> debiaser = CDFt.from_variable("pr", delta_shift="no_shift") # CDFt is a child-class of Debiaser
>>> debiased_cm_future = debiaser.apply(obs, cm_hist, cm_future)

Attributes:

variablestr

Variable that is meant to be debiased, by an initialisation of the debiaser. Default: "unknown".

reasonable_physical_rangeOptional[list]

Reasonable physical range of the variable to debias in the form [lower_bound, upper_bound]. It is checked against and warnings are raise if values fall outside the range. Default: None.

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
apply_location(obs, cm_hist, cm_future, **kwargs)	Applies the debiaser at one location.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

abstract classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

abstract apply_location(obs, cm_hist, cm_future, **kwargs)

Applies the debiaser at one location.

Parameters:

obsnp.ndarray

1-dimensional numpy array of observations of the meteorological variable at one location.

cm_histnp.ndarray

1-dimensional numpy array of values of the historical climate model run (run during the same or a similar period as observations) at one location.

cm_futurenp.ndarray

1-dimensional numpy array of values of a climate model to debias (future run) at one location.

Returns:

np.ndarray

1-dimensional numpy array containing the debiased climate model values for the future run (cm_future).

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias RunningWindowDebiaser abstract-class

ibicus.debias LinearScaling class

class ibicus.debias.LinearScaling(delta_type, *, variable='unknown', reasonable_physical_range=None, running_window_mode=False, running_window_length=31, running_window_step_length=1)

Implements debiasing via linear scaling based on Maraun 2016.

Linear scaling corrects a climate model by the difference in the mean of observations and the mean of the climate model on the reference period, either additively or multiplicatively.

The present day model bias is calculated and then either subtracted or divided from the future climate model data.

Let \(x_{\text{obs}}\) be the observed timeseries \(x_{\text{cm_hist}}\) the simulated historical one and \(x_{\text{cm_fut}}\) the simulated future one (climate model historical and future run). Then additive linear scaling adjusts \(x_{\text{cm_fut}}\) as follows:

\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} - (\bar x_{\text{cm_hist}} - \bar x_{\text{obs}})\]

and multiplicative scaling:

\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} \cdot \frac{\bar x_{\text{obs}}}{\bar x_{\text{cm_hist}}}.\]

Here \(\bar x\) stands for the mean over all x-values.

Multiplicative scaling is classically used for precipitation (pr) and additive scaling for temperature (tas). Additive scaling amounts to a simple mean bias correction, whilst multiplicative one adjusts both mean and variance, but keeps their ration constant (Maraun 2016).

References:

• Maraun, D. Bias Correcting Climate Change Simulations - a Critical Review. Curr Clim Change Rep 2, 211–220 (2016). https://doi.org/10.1007/s40641-016-0050-x

Usage information:

• Default settings exist for: ["hurs", "pr", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax"].

• apply() requires: no additional arguments except obs, cm_hist, cm_future.

• The debiaser works with data in any time specification (daily, monthly, etc.).

Examples:

>>> debiaser = DeltaChange.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Attributes:

delta_typestr

One of ["additive", "multiplicative"]. Determines whether additive or multiplicative scaling is used.

running_window_modebool

Whether LinearScaling is used in running window over the year to account for seasonality. If running_window_mode = False then LinearScaling is applied on the whole period. Default: False.

running_window_lengthint

Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True. Default: 31.

running_window_step_lengthint

Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if running_window_mode = True. Default: 1.

variablestr

Variable for which the debiasing is used

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias DeltaChange class

class ibicus.debias.DeltaChange(delta_type, running_window_mode=False, running_window_length=31, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)

Implements ‘delta change’ method based on Maraun 2016 as reference.

This is technically not a bias correction method because the future climate model output is not transformed. Instead, the delta change method applies the climate change trend from the model to historical observations, therefore generating modified observations rather than a modified model output. So the output by apply() from this method has the same number of timesteps as the obs data, and not the same number as cm_fut like other debiasers.

Let \(x_{\text{obs}}\) be the observed timeseries \(x_{\text{cm_hist}}\) the simulated historical one and \(x_{\text{cm_fut}}\) the simulated future one (climate model historical and future run). For an additive change a future timeseries is generated as:

\[x_{\text{obs}} + (\bar x_{\text{cm_fut}} - \bar x_{\text{cm_hist}})\]

and for multiplicative change:

\[x_{\text{obs}} \cdot \frac{\bar x_{\text{cm_fut}}}{\bar x_{\text{cm_hist}}}.\]

Here \(\bar x\) stands for the mean over all x-values.

Multiplicative change is typically used for precipitation and additive scaling for temperature.

References:

• Maraun, D. Bias Correcting Climate Change Simulations - a Critical Review. Curr Clim Change Rep 2, 211–220 (2016). https://doi.org/10.1007/s40641-016-0050-x

Usage information:

• Default settings exist for: ["hurs", "pr", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax"].

• apply() requires: no additional arguments except obs, cm_hist, cm_future.

• The debiaser works with data in any time specification (daily, monthly, etc.).

Examples:

>>> debiaser = DeltaChange.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Attributes:

delta_typestr

One of ["additive", "multiplicative"]. Determines whether additive or multiplicative scaling is used.

running_window_modebool

Whether DeltaChange is used in running window over the year to account for seasonality. If running_window_mode = False then DeltaChange is applied on the whole period. Default: False.

running_window_lengthint

Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True. Default: 31.

running_window_step_lengthint

Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if running_window_mode = True. Default: 1.

variablestr

Variable for which the debiasing is used

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias QuantileMapping class

class ibicus.debias.QuantileMapping(distribution=None, mapping_type='nonparametric', detrending='no_detrending', cdf_threshold=1e-10, *, variable='unknown', reasonable_physical_range=None, running_window_mode=False, running_window_length=31, running_window_step_length=1)

Implements (detrended) Quantile Mapping based on Cannon et al. 2015 and Maraun 2016.

(Parametric) quantile mapping maps every quantile of the climate model distribution to the corresponding quantile in observations during the reference period. Optionally, additive or multiplicative detrending of the mean can be applied to make the method trend preserving in the mean. Most methods build on quantile mapping.

Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. Let \(F\) be a CDF. The future climate projections \(x_{\text{cm_fut}}\) are then mapped using a QQ-mapping between \(F_{\text{cm_hist}}\) and \(F_{\text{obs}}\), so:

\[x_{\text{cm_fut}} \rightarrow F^{-1}_{\text{obs}}(F_{\text{cm_hist}}(x_{\text{cm_fut}}))\]

If detrended quantile mapping is used then \(x_{\text{cm_fut}}\) is first rescaled and then the mapped value is scaled back either additively or multiplicatively. That means for additive detrending:

\[x_{\text{cm_fut}} \rightarrow F^{-1}_{\text{obs}}(F_{\text{cm_hist}}(x_{\text{cm_fut}} + \bar x_{\text{cm_hist}} - \bar x_{\text{cm_fut}})) + \bar x_{\text{cm_fut}} - \bar x_{\text{cm_hist}}\]

and for multiplicative detrending.

\[x_{\text{cm_fut}} \rightarrow F^{-1}_{\text{obs}}\left(F_{\text{cm_hist}}\left(x_{\text{cm_fut}} \cdot \frac{\bar x_{\text{cm_hist}}}{\bar x_{\text{cm_fut}}}\right)\right) \cdot \frac{\bar x_{\text{cm_fut}}}{\bar x_{\text{cm_hist}}}\]

Here \(\bar x_{\text{cm_fut}}\) designs the mean of \(x_{\text{cm_fut}}\) and similar for \(x_{\text{cm_hist}}\). Detrended Quantile Mapping accounts for changes in the projected values and is thus trend-preserving in the mean.

For precipitation a distribution or model is needed that accounts for the mixed zero and positive value character. Default is a precipitation hurdle model (see ibicus.utils.gen_PrecipitationHurdleModel). However, other models are also possible, for_precipitation() helps with the initialisation of different precipitation methods.

References:

• Cannon, A. J., Sobie, S. R., & Murdock, T. Q. (2015). Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? In Journal of Climate (Vol. 28, Issue 17, pp. 6938–6959). American Meteorological Society. https://doi.org/10.1175/jcli-d-14-00754.1

• Maraun, D. (2016). Bias Correcting Climate Change Simulations - a Critical Review. In Current Climate Change Reports (Vol. 2, Issue 4, pp. 211–220). Springer Science and Business Media LLC. https://doi.org/10.1007/s40641-016-0050-x

Usage information:

• Default settings exist for: ["hurs", "pr", "psl", "rlds", "sfcWind", "tas", "tasmin", "tasmax"].

• apply() requires: no additional arguments except obs, cm_hist, cm_future.

• Next to from_variable() a for_precipitation()-method exists to help you initialise the debiaser for pr.

• The debiaser works with data in any time specification (daily, monthly, etc.), although some of the default distributions have the best fit to daily data.

Examples:

Initialising using from_variable:

>>> debiaser = QuantileMapping.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Initialising using for_precipitation:

>>> debiaser = QuantileMapping.for_precipitation(model_type = "hurdle")
>>> debiaser.apply(obs, cm_hist, cm_future)

Attributes:

distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel, None]

Distribution or statistical model used to compute the CDFs F. Default: None. Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by the ibicus.utils.StatisticalModel class.

mapping_typestr

One of ["parametric", "nonparametric"]. Whether quantile mapping is done using parametric CDFs or using nonparametric density estimation (“empirical quantile mapping”). Default: nonparametric.

detrendingstr

One of ["additive", "multiplicative", "no_detrending"]. What kind of scaling is applied to the future climate model data before quantile mapping. Default: "no_detrending".

cdf_thresholdfloat

Threshold to round CDF-values away from zero and one. Default: 1e-10.

running_window_modebool

Whether QuantileMapping is used in running window over the year to account for seasonality. If running_window_mode = False then QuantileMapping is applied on the whole period. Default: False.

running_window_lengthint

Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True. Default: 31.

running_window_step_lengthint

Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if running_window_mode = True. Default: 1.

variablestr

Variable for which the debiasing is done. Default: "unknown".

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
for_precipitation([mapping_type, ...])	Instanciates the class to a precipitation-debiaser.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

classmethod for_precipitation(mapping_type='parametric', model_type='hurdle', amounts_distribution=<scipy.stats._continuous_distns.gamma_gen object>, censoring_threshold=1.1574074074074074e-06, hurdle_model_randomization=True, hurdle_model_kwds_for_distribution_fit={'floc': 0, 'fscale': None}, **kwargs)

Instanciates the class to a precipitation-debiaser. This allows granular setting of available precipitation models without needing to explicitly specify the precipitation censored model for example.

Parameters:

model_typestr

One of ["censored", "hurdle", "ignore_zeros"]. Model type to be used. See ibicus.utils.gen_PrecipitationGammaLeftCensoredModel, ibicus.utils.gen_PrecipitationHurdleModel and ibicus.utils.gen_PrecipitationIgnoreZeroValuesModel for more details.

amounts_distributionscipy.stats.rv_continuous

Distribution used for precipitation amounts. For the censored model only scipy.stats.gamma is possible.

censoring_thresholdfloat

The censoring-value if a censored precipitation model is used.

hurdle_model_randomizationbool

Whether when computing the cdf-values for a hurdle model randomization shall be used. See ibicus.utils.gen_PrecipitationHurdleModel for more details.

hurdle_model_kwds_for_distribution_fitdict

Dict of parameters used for the distribution fit inside a hurdle model. Default: location of distribution is fixed at zero (floc = 0) to stabilise Gamma distribution fits in scipy.

**kwargs:

All other class attributes that shall be set and where the standard values shall be overwritten.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias ScaledDistributionMapping class

class ibicus.debias.ScaledDistributionMapping(distribution, mapping_type, pr_lower_threshold=1.1574074074074074e-06, distribution_fit_kwargs={}, cdf_threshold=1e-10, running_window_mode=False, running_window_length=91, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)

Implements Scaled Distribution Matching (SDM) based on Switanek et al. 2017.

SDM is conceptually similar to QDM, and in the same ‘family’ as CDFt and ECDFM. It is a parametric quantile mapping approach that also attempts to be trend preserving in all quantiles. In addition to the quantile mapping the method also contains an event likelihood adjustment.

SDM scales the observed distribution by changes in magnitude and additionally likelihood of events – either multiplicatively (for precipitation) or additively (for temperature).

Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. Let \(F\) be a parametric CDF.

1. Temperature (tas): absolute scaled distribution mapping

First CDFs are fitted to both historical and future climate model values as well as observations. The default settings for tas use a normal distribution. Then the scaling is calculated as:

\[\text{scaling} = [F^{-1}_{\text{cm_fut}}(F_{\text{cm_fut}}(x_{\text{cm_fut}})) - F^{-1}_{\text{cm_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))] * \frac{\sigma_\text{obs}}{\sigma_\text{cm_hist}}\]

where \(\sigma_\text{obs}\) and \(\sigma_\text{cm_hist}\) refers to the standard deviation of a normal distribution fitted to obs and cm_hist. Given the CDFs for obs, cm_hist, cm_fut then recurrence intervals for all three are calculated as:

\[\text{RI} = \frac{1}{0.5 - \|CDF - 0.5\|}\]

and a scaled recurrence interval (RI) and CDF as:

\[\text{RI}_{\text{scaled}} = \text{max}\left(1, \frac{\text{RI}_{\text{obs}} \cdot \text{RI}_{\text{cm_fut}}}{\text{RI}_{\text{cm_hist}}}\right)\]

\[\text{CDF}_{\text{scaled}} = 0.5 + \text{sgn}(\text{CDF}_{\text{obs}} - 0.5) \cdot \left| 0.5 - \frac{1}{\text{RI}_{\text{scaled}}} \right|.\]

Then the adjusted values are given as follows:

\[F^{-1}_{\text{obs}}(\text{CDF}_{\text{scaled}}) + \text{scaling}\]

2. Precipitation (pr): relative scaled distribution mapping

For precipitation first in obs, cm_hist and cm_fut all values below a given threshold are set to zero rain. Let \(\text{# rain}\) design the number of rain days and \(\text{# total}\) the total number of days. The bias corrected number of rain days is calculated as:

\[\text{# rain}_{\text{bc}} = \text{# rain}_{\text{cm_fut}} \cdot \frac{\text{# rain}_{\text{obs}} / \text{# total}_{\text{obs}}}{\text{# rain}_{\text{cm_hist}} / \text{# rain}_{\text{cm_hist}}}\]

Using all values bigger than the threshold a scaling is calculated as:

\[\text{scaling} = \frac{F^{-1}_{\text{cm_fut}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))}{F^{-1}_{\text{cm_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))}\]

\(\text{RI}_{\text{scaled}}\) and \(\text{CDF}_{\text{scaled}}\) are calculated similarly as for temperature. The final bias corrected precipitation values on rainy days are then given as:

\[F^{-1}_{\text{obs}}(\text{CDF}_{\text{scaled}}) \cdot \text{scaling}\]

The \(\text{# total}_{\text{bc}} - \text{# rain}_{\text{bc}}\) days with the smallest precipitation values in cm_fut are then set to zero and all bias corrected rain values are inserted at the correct locations, starting with the biggest one.

Warning

The relative SDM method does not currently allow correcting the number of precipitation days in cm_fut upwards, so to convert dry into rainy days. Should the calculated expected number of rainy days be higher than what is given inside the future climate model then the number of rainy days is left unadjusted. The method focuses on the biggest precipitation values, so this should not be an issue for most applications. However if such a correction is required this method might not be appropriate.

Reference:

• Switanek, M. B., Troch, P. A., Castro, C. L., Leuprecht, A., Chang, H.-I., Mukherjee, R., & Demaria, E. M. C. (2017). Scaled distribution mapping: a bias correction method that preserves raw climate model projected changes. In Hydrology and Earth System Sciences (Vol. 21, Issue 6, pp. 2649–2666). Copernicus GmbH. https://doi.org/10.5194/hess-21-2649-2017.

Usage information:

• Default settings exist for: ["pr", "tas", "tasmin", "tasmax"].

• apply() requires: no additional arguments except obs, cm_hist, cm_future.

• Next to from_variable() a for_precipitation()-method exists to help you initialise the debiaser for pr.

• The method has been developed for daily data, however application on data in other time specifications (monthly etc.) is possible.

Examples:

>>> debiaser = ScaledDistributionMapping.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Attributes:

distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]

Method used for the fit to the historical and future climate model outputs as well as the observations. Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by the ibicus.utils.StatisticalModel class.

mapping_typestr

One of ["absolute", "relative"]. Type of SDM used. Default are “absolute” for tas and "relative" for pr.

pr_lower_thresholdfloat

Lower threshold used for setting precipitation values to zero in relative SDM. Only used if mapping_type = "relative".

distribution_fit_kwargsdict

Dict of additional arguments passed to the distribution.fit-method. Useful for fixing certain parameters of a distribution. Default: {} (empty dict).

cdf_thresholdfloat

Threshold to round CDF-values away from zero and one. Default: 1e-10.

running_window_modebool

Whether ScaledDistributionMapping is used in running window over the year to account for seasonality. If running_window_mode = False then ScaledDistributionMapping is applied on the whole period. Default: False.

running_window_lengthint

Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True. Default: 91.

running_window_step_lengthint

Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if running_window_mode = True. Default: 1.

variablestr

Variable for which the debiasing is done. Default: "unknown".

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias CDFt class

class ibicus.debias.CDFt(SSR=False, delta_shift='additive', apply_by_month=True, running_window_mode=True, running_window_length=31, running_window_step_length=31, running_window_mode_over_years_of_cm_future=True, running_window_over_years_of_cm_future_length=17, running_window_over_years_of_cm_future_step_length=9, ecdf_method='linear_interpolation', iecdf_method='linear', *, variable='unknown', reasonable_physical_range=None)

Implements CDF-t based on Michelangeli et al. 2009, Vrac et al. 2012 and Famien et al. 2018, as well as Vrac et al. 2016 for precipitation.

CDFt is a non-parametric quantile mapping method that attempts to be trend-preserving in all quantiles. CDFt applies a concatenation between a quantile mapping of future and historical climate model data and a quantile mapping of the future climate model with historical observations. It also includes a running window over the future period to account for changes in the simulated trend.

Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. In this methodology, all cdfs are estimated empirically. Let \(F\) therefore be an empirical cdf. The future climate projections \(x_{\text{cm_fut}}\) are then mapped using a QQ-mapping between \(F_{\text{cm_fut}}\) and \(F_{\text{obs_fut}}\), with:

\[F_{\text{obs_fut}} := F_{\text{obs_hist}}(F^{-1}_{\text{cm_hist}}(F_{\text{cm_fut}})).\]

This means that \(x_{\text{cm_fut}}\) is mapped using the following formula:

\[x_{\text{cm_fut}} \rightarrow F^{-1}_{\text{obs_fut}}(F_{\text{cm_fut}}(x_{\text{cm_fut}})) = F^{-1}_{\text{cm_fut}}(F_{\text{cm_hist}}(F^{-1}_{\text{obs_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))))\]

Because an empirical CDF will not be able to map values outside its fitted range, a delta shift is applied to the future and historical climate model prior to fitting empirical CDFs. This ensures that the data is approximately in the same range. This delta shift can be additive:

\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} + \bar x_{\text{obs}} - \bar x_{\text{cm_hist}}\]

\[x_{\text{cm_hist}} \rightarrow x_{\text{cm_hist}} + \bar x_{\text{obs}} - \bar x_{\text{cm_hist}}\]

or multiplicative:

\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} \cdot \frac{\bar x_{\text{obs}}}{\bar x_{\text{cm_hist}}}\]

\[x_{\text{cm_hist}} \rightarrow x_{\text{cm_hist}} \cdot \frac{\bar x_{\text{obs}}}{\bar x_{\text{cm_hist}}}\]

Here \(\bar x\) stands for the mean over all x-values.

After this shift by the absolute or relative mean bias between cm_hist and obs are applied to both cm_fut and cm_hist, the cm_fut values are mapped as shown above using the QQ-mapping between \(F_{\text{cm_fut}}\) and \(F_{\text{obs_fut}}\).

• If SSR = True then Stochastic Singularity Removal (SSR) based on Vrac et al. 2016 is used to correct the precipitation occurrence in addition to amounts (default setting for pr). All zero values are first replaced by uniform draws between 0 and a small threshold (the minimum positive value of observation and model data). Then CDFt-mapping is used and afterwards all observations under the threshold are set to zero again.

• If running_window_mode_over_years_of_cm_future = True (default) then the method is used in a running window mode, running over the values of the future climate model. This helps to smooth discontinuities.

• If running_window_mode_within_year = True (default) then the method is used in a running window mode, running over the year to account for seasonalities.

• If apply_by_month = True (default: False) then CDF-t uses a running window within the year (running_window_mode_within_year) with a window length and step length of 31, so broadly applies it by month following Famien et al. 2018 to take seasonality into account. If apply_by_month = False and running_window_mode_within_year = False the method is applied to the whole year.

Warning

Currently only uneven sizes are allowed for window length and window step length. This allows symmetrical windows of the form [window_center - window length//2, window_center + window length//2] given an arbitrary window center.

References:

• Michelangeli, P.-A., Vrac, M., & Loukos, H. (2009). Probabilistic downscaling approaches: Application to wind cumulative distribution functions. In Geophysical Research Letters (Vol. 36, Issue 11). American Geophysical Union (AGU). https://doi.org/10.1029/2009gl038401

• Famien, A. M., Janicot, S., Ochou, A. D., Vrac, M., Defrance, D., Sultan, B., & Noël, T. (2018). A bias-corrected CMIP5 dataset for Africa using the CDF-t method – a contribution to agricultural impact studies. In Earth System Dynamics (Vol. 9, Issue 1, pp. 313–338). Copernicus GmbH. https://doi.org/10.5194/esd-9-313-2018

• Vrac, M., Drobinski, P., Merlo, A., Herrmann, M., Lavaysse, C., Li, L., & Somot, S. (2012). Dynamical and statistical downscaling of the French Mediterranean climate: uncertainty assessment. In Natural Hazards and Earth System Sciences (Vol. 12, Issue 9, pp. 2769–2784). Copernicus GmbH. https://doi.org/10.5194/nhess-12-2769-2012

• Vrac, M., Noël, T., & Vautard, R. (2016). Bias correction of precipitation through Singularity Stochastic Removal: Because occurrences matter. In Journal of Geophysical Research: Atmospheres (Vol. 121, Issue 10, pp. 5237–5258). American Geophysical Union (AGU). https://doi.org/10.1002/2015jd024511

Usage information:

• Default settings exist for: ["hurs", "pr", "psl", "rlds", "rsds", "sfcWind", "tas", "tasmin", "tasmax", "tasrange", "tasskew"].

• apply() requires: time arguments time_obs, time_cm_hist, and time_cm_future next to obs, cm_hist and cm_future. These are just 1d numpy-arrays of dates (multiple formats are possible as long as they as convertible to numpy or datetime dates) specifying the date for each value/timestep in obs, cm_hist and cm_future. If they are not specified they are inferred, assuming the first observation in all three observation/climate value arrays is on a 1st of January.

• The debiaser has been developed for and assumes daily data, however application on data using other time specifications (monthly etc.) is possible by setting apply_by_month = False, modifying the running window arguments and specifying the time arguments in apply().

Examples:

Running without dates (they are inferred assuming the first value in obs, cm_hist and cm_future always corresponds to a January 1st):

>>> debiaser = CDFt.from_variable("pr")
>>> debiaser.apply(obs, cm_hist, cm_future)

Running with dates:

>>> debiaser = CDFt.from_variable("pr")
>>> debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)

Attributes:

SSRbool

If Stochastic Singularity Removal (SSR) following Vrac et al. 2016 is applied to adjust the number of zero values (only relevant for pr).

delta_shiftstr

One of ["additive", "multiplicative", "no_shift"]. Type of shift applied to the data prior to fitting empirical distributions.

running_window_modebool

Controls whether CDF-t is applied in a running window over the year to account for seasonality. Default: True.

running_window_lengthint

Length of the running window over the year in days (default: 31 days): the amount of days used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True.

running_window_step_lengthint

Step length of the running window over the year in days (default 31 days): the amount of days that are bias adjusted/how far the running window is moved. Only relevant if running_window_mode = True.

running_window_mode_over_years_of_cm_futurebool

Whether CDF-t is used in running window mode, running over the values of the future climate model to help smooth discontinuities. Default: True.

running_window_over_years_of_cm_future_lengthint

Length of the running window in years: how many values are used to calculate the empirical CDF. Only relevant if running_window_mode_over_years_of_cm_future = True. Default: 17.

running_window_over_years_of_cm_future_step_lengthint

Step length of the running window in years: how many values are bias adjusted inside the running window. Only relevant if running_window_mode_over_years_of_cm_future = True. Default: 9.

apply_by_monthbool

Whether CDF-t is applied month by month (default) to account for seasonality. This is equivalent to a running window within the year with length 31 and step length 31. Default: Faöse.

variablestr

Variable for which the debiasing is done. Default: "unknown".

ecdf_methodstr

One of ["kernel_density", "linear_interpolation", "step_function"]. Method to calculate the empirical CDF. Default: "linear_interpolation".

iecdf_methodstr

One of ["inverted_cdf", "averaged_inverted_cdf", "closest_observation", "interpolated_inverted_cdf", "hazen", "weibull", "linear", "median_unbiased", "normal_unbiased"]. Method to calculate the inverse empirical CDF (empirical quantile function). Default: "linear".

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias ECDFM class

class ibicus.debias.ECDFM(distribution, cdf_threshold=1e-10, running_window_mode=False, running_window_length=91, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)

Implements Equidistant CDF Matching (ECDFM) based on Li et al. 2010.

ECDFM is a parametric quantile mapping method that attempts to be trend-preserving in all quantiles. ECDFM applies quantilewise correction by adding the difference between a quantile mapping of observations and future values and a quantile mapping of historical climate model values to the future climate model ones.

Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections. Let \(F_{\text{cm_hist}}\) be the cdf fitted as a parametric distribution to climate model output data in the reference period. The future climate projections \(x_{\text{cm_fut}}\) are then mapped to:

\[x_{\text{cm_fut}} \rightarrow x_{\text{cm_fut}} - F^{-1}_{\text{cm_hist}}(F_{\text{cm_fut}}(x_{\text{cm_fut}})) + F^{-1}_{\text{obs}}(F_{\text{cm_fut}}(x_{\text{cm_fut}}))\]

The difference between the future climate model data and the future climate model data quantile mapped to historical climate model data (de facto future model data bias corrected to historical model data) is added to a quantile mapping bias correction between observations and future climate model data. In essence, this method says that future climate model data can be bias corrected directly with reference period observations, if the quantile specific difference between present-day and future climate model simulations is taken into account. This allows for changes in higher moments in the climate model, compared to standard Quantile Mapping where just the mean is assumed to change in future climate.

The method was originally developed with monthly data in view, however the authors of this package think that there is no reason for the method not to be applicable to daily data.

Note

As opposed to most other publications, Li et al. use a 4-parameter beta distribution (scipy.stats.beta) for tas instead of a normal distribution. This can be slow for the fit at times. Consider modifying the distribution parameter for tas.

For precipitation a distribution or model is needed that accounts for mixed zero and positive value character. Default is a precipitation hurdle model (see ibicus.utils.gen_PrecipitationHurdleModel). However also different ones are possible. for_precipitation() helps with the initialisation of different precipitation methods.

Reference:

• Li, H., Sheffield, J., and Wood, E. F. (2010), Bias correction of monthly precipitation and temperature fields from Intergovernmental Panel on Climate Change AR4 models using equidistant quantile matching, J. Geophys. Res., 115, D10101, doi:10.1029/2009JD012882.

Usage information:

• Default settings exist for: ["hurs", "pr", "psl", "rlds", "sfcWind", "tas", "tasmin", "tasmax"].

• apply() requires: no additional arguments except obs, cm_hist, cm_future.

• Next to from_variable() a for_precipitation()-method exists to help you initialise the debiaser for pr.

• The debiaser has been developed for monthly data, however it works with data in any time specification (daily, monthly, etc.).

Examples:

Initialising using from_variable:

>>> debiaser = ECDFM.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Initialising using for_precipitation:

>>> debiaser = ECDFM.for_precipitation(model_type = "hurdle")
>>> debiaser.apply(obs, cm_hist, cm_future)

Attributes:

distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]

Method used for the fit to the historical and future climate model outputs as well as the observations. Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by the ibicus.utils.StatisticalModel class.

cdf_thresholdfloat

Threshold to round CDF-values away from zero and one. Default: 1e-10.

running_window_modebool

Whether DeltaChange is used in running window over the year to account for seasonality. If running_window_mode = False then DeltaChange is applied on the whole period. Default: False.

running_window_lengthint

Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True. Default: 91.

running_window_step_lengthint

Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if running_window_mode = True. Default: 1.

variablestr

Variable for which the debiasing is done. Default: “unknown”.

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
for_precipitation([model_type, ...])	Instanciates the class to a precipitation-debiaser.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

classmethod for_precipitation(model_type='hurdle', amounts_distribution=<scipy.stats._continuous_distns.gamma_gen object>, censoring_threshold=0.1, hurdle_model_randomization=True, hurdle_model_kwds_for_distribution_fit={'floc': 0, 'fscale': None}, **kwargs)

Instanciates the class to a precipitation-debiaser. This allows granular setting of available precipitation models without needing to explicitly specify the precipitation censored model for example.

Parameters:

model_typestr

One of ["censored", "hurdle", "ignore_zeros"]. Model type to be used. See ibicus.utils.gen_PrecipitationGammaLeftCensoredModel, ibicus.utils.gen_PrecipitationHurdleModel and ibicus.utils.gen_PrecipitationIgnoreZeroValuesModel for more details.

amounts_distributionscipy.stats.rv_continuous

Distribution used for precipitation amounts. For the censored model only scipy.stats.gamma is possible.

censoring_thresholdfloat

The censoring-value if a censored precipitation model is used.

hurdle_model_randomizationbool

Whether when computing the cdf-values for a hurdle model randomization shall be used. See ibicus.utils.gen_PrecipitationHurdleModel for more details.

hurdle_model_kwds_for_distribution_fitdict

Dict of parameters used for the distribution fit inside a hurdle model. Default: {"floc": 0, "fscale": None} location of distribution is fixed at zero (``floc = 0) to stabilise Gamma distribution fits in scipy.

**kwargs:

All other class attributes that shall be set and where the standard values shall be overwritten.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias QuantileDeltaMapping class

class ibicus.debias.QuantileDeltaMapping(distribution, trend_preservation, censor_values_to_zero=False, censoring_threshold=5.787037037037037e-07, running_window_mode=True, running_window_length=91, running_window_step_length=31, running_window_mode_over_years_of_cm_future=True, running_window_over_years_of_cm_future_length=31, running_window_over_years_of_cm_future_step_length=1, ecdf_method='linear_interpolation', cdf_threshold=None, *, variable='unknown', reasonable_physical_range=None)

Implements Quantile Delta Mapping based on Cannon et al. 2015.

QDM is a parametric quantile mapping method that also attempts to be trend-preserving. It extends ECDFM such that the two quantile mappings defined there are not only added but also divided by each other to create multiplicative correction. Furthermore it includes both a running window over the year: to account for seasonality, as well as one over the future period to account for changes in trends.

Let cm refer to climate model output, obs to observations and hist/future to whether the data was collected from the reference period or is part of future projections.

Similar to the Equidistant CDF Matching Method based on Li et al. 2010 and implemented in ECDFM, this method bias corrects the future climate model data directly using reference period observations. This is then multiplied by the quotient of future climate model data and a quantile mapping between past and future climate model data to account for the change from past to future period in all quantiles.

In mathematical terms, the transformation can be written as:

\[x_{\text{cm_fut, bc}} (t) = x_{\text{cm_fut}} (t) \cdot \frac{F^{-1}_{\text{obs}}(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}(t)))}{F^{-1}_{\text{cm_hist}}(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}))}\]

This variation preserves relative trends. If absolute trends are to be preserved, the transformation is equivalent to the transformation introduced in Li et al 2010:

\[x_{\text{cm_fut, bc}} (t) = x_{\text{cm_fut}} (t) + F^{-1}_{\text{obs}}(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}(t))) - F^{-1}_{\text{cm_hist}}(\hat F_{\text{cm_fut}}^{(t)} ( x_{\text{cm_fut}})).\]

Hereby \(\hat F_{\text{cm_fut}}^{(t)}\) is the empirical CDF of future climate model values in a window around t. \(F^{-1}_{\text{obs}}\) is the inverse CDF estimated by fitting a distribution to observations. \(F^{-1}_{\text{cm_hist}}\) is the inverse CDF estimated by fitting a distribution to the historical climate model run.

Delta Quantile Mapping is approximately trend preserving in all quantiles because the absolute \(\Delta_{\text{cm}}^{\text{abs}}\) or relative change \(\Delta_{\text{cm}}^{\text{rel}}\) is calculated and applied for each quantile individually.

Running window:

• running_window_over_year: controls whether the methodology and mapping is applied on a running window over the year to account for seasonality. running_window_over_year_length and running_window_over_year_step_length control the length (how many days are included in the running window) and step length (by how far the window is shifted and how many days inside are debiased) respectively.
  
  

• running_window_mode_over_years_of_cm_future controls whether a running window is used to estimate the empirical CDF \(\hat F_{\text{cm_fut}}^{(t)}(x_{\text{cm_fut}}(t))\) and time-dependent quantiles or if this is done statically on the whole future climate model run. running_window_over_years_of_cm_future_length and running_window_over_years_of_cm_future_step_length control the length and step length of this window respectively. Estimating this information in a running window has the advantage of accounting for changes in trends.

If both running windows are active then first the running window inside the year is used to account for seasonality. Values are chunked according to this one. Afterwards the running window over years is used and values are further split up. This is just a choice made for computational efficiency and the order of running window application/chunking does not matter.

Warning

Currently only uneven sizes are allowed for window length and window step length. This allows symmetrical windows of the form [window_center - window length//2, window_center + window length//2`] given an arbitrary window center. This affects both within year and over year window.

References:

• Cannon, A. J., Sobie, S. R., & Murdock, T. Q. (2015). Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? In Journal of Climate (Vol. 28, Issue 17, pp. 6938–6959). American Meteorological Society. https://doi.org/10.1175/jcli-d-14-00754.1

Usage information:

• Default settings exist for: ["hurs", "pr", "psl", "rlds", "sfcWind", "tas", "tasmin", "tasmax"].

• apply() requires: time arguments time_obs, time_cm_hist, and time_cm_future next to obs, cm_hist and cm_future. These are just 1d numpy-arrays of dates (multiple formats are possible as long as they as convertible to numpy or datetime dates) specifying the date for each value/timestep in obs, cm_hist and cm_future. If they are not specified they are inferred, assuming the first observation in all three observation/climate value arrays is on a 1st of January.

• Next to from_variable() a for_precipitation()-method exists to help you initialise the debiaser for pr.

• The debiaser has been developed for and assumes daily data, although application on data using other time specifications (monthly etc.) is possible by setting running_window_mode_within_year = False, modifying the running window arguments for the running window over the years of cm_future, and specifying the time arguments in apply().

Examples:

Running without dates (they are inferred assuming the first value in obs, cm_hist and cm_future always corresponds to a January 1st):

>>> debiaser = QuantileDeltaMapping.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Running with dates and initialising using for_precipitation():

>>> debiaser = QuantileDeltaMapping.for_precipitation(censoring_threshold = 0.1/86400)
>>> debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)

Attributes:

distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]

Distribution or statistical model used to compute the CDF \(F\) of observations and historical climate model values.
Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by the ibicus.utils.StatisticalModel class.

trend_preservationstr

One of ["absolute", "relative"]. If "absolute" then absolute trend preservation is used, if "relative" then relative trend preservation is used.

censor_values_to_zerobool

Whether values below a censoring threshold shall be censored to zero. Only relevant for precipitation. Default: False.

censoring_thresholdfloat

Threshold below which values shall be censored to zero if censor_values_to_zero = True. Relevant mainly for precipitation.
If it is used (so censor_values_to_zero = True) one needs to make sure that the distribution fits to censored data, knows the correct censoring_threshold and assumes all observations under the specified censoring_threshold are zero/censored.
If the standard for_precipitation and from_variable methods are used to construct the class this is ensured by default. However if this parameter is changed manually or own distributions for precipitation are specified problems can arise.

running_window_modebool

Whether QuantileDeltaMapping is used in running window over the year to account for seasonality. If running_window_mode = False then QuantileDeltaMapping is applied on the whole period. Default: True.

running_window_lengthint

Length of the running window in days: how many values are used to calculate the bias adjustment transformation. Only relevant if running_window_mode = True. Default: 91.

running_window_step_lengthint

Step length of the running window in days: how many values are bias adjusted inside the running window and by how far it is moved. Only relevant if running_window_mode = True. Default: 31.

running_window_mode_over_years_of_cm_futurebool

Controls whether the methodology is applied on a running time window, running over the years of cm_fut to calculate time dependent quantiles in future climate model values.

running_window_over_years_of_cm_future_lengthint

Length of the time window centered around t to calculate time dependent quantiles in future climate model values (default: 31 years). Only relevant if running_window_mode_over_years_of_cm_future = True.

running_window_over_years_of_cm_future_step_lengthint

Step length of the time window centered around t to calculate time dependent quantiles in future climate model values (default: 1 year). Only relevant if running_window_mode_over_years_of_cm_future = True.

variablestr

Variable for which the debiasing is done. Default: "unknown".

ecdf_methodstr

One of ["kernel_density", "linear_interpolation", "step_function"]. Method used to calculate the empirical CDF. Default: "linear_interpolation".

cdf_thresholdOptional[float]

Threshold for the CDF-values to round away from 0 and 1. Default: None. It is then set to 1 / (self.running_window_within_year_length * self.running_window_over_years_of_cm_future_length + 1)

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
for_precipitation([censoring_threshold])	Instanciates the class to a precipitation-debiaser.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

classmethod for_precipitation(censoring_threshold=5.787037037037037e-07, **kwargs)

Instanciates the class to a precipitation-debiaser.

Parameters:

censoring_threshold: float

The censoring-value under which precipitation amounts are assumed zero/censored.

**kwargs:

All other class attributes that shall be set and where the standard values shall be overwritten.

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.

ibicus.debias ISIMIP class

class ibicus.debias.ISIMIP(trend_preservation_method, distribution, nonparametric_qm, detrending, lower_bound=inf, lower_threshold=inf, upper_bound=-inf, upper_threshold=-inf, scale_by_annual_cycle_of_upper_bounds=False, window_length_annual_cycle_of_upper_bounds=31, impute_missing_values=False, detrending_with_significance_test=True, trend_transfer_only_for_values_within_threshold=True, bias_correct_frequencies_of_values_beyond_thresholds=True, event_likelihood_adjustment=False, ks_test_for_goodness_of_cdf_fit=True, ecdf_method='linear_interpolation', iecdf_method='linear', mode_non_parametric_qm='normal', running_window_mode=True, running_window_length=31, running_window_step_length=1, *, variable='unknown', reasonable_physical_range=None)

Implements the ISIMIP3b and ISIMIP3BASD bias adjustment methodology based on Lange 2019 and Lange 2021.

ISIMIP is a semi-parametric quantile mapping method that attempts to be trend-preserving in all quantiles by generating ‘pseudo future observations’ and executing the quantile mapping between the future climate model and the pseudo future observations. ISIMIP includes special cases for several of the variables, and for a complete description of the methodology we refer to the ISIMIP documentation.

This notebook demonstrates that the results obtained with this implementation of ISIMIP are consistent with the reference implementation.

ISIMIP covers the following variables:

["hurs", "pr", "prsnratio", "psl", "rsds", "rlds", "sfcwind", "tas", "tasrange", "taskew"]

The variables tasmin, tasmax and prsn can be derived as: tasrange = tasmin - tasmax, tasskew = (tas-tasmin)/(tasmax-tasmin) and prsnratio = prsn/pr. The ISIMIP methodology consists of 8 steps that cover the special cases and particularities for the different variables. We refer to Lange 2019 for a complete description. In the following the steps shall only be sketched. Let obs refer to the observation dataset and cm_hist and cm_fut to historical and future climate model values.

Steps:

1. step1(): (For rsds only). If scale_by_annual_cycle_of_upper_bounds = True then an annual cycle of upper bounds is calculated to scale the values in obs, cm_hist and cm_fut onto the interval [0,1]. This is to account physical upper bounds that vary over the year, while other variables have static upper and lower bounds.
   
   

2. step2(): (For prsnratio only). If impute_missing_values = True missing values (passed in either as masked array or array with np.nan values) in obs, cm_hist and cm_fut are imputed via a somewhat trend preserving sampling from all other non-missing values within a given time window.
   
   

3. step3(): (For tas, psl and rlds). If detrending = True the timeseries are detrended to account for trends within historical and future period (not for trends between historical and future one as this is done in step 5). Trends are estimated via linear regression on the annual means and then removed from the daily observations. If detrending_with_significance_test = True then a significance test is first used to determine whether the trend is to be removed.
   
   

4. step4(): If the variable has either a lower or upper bound and threshold, values beyond the threshold are randomized within between the respective bound and threshold. This allows to quantile map these values in step 6. Randomization is done uniformly (different than from Lange 2019 to reflect a change in ISIMIP3b v2.2), however ranks are preserved.
   
   

5. step5(): Here pseudo future observations are generated by transferring a climate change signal onto the observations. This is done differently depending on the trend_preservation_method: one of ["additive", "multiplicative", "mixed", "bounded"] to reflect differences in the trends of different variables. For bounded variables if trend_transfer_only_for_values_within_threshold = True then trend transfer only happens for the values within thresholds, else for all values.
   
   

6. step6(): Given the pseudo future observations from step 5 a quantile mapping is used mapping the values of cm_fut to the pseudo future observations to generate debiased future climate model values.

   • If event_likelihood_adjustment = True also an event likelihood adjustment is done, following Switanek et al. 2017 and as originally described in Lange 2019, but removed in the ISIMIP3b run.

   • By default the quantile mapping is done parametically using the distribution in self.distribution, however if nonparametric_qm = True or ks_test_for_goodness_of_cdf_fit = True and a Kolmogorov Smirnov statistic indicats a bigger misfit a nonparametric quantile mapping is used instead (default for hurs, psrnratio, rsds, tasskew).

   • If bias_correct_frequencies_of_values_beyond_thresholds = True (default) also the frequencies of values beyond thresholds are adjusted to. This adjusts eg. the number of precipitation dry days.
     
     

7. step7(): If detrending = True then the trend removed in step 3 is restored onto the debiased cm_fut.
   
   

8. step8(): (For rsds only). If scale_by_annual_cycle_of_upper_bounds = True then a debiased annual cycle of upper bounds is calculated to rescale the debiased climate model values from [0,1] back to the original scale.
   
   

As for all other debiasers all steps are applied locationwise and if running_window_mode = True (default) in a running window with step size running_window_step_length and length running_window_length. Otherwise steps are applied on a month by month basis to account for seasonality.

Warning

Currently only uneven sizes are allowed for window length and window step length. This allows symmetrical windows of the form [window_center - window length//2, window_center + window length//2] given an arbitrary window center.

All variables have different default settings and using the ISIMIP.from_variable(variable, **kwargs) method it is possible to initialize a debiaser with these settings eg.

>>> debiaser = ISIMIP.from_variable("tas")

However depending on the debiasing region and usecase at hand it might be important to modify some of the default settings, such as for example:

>>> debiaser = ISIMIP.from_variable("tas", detrending = False)

or

>>> debiaser = ISIMIP.from_variable("tas")
>>> debiaser.detrending = False

If run with the default options, the results generated in this package are exactly identical to v3.0.1 (2022-06-27) of the ISIMIP3BASD reference implementation, as shown in the notebook 05 ISIMIP Consistency. By setting parameters appropriately reproducing lower versions is possible and also consistency with future versions is considered.

In contrast to the reference implementation implementation of ISIMIP this code includes further options for customization of the ISIMIP behavior, allows more flexible usage given that it operates on a numerical basis and handles a variety of cases that might lead to bugs in the reference implementation.

References:

• Lange, S. (2019). Trend-preserving bias adjustment and statistical downscaling with ISIMIP3BASD (v1.0). In Geoscientific Model Development (Vol. 12, Issue 7, pp. 3055–3070). Copernicus GmbH. https://doi.org/10.5194/gmd-12-3055-2019

• Lange, S. (2022). ISIMIP3BASD (3.0.1) [Computer software]. Zenodo. https://doi.org/10.5281/ZENODO.6758997

• Switanek, M. B., Troch, P. A., Castro, C. L., Leuprecht, A., Chang, H.-I., Mukherjee, R., & Demaria, E. M. C. (2017). Scaled distribution mapping: a bias correction method that preserves raw climate model projected changes. In Hydrology and Earth System Sciences (Vol. 21, Issue 6, pp. 2649–2666). Copernicus GmbH. https://doi.org/10.5194/hess-21-2649-2017.

Usage information:

• Default settings exist for: ["hurs", "pr", "prsnratio", "psl", "rlds", "rsds", "sfcWind", "tas", "tasrange", "tasskew"].

• apply() requires: time arguments time_obs, time_cm_hist, and time_cm_future next to obs, cm_hist and cm_future. These are just 1d numpy-arrays of dates (multiple formats are possible as long as they as convertible to numpy or datetime dates) specifying the date for each value/timestep in obs, cm_hist and cm_future. If they are not specified they are inferred, assuming the first observation in all three observation/climate value arrays is on a 1st of January.

• The debiaser assumes daily data.

Examples:

Running without dates (they are inferred assuming the first value in obs, cm_hist and cm_future always corresponds to a January 1st):

>>> debiaser = ISIMIP.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future)

Running with dates:

>>> debiaser = ISIMIP.from_variable("tas")
>>> debiaser.apply(obs, cm_hist, cm_future, time_obs = time_obs, time_cm_hist = time_cm_hist, time_cm_future = time_cm_future)

Core Parameters:

Parameters:

trend_preservation_methodstr

One of ["additive", "multiplicative", "mixed", "bounded"]. Method of how the climate change trend is transferred to observations to generate the pseudo future observations. Depends on the kind of climate change trend expected in the variable.

distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]

Distribution or statistical model used for the quantile mapping in step 6 if nonparametric_qm = False.
Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by the ibicus.utils.StatisticalModel class.

nonparametric_qmbool

Whether nonparametric quantile mapping is used for debiasing in step 6. If nonparametric_qm = False then quantile mapping is done using the distribution in self.distribution.

detrendingbool

Whether detrending is applied in step4 to account for within obs, cm_hist and cm_fut trends.

Full list of attributes:

Attributes:

lower_boundfloat

Variable specific: Lower bound of a variable. Default: -np.inf.

lower_thresholdfloat

Variable specific: Lower threshold of a variable. Default: -np.inf.

upper_boundfloat

Variable specific: Upper bound of a variable. Default: np.inf.

upper_thresholdfloat

Variable specific: Upper threshold of a variable. Default: np.inf.

scale_by_annual_cycle_of_upper_bounds**bool

Step 1: Whether in step 1 a scaling by an estimated annual cycle of upper bounds is done to scale obs, cm_hist and cm_fut values onto [0,1]. Default: False.

window_length_annual_cycle_of_upper_bounds**int

Step 1: Window length to estimate an annual cycle of upper bounds to rescale. Only relevant if scale_by_annual_cycle_of_upper_bounds = True. Default: 31.

impute_missing_valuesbool

Step 2: Whether in step 2 missing values (indicated either by a mask of np.nan values) are imputed using the other values within the window.

detrendingbool

Step 3: Whether detrending is applied in step4 to account for within obs, cm_hist and cm_fut trends.

detrending_with_significance_testbool

Step 3: Whether detrending is applied conditional on a significance test for the trend. Only relevant if detrending = True. Default: True.

trend_preservation_methodstr

Step 5: One of ["additive", "multiplicative", "mixed", "bounded"]. Method of how the climate change trend is transferred to observations to generate pseudo future ones.

trend_transfer_only_for_values_within_thresholdbool

Step 5: Whether the trend transfer is done only for the values within thresholds. Default: True.

distributionUnion[scipy.stats.rv_continuous, scipy.stats.rv_discrete, scipy.stats.rv_histogram, StatisticalModel]

Step 6: Distribution or statistical model used for the quantile mapping in step 6 if nonparametric_qm = False.
Usually a distribution in scipy.stats.rv_continuous, but can also be an empirical distribution as given by scipy.stats.rv_histogram or a more complex statistical model as wrapped by the ibicus.utils.StatisticalModel class.

nonparametric_qmbool

Step 6: Whether nonparametric quantile mapping is used for debiasing in step 6. If nonparametric_qm = False then quantile mapping is done using the distribution in distribution.

bias_correct_frequencies_of_values_beyond_thresholdsbool

Step 6: Whether frequencies of values beyond thresholds are adjusted/bias corrected. Default: True.

event_likelihood_adjustmentbool

Step 6: Whether in conjuction with quantile mapping an event likelihood adjustment is used inspired by Switanek et al. 2017. Default: False.

ks_test_for_goodness_of_cdf_fitbool

Step 6: Whether a Kolmogorov Smirnov statistic is used to diagnose misfit of the parametric CDF prior to quantile mapping. If such misfit is diagnosed nonparametric quantile mapping is used. Only relevant if nonparametric_qm = False. Default: True.

ecdf_methodstr

Computation: One of ["kernel_density", "linear_interpolation", "step_function"]. Method to calculate the empirical CDF. Default: "linear_interpolation".

iecdf_methodstr

Computation: One of ["inverted_cdf", "averaged_inverted_cdf", closest_observation", "interpolated_inverted_cdf", "hazen", "weibull", "linear", "median_unbiased", "normal_unbiased"]. Method to calculate the inverse empirical CDF (empirical quantile function). Default: "linear".

mode_non_parametric_qmstr

Computation: One of ["normal", "isimipv3.0"]. Computation method used for nonparametric quantile mapping. If "normal" this is based on estimated empirical CDFs and inverse empirical CDFs. If "isimipv3.0" this is based on estimated ranks using scipy.stats.rankdata which can lead to inconsistencies. Default: "normal".

running_window_modebool

Iteration: Whether ISIMIP is used in running window mode to account for seasonalities. If running_window_mode = False then ISIMIP is applied by months. Default: True.

running_window_lengthint

Iteration: Length of the running window in days: how many values are used to the debiased climate model values. Only relevant if running_window_mode = True. Default: 31.

running_window_step_lengthint

Iteration: Step length of the running window in days: how many values are debiased inside the running window. Only relevant if running_window_mode = True. Default: 1.

variablestr

Variable for which the debiasing is done. Default: "unknown".

Methods

apply(obs, cm_hist, cm_future[, ...])	Applies the debiaser onto given data.
from_variable(variable, **kwargs)	Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.
step1(obs_hist, cm_hist, cm_future, ...)	Step 1: if scale_by_annual_cycle_of_upper_bounds = True (rsds only).
step2(obs_hist, cm_hist, cm_future)	Step 2: impute values for prsnratio which are missing on days where there is no precipitation.
step3(obs_hist, cm_hist, cm_future, ...)	Step 3: Linear trend removal if detrending = True.
step4(obs_hist, cm_hist, cm_future)	Step4: If the variable is bounded then values between the threshold and corresponding bound (including values equal to the bound) are randomized uniformly between the threshold and bound and resorted according to the order in which they were before.
step5(obs_hist, cm_hist, cm_future)	Step 5: generates pseudo-future observations by transfering simulated trends to historical recorded observations.
step6(obs_hist, obs_future, cm_hist, cm_future)	Step 6: parametric quantile mapping between cm_future and obs_future (the pseudo-future observations) to debias the former.
step7(cm_future, trend_cm_future)	Step 7: If detrending = True adds the trend removed in step 3 back to the debiased cm_future values.
step8(cm_future, debiased_annual_cycle, ...)	Step 8: if scale_by_annual_cycle_of_upper_bounds = True (rsds only).

classmethod from_variable(variable, **kwargs)

Instanciates the class from a variable: either a string referring to a standard variable name or a Variable object.

Parameters:

variableUnion[str, Variable]

String or Variable object referring to standard meteorological variable for which default settings can be used.

**kwargs:

All other class attributes that shall be set and where the standard values for variable shall be overwritten.

Returns:

Debiaser

Instance of the class for the given variable.

step1(obs_hist, cm_hist, cm_future, time_obs_hist, time_cm_hist, time_cm_future)

Step 1: if scale_by_annual_cycle_of_upper_bounds = True (rsds only). Scales obs, cm_hist and cm_future values to [0, 1] by computing an annual cycle of upper bounds as “running mean values of running maximum values of multiyear daily maximum values.” (Lange 2019). Furthermore return a debiased annual cycle of upper bounds to rescale cm_future values in step8.

step2(obs_hist, cm_hist, cm_future)

Step 2: impute values for prsnratio which are missing on days where there is no precipitation. They are imputed by effectively sampling the iecdf (see Lange 2019 and ISIMIP3b factsheet for the method).

step3(obs_hist, cm_hist, cm_future, years_obs_hist, years_cm_hist, years_cm_future)

Step 3: Linear trend removal if detrending = True. This is because certain variables (eg. temp) can have substantial trends also within training and application period (not only between). These trends are removed to “prevent a confusion of these trends with interannual variability during quantile mapping (steps 5 and6)” (Lange 2019). The trend for cm_future is subsequently added again in step7. Trends are calculated by linearly regressing the yearly mean values y_i against years t_i. From each observation in year t_i then the slope * (t_i - mean(t_i)) is removed (normalising the trend such that the sum over all t_i is zero). See Lange 2019 for the exact method.

If detrending_with_significance_test = True (default in ISIMIP v2.5) then linear trends are only removed subject it being significant (p-value < 0.05).

step4(obs_hist, cm_hist, cm_future)

Step4: If the variable is bounded then values between the threshold and corresponding bound (including values equal to the bound) are randomized uniformly between the threshold and bound and resorted according to the order in which they were before.

step5(obs_hist, cm_hist, cm_future)

Step 5: generates pseudo-future observations by transfering simulated trends to historical recorded observations. This makes the ISIMIP-method trend-preserving.

step6(obs_hist, obs_future, cm_hist, cm_future)

Step 6: parametric quantile mapping between cm_future and obs_future (the pseudo-future observations) to debias the former. Core of the bias adjustment method. For (partly) bounded climate variables additionally the frequency of values is bias-adjusted before the other observations are quantile mapped. If event_likelihood_adjustment = True then additionally to “normal quantile mapping” the likelihood of individual events is adjusted (see Lange 2019 for the method which is based on Switanek 2017).

step7(cm_future, trend_cm_future)

Step 7: If detrending = True adds the trend removed in step 3 back to the debiased cm_future values.

step8(cm_future, debiased_annual_cycle, time_cm_future)

Step 8: if scale_by_annual_cycle_of_upper_bounds = True (rsds only). Scales debiased cm_future on [0, 1] back to the usual scale using the debiased annual cycle

apply(obs, cm_hist, cm_future, progressbar=True, parallel=False, nr_processes=4, failsafe=False, **kwargs)

Applies the debiaser onto given data.

Parameters:

obsnp.ndarray

3-dimensional numpy array of observations of the meteorological variable. The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations.

cm_histnp.ndarray

3-dimensional numpy array of values of a climate model run during the same or a similar period as observations (historical run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

cm_futurenp.ndarray

3-dimensional numpy array of values of a climate model to debias (future run). The first dimension should correspond to temporal steps and the 2nd and 3rd one to locations. Shape in the 2nd and 3rd dimension needs to be the same as for obs.

progressbarbool = False

Whether a progressbar shall be shown to indicate the debiaser status. Default: True.

parallelbool = False

Whether the debiasing shall be executed in parallel. No progressbar is shown in this case. Default: False.

nr_processesint = 4

Number of processes for parallel code execution. Default: 4.

failsafebool = False

Whether execution shall run in a failsafe mode. Debiasing then continues if it encounters an error at one location and returns np.nan at this location. Default: False.

Returns:

np.ndarray

3-dimensional numpy array containing the debiased climate model values for the future run (cm_future). Has the spatial dimensions as obs, cm_hist, cm_future.