Skip to contents

Right-Censored Log Loss Survival Measure

Source: R/MeasureSurvRCLL.R
mlr_measures_surv.rcll.Rd

Calculates the right-censored logarithmic (log), loss.

Details

The RCLL, in the context of probabilistic predictions, is defined by $$L(f, t, \Delta) = -log(\Delta f(t) + (1 - \Delta) S(t))$$ where \(\Delta\) is the censoring indicator, \(f\) the probability density function and \(S\) the survival function. RCLL is proper given that censoring and survival distribution are independent, see Rindt et al. (2022).

Note: Even though RCLL is a proper scoring rule, the calculation of \(f(t)\) (which in our case is discrete, i.e. it is a probability mass function) for time points in the test set that don't exist in the predicted survival matrix (distr), results in 0 values, which are substituted by "eps" in our implementation, therefore skewing the result towards \(-log(eps)\). This problem is also discussed in Rindt et al. (2022), where the authors perform interpolation to get non-zero values for the \(f(t)\). Until this is handled in mlr3proba some way, we advise against using this measure for model evaluation.

Dictionary

This Measure can be instantiated via the dictionary mlr_measures or with the associated sugar function msr():

MeasureSurvRCLL$new()
mlr_measures$get("surv.rcll")
msr("surv.rcll")

Parameters

IdTypeDefaultLevelsRange
epsnumeric1e-15\([0, 1]\)
selogicalFALSETRUE, FALSE-
ERVlogicalFALSETRUE, FALSE-
na.rmlogicalTRUETRUE, FALSE-

Meta Information

  • Type: "surv"

  • Range: \([0, \infty)\)

  • Minimize: TRUE

  • Required prediction: distr

Parameter details

  • eps (numeric(1))
    Very small number to substitute zero values in order to prevent errors in e.g. log(0) and/or division-by-zero calculations. Default value is 1e-15.

  • se (logical(1))
    If TRUE then returns standard error of the measure otherwise returns the mean across all individual scores, e.g. the mean of the per observation scores. Default is FALSE (returns the mean).

  • ERV (logical(1))
    If TRUE then the Explained Residual Variation method is applied, which means the score is standardized against a Kaplan-Meier baseline. Default is FALSE.

  • na.rm (logical(1))
    If TRUE (default) then removes any NAs in individual score calculations.

References

Avati, Anand, Duan, Tony, Zhou, Sharon, Jung, Kenneth, Shah, H N, Ng, Y A (2020). “Countdown Regression: Sharp and Calibrated Survival Predictions.” Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, 115(4), 145–155. https://proceedings.mlr.press/v115/avati20a.html.

Rindt, David, Hu, Robert, Steinsaltz, David, Sejdinovic, Dino (2022). “Survival regression with proper scoring rules and monotonic neural networks.” Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, 151(4), 1190–1205. https://proceedings.mlr.press/v151/rindt22a.html.

See also

Other survival measures: mlr_measures_surv.calib_alpha, mlr_measures_surv.calib_beta, mlr_measures_surv.calib_index, mlr_measures_surv.chambless_auc, mlr_measures_surv.cindex, mlr_measures_surv.dcalib, mlr_measures_surv.graf, mlr_measures_surv.hung_auc, mlr_measures_surv.intlogloss, mlr_measures_surv.logloss, mlr_measures_surv.mae, mlr_measures_surv.mse, mlr_measures_surv.nagelk_r2, mlr_measures_surv.oquigley_r2, mlr_measures_surv.rmse, mlr_measures_surv.schmid, mlr_measures_surv.song_auc, mlr_measures_surv.song_tnr, mlr_measures_surv.song_tpr, mlr_measures_surv.uno_auc, mlr_measures_surv.uno_tnr, mlr_measures_surv.uno_tpr, mlr_measures_surv.xu_r2

Other Probabilistic survival measures: mlr_measures_surv.graf, mlr_measures_surv.intlogloss, mlr_measures_surv.logloss, mlr_measures_surv.schmid

Other distr survival measures: mlr_measures_surv.calib_alpha, mlr_measures_surv.calib_index, mlr_measures_surv.dcalib, mlr_measures_surv.graf, mlr_measures_surv.intlogloss, mlr_measures_surv.logloss, mlr_measures_surv.schmid

Super classes

mlr3::Measure -> mlr3proba::MeasureSurv -> MeasureSurvRCLL

Methods

Public methods

Inherited methods

Method new()

Creates a new instance of this R6 class.

Usage

MeasureSurvRCLL$new(ERV = FALSE)

Arguments

ERV

(logical(1))
Standardize measure against a Kaplan-Meier baseline (Explained Residual Variation)

Method clone()

The objects of this class are cloneable with this method.

Usage

MeasureSurvRCLL$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.