Integrated Schmid Score Survival Measure

Source: R/MeasureSurvSchmid.R

Calculates the Integrated Schmid Score (ISS), aka integrated absolute loss.

For an individual who dies at time \(t\), with predicted Survival function, \(S\), the Schmid Score at time \(t^*\) is given by $$L(S,t|t^*) = [(S(t^*))I(t \le t^*, \delta = 1)(1/G(t))] + [((1 - S(t^*)))I(t > t^*)(1/G(t^*))]$$ # nolint where \(G\) is the Kaplan-Meier estimate of the censoring distribution.

The re-weighted ISS, ISS* is given by $$L(S,t|t^*) = [(S(t^*))I(t \le t^*, \delta = 1)(1/G(t))] + [((1 - S(t^*)))I(t > t^*)(1/G(t))]$$ # nolint where \(G\) is the Kaplan-Meier estimate of the censoring distribution, i.e. always weighted by \(G(t)\). ISS* is strictly proper when the censoring distribution is independent of the survival distribution and when G is fit on a sufficiently large dataset. ISS is never proper. Use proper = FALSE for ISS and proper = TRUE for ISS*, in the future the default will be changed to proper = TRUE. Results may be very different if many observations are censored at the last observed time due to division by 1/eps in proper = TRUE.

If integrated == FALSE then the sample mean is taken for the single specified times, \(t^*\), and the returned score is given by $$L(S,t|t^*) = \frac{1}{N} \sum_{i=1}^N L(S_i,t_i|t^*)$$ where \(N\) is the number of observations, \(S_i\) is the predicted survival function for individual \(i\) and \(t_i\) is their true survival time.

If integrated == TRUE then an approximation to integration is made by either taking the sample mean over all \(T\) unique time-points (method == 1), or by taking a mean weighted by the difference between time-points (method == 2). Then the sample mean is taken over all \(N\) observations. $$L(S) = \frac{1}{NT} \sum_{i=1}^N \sum_{j=1}^T L(S_i,t_i|t^*_j)$$

Details

If task and train_set are passed to $score then G is fit on training data, otherwise testing data. The first is likely to reduce any bias caused by calculating parts of the measure on the test data it is evaluating. The training data is automatically used in scoring resamplings.

Dictionary

This Measure can be instantiated via the dictionary mlr_measures or with the associated sugar function msr():

MeasureSurvSchmid$new()
mlr_measures$get("surv.schmid")
msr("surv.schmid")

Meta Information

  • Type: "surv"

  • Range: \([0, \infty)\)

  • Minimize: TRUE

  • Required prediction: distr

References

Schemper, Michael, Henderson, Robin (2000). “Predictive Accuracy and Explained Variation in Cox Regression.” Biometrics, 56, 249--255. doi: 10.1002/sim.1486 .

Schmid, Matthias, Hielscher, Thomas, Augustin, Thomas, Gefeller, Olaf (2011). “A Robust Alternative to the Schemper-Henderson Estimator of Prediction Error.” Biometrics, 67(2), 524--535. doi: 10.1111/j.1541-0420.2010.01459.x .

See also

Other survival measures: mlr_measures_surv.calib_alpha, mlr_measures_surv.calib_beta, mlr_measures_surv.chambless_auc, mlr_measures_surv.cindex, mlr_measures_surv.dcalib, mlr_measures_surv.graf, mlr_measures_surv.hung_auc, mlr_measures_surv.intlogloss, mlr_measures_surv.logloss, mlr_measures_surv.mae, mlr_measures_surv.mse, mlr_measures_surv.nagelk_r2, mlr_measures_surv.oquigley_r2, mlr_measures_surv.rmse, mlr_measures_surv.song_auc, mlr_measures_surv.song_tnr, mlr_measures_surv.song_tpr, mlr_measures_surv.uno_auc, mlr_measures_surv.uno_tnr, mlr_measures_surv.uno_tpr, mlr_measures_surv.xu_r2

Other Probabilistic survival measures: mlr_measures_surv.graf, mlr_measures_surv.intlogloss, mlr_measures_surv.logloss

Other distr survival measures: mlr_measures_surv.calib_alpha, mlr_measures_surv.dcalib, mlr_measures_surv.graf, mlr_measures_surv.intlogloss, mlr_measures_surv.logloss

Super classes

mlr3::Measure -> mlr3proba::MeasureSurv -> mlr3proba::MeasureSurvIntegrated -> MeasureSurvSchmid

Active bindings

se

(logical(1))
If TRUE returns the standard error of the measure.

Methods

Public methods

Inherited methods

Method new()

Creates a new instance of this R6 class.

Usage

MeasureSurvSchmid$new(
  integrated = TRUE,
  times,
  method = 2,
  se = FALSE,
  proper = FALSE,
  eps = 0.001
)

Arguments

integrated

(logical(1))
If TRUE (default), returns the integrated score; otherwise, not integrated.

times

(numeric())
If integrate == TRUE then a vector of time-points over which to integrate the score. If integrate == FALSE then a single time point at which to return the score.

method

(integer(1))
If integrate == TRUE selects the integration weighting method. method == 1 corresponds to weighting each time-point equally and taking the mean score over discrete time-points. method == 2 corresponds to calculating a mean weighted by the difference between time-points. method == 2 is default to be in line with other packages.

se

(logical(1))
If TRUE returns the standard error of the measure.

proper

(logical(1))
If TRUE then weights scores by the censoring distribution at the observed event time, which results in a strictly proper scoring rule if censoring and survival time distributions are independent and a sufficiently large dataset is used to weight the measure. If FALSE then weights scores by the Graf method which is the more common usage but the loss is not proper. In v0.5.0, the default will be changed to TRUE.

eps

(numeric(1))
Very small number to set zero-valued predicted probabilities to in order to prevent errors in log(0) and 1/0 calculation.

Method clone()

The objects of this class are cloneable with this method.

Usage

MeasureSurvSchmid$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.