Package website: release | dev
Probabilistic Supervised Learning for mlr3.
What is mlr3proba ?
mlr3proba is a machine learning toolkit for making probabilistic predictions within the mlr3 ecosystem. It currently supports the following tasks:
- Probabilistic supervised regression - Supervised regression with a predictive distribution as the return type.
- Predictive survival analysis - Survival analysis where individual predictive hazards can be queried. This is equivalent to probabilistic supervised regression with censored observations.
- Unconditional distribution estimation, where the distribution is returned. Sub-cases are density estimation and unconditional survival estimation.
Key features of mlr3proba are
- A unified fit/predict model interface to any probabilistic predictive model (frequentist, Bayesian, or other)
- Pipeline/model composition
- Task reduction strategies
- Domain-agnostic evaluation workflows using task specific algorithmic performance measures.
mlr3proba makes use of the distr6 probability distribution interface as its probabilistic predictive return type.
Feature Overview
The current mlr3proba release focuses on survival analysis, and contains:
- Task frameworks for survival analysis (
TaskSurv) - A comprehensive selection of 17 predictive survival learners
- A comprehensive selection of 21 performance measures for predictive survival learners, with respect to prognostic index (continuous rank) prediction, and probabilistic (distribution) prediction
- PipeOps integrated with mlr3pipelines, for basic pipeline building, and reduction/composition strategies using linear predictors and baseline hazards.
Roadmap
The vision of mlr3proba is to provide comprehensive machine learning functionality to the mlr3 ecosystem for continuous probabilistic return types.
The lifecycle of the survival task and features are considered maturing and any major changes are unlikely.
The density and probabilistic supervised regression tasks are currently in the early stages of development. Task frameworks have been drawn up, but may not be stable; learners need to be interfaced, and contributions are very welcome (see issues).
Survival Analysis
Survival Learners
| ID | Learner | Package |
|---|---|---|
| surv.blackboost | Gradient Boosting with Regression Trees | mboost |
| surv.coxph | Cox Proportional Hazards | survival |
| surv.cvglmnet | Cross-Validated GLM with Elastic Net Regularization | glmnet |
| surv.flexible | Flexible Parametric Spline Models | flexsurv |
| surv.gamboost | Gradient Boosting for Additive Models | mboost |
| surv.gbm | Generalized Boosting Regression Modeling | gbm |
| surv.glmboost | Gradient Boosting with Component-wise Linear Models | mboost |
| surv.glmnet | GLM with Elastic Net Regularization | glmnet |
| surv.kaplan | Kaplan-Meier Estimator | survival |
| surv.mboost | Gradient Boosting for Generalized Additive Models | mboost |
| surv.nelson | Nelson-Aalen Estimator | survival |
| surv.parametric | Fully Parametric Survival Models | survival |
| surv.penalized | L1 and L2 Penalized Estimation in GLMs | penalized |
| surv.randomForestSRC | RandomForestSRC Survival Forest | randomForestSRC |
| surv.ranger | Ranger Survival Forest | ranger |
| surv.rpart | Rpart Survival Forest | rpart |
| surv.svm | Regression, Ranking and Hybrid Support Vector Machines | survivalsvm |
Survival Measures
| ID | Learner | Package |
|---|---|---|
| surv.beggC | Begg’s C-Index | survAUC |
| surv.chamblessAUC | Chambless and Diao’s AUC | survAUC |
| surv.gonenC | Gonen and Heller’s C-Index | survAUC |
| surv.graf | Integrated Graf Score | mlr3proba |
| surv.grafSE | Standard Error of Integrated Graf Score | mlr3proba |
| surv.harrellC | Harrell’s C-Index | mlr3proba |
| surv.hungAUC | Hung and Chiang’s AUC | survAUC |
| surv.intlogloss | Integrated Log Loss | mlr3proba |
| surv.intloglossSE | Standard Error of Integrated Log Loss | mlr3proba |
| surv.logloss | Log Loss | mlr3proba |
| surv.loglossSE | Standard Error of Log Loss | mlr3proba |
| surv.nagelkR2 | Nagelkerke’s R2 | survAUC |
| surv.oquigleyR2 | O’Quigley, Xu, and Stare’s R2 | survAUC |
| surv.songAUC | Song and Zhou’s AUC | survAUC |
| surv.songTNR | Song and Zhou’s TNR | survAUC |
| surv.songTPR | Song and Zhou’s TPR | survAUC |
| surv.unoAUC | Uno’s AUC | survAUC |
| surv.unoC | Uno’s C-Index | survAUC |
| surv.unoTNR | Uno’s TNR | survAUC |
| surv.unoTPR | Uno’s TPR | survAUC |
| surv.xuR2 | Xu and O’Quigley’s R2 | survAUC |
Near-Future Plans
- Add
TaskDensity,PredictionDensity,LearnerDensity, and associated learners/measures - Add
probpredict type toTaskRegr, and associated learners/measures - Allow
MeasureSurvto return measures at multiple time-points simultaneously - Improve estimation of integrated scores, and re-implement survAUC scores in mlr3proba
- Continue to add survival measures and learners
Bugs, Questions, Feedback
mlr3proba is a free and open source software project that encourages participation and feedback. If you have any issues, questions, suggestions or feedback, please do not hesitate to open an “issue” about it on the GitHub page!
In case of problems / bugs, it is often helpful if you provide a “minimum working example” that showcases the behaviour (but don’t worry about this if the bug is obvious).
Similar Projects
Predecessors to this package are previous instances of survival modelling in mlr. The skpro package in the python/scikit-learn ecosystem follows a similar interface for probabilistic supervised learning and is an architectural predecessor. Several packages exist which allow probabilistic predictive modelling with a Bayesian model specific general interface, such as jags and stan. For implementation of a few survival models and measures, a central package is survival. There does not appear to be a package that provides an architectural framework for distribution/density estimation, see this list for a review of density estimation packages in R.
