Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,6 @@ test/testthat/snaps
docs/
docs
/doc/
.DS_Store
/Meta/
CRAN-SUBMISSION
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: mcboost
Type: Package
Title: Multi-Calibration Boosting
Version: 0.4.3-9000
Version: 0.4.4
Authors@R:
c(person(given = "Florian",
family = "Pfisterer",
Expand Down Expand Up @@ -69,7 +69,7 @@ Suggests:
covr,
testthat (>= 3.1.0)
Roxygen: list(markdown = TRUE, r6 = TRUE)
RoxygenNote: 7.3.1
RoxygenNote: 7.3.2
VignetteBuilder: knitr
Collate:
'AuditorFitters.R'
Expand Down
4 changes: 4 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# dev

# mcboost 0.4.4

* Fix link anchors

# mcboost 0.4.3

* Compatibility with upcoming 'paradox' release.
Expand Down
5 changes: 2 additions & 3 deletions R/MCBoost.R
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,6 @@ MCBoost = R6::R6Class("MCBoost",
labels = private$assert_labels(labels, ...)
pred_probs = private$assert_prob(do.call(self$predictor, discard(list(data, predictor_args), is.null)), data, ...)
buckets = private$create_buckets(pred_probs)

# Compute residuals and multiply with weighting for low-degree MC.
resid = private$compute_residuals(pred_probs, labels)
weighted_preds = private$compute_weighted_preds(pred_probs)
Expand Down Expand Up @@ -285,7 +284,7 @@ MCBoost = R6::R6Class("MCBoost",
},
#' @description
#' Predict a dataset with multi-calibrated predictions
#' @param x [`data.table`] \cr
#' @param x [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param t [`integer`] \cr
#' Number of multi-calibration steps to predict. Default: `Inf` (all).
Expand Down Expand Up @@ -320,7 +319,7 @@ MCBoost = R6::R6Class("MCBoost",
#' Compute the auditor effect for each instance which are the cumulative
#' absolute predictions of the auditor. It indicates "how much"
#' each observation was affected by multi-calibration on average across iterations.
#' @param x [`data.table`] \cr
#' @param x [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param aggregate [`logical`] \cr
#' Should the auditor effect be aggregated across iterations? Defaults to `TRUE`.
Expand Down
10 changes: 5 additions & 5 deletions R/PipeOpLearnerPred.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#'
#' @usage NULL
#' @name mlr_pipeops_mcboost
#' @format [`R6Class`] inheriting from [`mlr3pipelines::PipeOp`].
#' @format [`R6Class`][R6::R6Class] inheriting from [`mlr3pipelines::PipeOp`].
#'
#' @description
#' [`mlr3pipelines::PipeOp`] that trains a [`Learner`][mlr3::Learner] and passes its predictions forward during training and prediction.
Expand Down Expand Up @@ -33,17 +33,17 @@
#' [`mlr3pipelines::PipeOpTaskPreproc`]. It is a named `list` with the inherited members, as well as:
#' * `model` :: `any`\cr
#' Model created by the [`Learner`][mlr3::Learner]'s `$.train()` function.
#' * `train_log` :: [`data.table`] with columns `class` (`character`), `msg` (`character`)\cr
#' * `train_log` :: [`data.table`][data.table::data.table] with columns `class` (`character`), `msg` (`character`)\cr
#' Errors logged during training.
#' * `train_time` :: `numeric(1)`\cr
#' Training time, in seconds.
#' * `predict_log` :: `NULL` | [`data.table`] with columns `class` (`character`), `msg` (`character`)\cr
#' * `predict_log` :: `NULL` | [`data.table`][data.table::data.table] with columns `class` (`character`), `msg` (`character`)\cr
#' Errors logged during prediction.
#' * `predict_time` :: `NULL` | `numeric(1)`
#' Prediction time, in seconds.
#'
#' @section Fields:
#' Fields inherited from [`PipeOp`], as well as:
#' Fields inherited from [`PipeOp`][mlr3pipelines::PipeOp], as well as:
#' * `learner` :: [`Learner`][mlr3::Learner]\cr
#' [`Learner`][mlr3::Learner] that is being wrapped. Read-only.
#' * `learner_model` :: [`Learner`][mlr3::Learner]\cr
Expand All @@ -61,7 +61,7 @@ PipeOpLearnerPred = R6Class("PipeOpLearnerPred",
#' @description
#' Initialize a Learner Predictor PipeOp. Can be used to wrap trained or untrainted
#' mlr3 learners.
#' @param learner [`Learner`]\cr
#' @param learner [`Learner`][mlr3::Learner]\cr
#' The learner that should be wrapped.
#' @param id [`character`] \cr
#' The `PipeOp`'s id. Defaults to "mcboost".
Expand Down
4 changes: 2 additions & 2 deletions R/PipeOpMCBoost.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
#'
#' @usage NULL
#' @name mlr_pipeops_mcboost
#' @format [`R6Class`] inheriting from [`mlr3pipelines::PipeOp`].
#' @format [`R6Class`][R6::R6Class] inheriting from [`mlr3pipelines::PipeOp`].
#'
#' @description
#' Post-process a learner prediction using multi-calibration.
#' For more details, please refer to \url{https://arxiv.org/pdf/1805.12317.pdf} (Kim et al. 2018)
#' For more details, please refer to \url{http://arxiv.org/pdf/1805.12317} (Kim et al. 2018)
#' or the help for [`MCBoost`].
#' If no `init_predictor` is provided, the preceding learner's predictions
#' corresponding to the `prediction` slot are used as an initial predictor for `MCBoost`.
Expand Down
2 changes: 1 addition & 1 deletion R/PipelineMCBoost.R
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
#' Note: An initial predictor can also be supplied via the `init_predictor` parameter.
#' @param param_vals `list` \cr
#' List of parameter values passed on to `MCBoost$new`.
#' @return (mlr3pipelines) [`Graph`]
#' @return (mlr3pipelines) [`Graph`][mlr3pipelines::Graph]
#' @examples
#' \dontrun{
#' library("mlr3pipelines")
Expand Down
14 changes: 7 additions & 7 deletions R/Predictor.R
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Predictor = R6::R6Class("Predictor",
},
#' @description
#' Predict a dataset with constant predictions.
#' @param data [`data.table`] \cr
#' Prediction data.
#' @param data [`data.table`][data.table::data.table] \cr
#r Prediction data.
#' @param ... [`any`] \cr
#' Not used, only for compatibility with other methods.
predict = function(data, ...) {
Expand Down Expand Up @@ -56,7 +56,7 @@ ConstantPredictor = R6::R6Class("ConstantPredictor",
},
#' @description
#' Predict a dataset with constant predictions.
#' @param data [`data.table`] \cr
#' @param data [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param ... [`any`] \cr
#' Not used, only for compatibility with other methods.
Expand Down Expand Up @@ -95,7 +95,7 @@ LearnerPredictor = R6::R6Class("LearnerPredictor",
},
#' @description
#' Predict a dataset with leaner predictions.
#' @param data [`data.table`] \cr
#' @param data [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param ... [`any`] \cr
#' Not used, only for compatibility with other methods.
Expand Down Expand Up @@ -169,7 +169,7 @@ SubpopPredictor = R6::R6Class("SubpopPredictor",
},
#' @description
#' Predict a dataset with sub-population predictions.
#' @param data [`data.table`] \cr
#' @param data [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param ... [`any`] \cr
#' Not used, only for compatibility with other methods.
Expand Down Expand Up @@ -211,7 +211,7 @@ SubgroupModel = R6::R6Class("SubgroupModel",
},
#' @description
#' Predict a dataset with sub-population predictions.
#' @param data [`data.table`] \cr
#' @param data [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param subgroup_masks [`list`] \cr
#' List of subgroup masks for the data.
Expand Down Expand Up @@ -274,7 +274,7 @@ CVLearnerPredictor = R6::R6Class("CVLearnerPredictor",
},
#' @description
#' Predict a dataset with leaner predictions.
#' @param data [`data.table`] \cr
#' @param data [`data.table`][data.table::data.table] \cr
#' Prediction data.
#' @param ... [`any`] \cr
#' Not used, only for compatibility with other methods.
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

## What does it do?

**mcboost** implements Multi-Calibration Boosting ([Hebert-Johnson et al., 2018](https://proceedings.mlr.press/v80/hebert-johnson18a.html); [Kim et al., 2019](https://arxiv.org/pdf/1805.12317.pdf)) for the multi-calibration of a machine learning model's prediction. Multi-Calibration works best in scenarios where the underlying data & labels are unbiased but a bias is introduced within the algorithm's fitting procedure. This is often the case, e.g. when an algorithm fits a majority population while ignoring or under-fitting minority populations.
**mcboost** implements Multi-Calibration Boosting ([Hebert-Johnson et al., 2018](https://proceedings.mlr.press/v80/hebert-johnson18a.html); [Kim et al., 2019](https://arxiv.org/pdf/1805.12317)) for the multi-calibration of a machine learning model's prediction. Multi-Calibration works best in scenarios where the underlying data & labels are unbiased but a bias is introduced within the algorithm's fitting procedure. This is often the case, e.g. when an algorithm fits a majority population while ignoring or under-fitting minority populations.

For more information and example, see the package's [website](https://mlr-org.github.io/mcboost/).

Expand Down Expand Up @@ -99,7 +99,7 @@ mc$predict_probs(test_data)

### Multi-Calibration

While `mcboost` in its defaults implements Multi-Accuracy ([Kim et al., 2019](https://arxiv.org/pdf/1805.12317.pdf)),
While `mcboost` in its defaults implements Multi-Accuracy ([Kim et al., 2019](http://arxiv.org/pdf/1805.12317)),
it can also multi-calibrate predictors ([Hebert-Johnson et al., 2018](http://proceedings.mlr.press/v80/hebert-johnson18a.html)).
In order to achieve this, we have to set the following hyperparameters:

Expand Down Expand Up @@ -152,7 +152,7 @@ If you use `mcboost`, please cite our package as well as the two papers it is ba

```
@article{pfisterer2021,
author = {Pfisterer, Florian and Kern, Christoph and Dandl, Susanne and Sun, Matthew and
author = {Pfisterer, Florian and Kern, Christoph and Dandl, Susanne and Sun, Matthew and
Kim, Michael P. and Bischl, Bernd},
title = {mcboost: Multi-Calibration Boosting for R},
journal = {Journal of Open Source Software},
Expand Down
8 changes: 4 additions & 4 deletions attic/PipeOpMCBoostSurv.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
#'
#' @usage NULL
#' @name mlr_pipeops_mcboostsurv
#' @format [`R6Class`] inheriting from [`mlr3pipelines::PipeOp`].
#' @format [`R6Class`][R6::R6Class] inheriting from [`PipeOp`][mlr3pipelines::PipeOp].
#'
#' @description
#' Post-process a survival learner prediction using multi-calibration.
#' For more details, please refer to \url{https://arxiv.org/pdf/1805.12317.pdf} (Kim et al. 2018)
#' For more details, please refer to \url{https://arxiv.org/pdf/1805.12317} (Kim et al. 2018)
#' or the help for [`MCBoostSurv`].
#' If no `init_predictor` is provided, the preceding learner's predictions
#' corresponding to the `prediction` slot are used as an initial predictor for `MCBoostSurv`.
Expand Down Expand Up @@ -87,7 +87,7 @@ PipeOpMCBoostSurv = R6Class("PipeOpMCBoostSurv",
super$initialize(id,
param_set = param_set, param_vals = param_vals, packages = c("mlr3proba", "survival"),
input = data.table(
name = c("data", "prediction"),
name = c("data", "prediction"),
train = c("TaskSurv", "TaskSurv"),
predict = c("TaskSurv", "TaskSurv")
),
Expand Down Expand Up @@ -162,7 +162,7 @@ PipeOpMCBoostSurv = R6Class("PipeOpMCBoostSurv",
#' All parameters can be adjusted through the resulting Graph's `param_set`.
#' @param param_vals `list` \cr
#' List of parameter values passed on to `MCBoostSurv$new`
#' @return (mlr3pipelines) [`Graph`]
#' @return (mlr3pipelines) [`Graph`][mlr3pipelines::Graph]
#' @examples
#' library("mlr3pipelines")
#' gr = ppl_mcboostsurv()
Expand Down
6 changes: 1 addition & 5 deletions cran-comments.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,2 @@
## R CMD check
# R CMD CHECK Results

0 errors | 0 warnings | 1 note

New maintainer:
Sebastian Fischer <sebf.fischer@gmail.com>
2 changes: 1 addition & 1 deletion man-roxygen/params_data_label.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#' @param data [`data.table`]\cr
#' @param data [`data.table`][data.table::data.table]\cr
#' Features.
#' @param labels [`numeric`]\cr
#' One-hot encoded labels (of same length as data).
2 changes: 1 addition & 1 deletion man-roxygen/params_data_resid.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#' @param data [`data.table`]\cr
#' @param data [`data.table`][data.table::data.table]\cr
#' Features.
#' @param resid [`numeric`]\cr
#' Residuals (of same length as data).
4 changes: 2 additions & 2 deletions man/AuditorFitter.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/CVLearnerAuditorFitter.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/LearnerAuditorFitter.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions man/MCBoost.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/SubgroupAuditorFitter.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/SubpopAuditorFitter.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 7 additions & 7 deletions man/mlr_pipeops_mcboost.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/ppl_mcboost.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.