% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/int_pctl.R
\name{int_pctl.tune_results}
\alias{int_pctl.tune_results}
\title{Bootstrap confidence intervals for performance metrics}
\usage{
\method{int_pctl}{tune_results}(
  .data,
  metrics = NULL,
  eval_time = NULL,
  times = 1001,
  parameters = NULL,
  alpha = 0.05,
  allow_par = TRUE,
  event_level = "first",
  keep_replicates = FALSE,
  ...
)
}
\arguments{
\item{.data}{A object with class \code{tune_results} where the \code{save_pred = TRUE}
option was used in the control function.}

\item{metrics}{A \code{\link[yardstick:metric_set]{yardstick::metric_set()}}. By default, it uses the same
metrics as the original object.}

\item{eval_time}{A vector of evaluation times for censored regression models.
\code{NULL} is appropriate otherwise. If \code{NULL} is used with censored models, a
evaluation time is selected, and a warning is issued.}

\item{times}{The number of bootstrap samples.}

\item{parameters}{An optional tibble of tuning parameter values that can be
used to filter the predicted values before processing. This tibble should
only have columns for each tuning parameter identifier (e.g. \code{"my_param"}
if \code{tune("my_param")} was used).}

\item{alpha}{Level of significance.}

\item{allow_par}{A logical to allow parallel processing (if a parallel
backend is registered).}

\item{event_level}{A single string. Either \code{"first"} or \code{"second"} to specify
which level of truth to consider as the "event".}

\item{keep_replicates}{A logic for saving the individual estimates from each
bootstrap sample (as a list column called \code{.values}).}

\item{...}{Not currently used.}
}
\value{
A tibble of metrics with additional columns for \code{.lower} and
\code{.upper} (and potentially, \code{.values}).
}
\description{
Using out-of-sample predictions, the bootstrap is used to create percentile
confidence intervals.
}
\details{
For each model configuration (if any), this function takes bootstrap samples
of the out-of-sample predicted values. For each bootstrap sample, the metrics
are computed and these are used to compute confidence intervals.
See \code{\link[rsample:int_pctl]{rsample::int_pctl()}} and the references therein for more details.

Note that the \code{.estimate} column is likely to be different from the results
given by \code{\link[=collect_metrics]{collect_metrics()}} since a different estimator is used. Since
random numbers are used in sampling, set the random number seed prior to
running this function.

The number of bootstrap samples should be large to have reliable intervals.
The defaults reflect the fewest samples that should be used.

The computations for each configuration can be extensive. To increase
computational efficiency parallel processing can be used. The \pkg{future}
package is used here. To execute the resampling iterations in parallel,
specify a \link[future:plan]{plan} with future first. The \code{allow_par} argument
can be used to avoid parallelism.

Also, if a censored regression model used numerous evaluation times, the
computations can take a long time unless the times are filtered with the
\code{eval_time} argument.
}
\examples{
\dontshow{if (!tune:::is_cran_check()) withAutoprint(\{ # examplesIf}
if (rlang::is_installed("modeldata")) {
  data(Sacramento, package = "modeldata")
  library(rsample)
  library(parsnip)

  set.seed(13)
  sac_rs <- vfold_cv(Sacramento)

  lm_res <-
    linear_reg() |>
    fit_resamples(
      log10(price) ~ beds + baths + sqft + type + latitude + longitude,
      resamples = sac_rs,
      control = control_resamples(save_pred = TRUE)
    )

  set.seed(31)
  int_pctl(lm_res)
}
\dontshow{\}) # examplesIf}
}
\references{
Davison, A., & Hinkley, D. (1997). \emph{Bootstrap Methods and their
Application}. Cambridge: Cambridge University Press.
doi:10.1017/CBO9780511802843
}
\seealso{
\code{\link[rsample:int_pctl]{rsample::int_pctl()}}
}
