% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/simulate_suite.R
\name{simulate_leakage_suite}
\alias{simulate_leakage_suite}
\title{Simulate leakage scenarios and audit results}
\usage{
simulate_leakage_suite(
  n = 500,
  p = 20,
  prevalence = 0.5,
  mode = c("subject_grouped", "batch_blocked", "study_loocv", "time_series"),
  learner = c("glmnet", "ranger"),
  leakage = c("none", "subject_overlap", "batch_confounded", "peek_norm", "lookahead"),
  preprocess = NULL,
  rho = 0,
  K = 5,
  repeats = 1,
  horizon = 0,
  B = 200,
  seeds = 1:10,
  parallel = FALSE,
  signal_strength = 1,
  verbose = FALSE
)
}
\arguments{
\item{n}{Integer scalar. Number of samples to simulate (default 500). Larger
values stabilize the Monte Carlo summary but increase runtime.}

\item{p}{Integer scalar. Number of baseline predictors before any leakage
feature is added (default 20). Increasing \code{p} changes the signal-to-noise
ratio and increases fitting time.}

\item{prevalence}{Numeric scalar in (0, 1). Target prevalence of class 1 in
the simulated outcome (default 0.5). Changing this alters class imbalance
and can affect AUC and the permutation gap.}

\item{mode}{Character scalar. Cross-validation scheme passed to
\code{make_split_plan()}; one of \code{"subject_grouped"},
\code{"batch_blocked"}, \code{"study_loocv"}, \code{"time_series"}.
Defaults to \code{"subject_grouped"}. This controls how samples are grouped
into folds (by subject, batch, study, or time) and therefore which leakage
mechanisms are realistically challenged.}

\item{learner}{Character scalar. Base learner, \code{"glmnet"} (default) or
\code{"ranger"}. Requires the corresponding package in \code{Suggests}.
Switching learners changes the fitted model, runtime, and performance.}

\item{leakage}{Character scalar. Leakage mechanism to inject; one of
\code{"none"}, \code{"subject_overlap"}, \code{"batch_confounded"},
\code{"peek_norm"}, \code{"lookahead"}. Leakage is added as an extra
predictor: \code{"subject_overlap"} adds per-subject mean outcome,
\code{"batch_confounded"} adds per-batch mean outcome, \code{"peek_norm"}
adds the globally normalized (z-scored) outcome, and \code{"lookahead"} adds the next-time
outcome. Changing this controls whether and how leakage is present.}

\item{preprocess}{Optional preprocessing list or recipe passed to
[fit_resample()]. When NULL (default), the simulator uses the
fit_resample defaults; for \code{"peek_norm"} leakage, normalization is
set to \code{"none"} to avoid attenuating the constant leakage feature.}

\item{rho}{Numeric scalar in [-1, 1]. AR(1)-style autocorrelation applied to
each predictor across row order (default 0). Higher absolute values increase
serial correlation and make time-ordered leakage more pronounced.}

\item{K}{Integer scalar. Number of folds/partitions (default 5). Used as the
fold count for \code{"subject_grouped"} and \code{"batch_blocked"}, and as
the number of rolling partitions for \code{"time_series"}. Ignored for
\code{"study_loocv"} (folds equal the number of studies).}

\item{repeats}{Integer scalar >= 1. Number of repeated CV runs for
\code{"subject_grouped"} and \code{"batch_blocked"} (default 1). Increasing
\code{repeats} increases the number of folds and runtime. Ignored for
\code{"study_loocv"} and \code{"time_series"}.}

\item{horizon}{Numeric scalar >= 0. Minimum time gap enforced between train
and test for \code{"time_series"} splits (default 0). Larger values make the
split more conservative and can reduce leakage from temporal proximity.}

\item{B}{Integer scalar >= 1. Number of permutations used by
\code{audit_leakage()} to compute the permutation gap and p-value (default
200). Larger values yield more stable p-values but increase runtime.}

\item{seeds}{Integer vector. Monte Carlo seeds (default \code{1:10}). One row
of output is produced per seed; changing \code{seeds} changes the simulated
datasets and splits.}

\item{parallel}{Logical scalar. If \code{TRUE}, evaluates seeds in parallel
using \code{future.apply} (if installed). Results are identical to sequential
execution; only runtime changes.}

\item{signal_strength}{Numeric scalar. Scales the linear predictor before
sampling outcomes (default 1). Larger values increase class separation and
tend to increase AUC; smaller values make the task harder.}

\item{verbose}{Logical scalar. If \code{TRUE}, prints progress messages for
each seed. Does not affect results.}
}
\value{
A \code{LeakSimResults} data frame with one row per seed and columns:
\itemize{
  \item \code{seed}: seed used for data generation, splitting, and auditing.
  \item \code{metric_obs}: observed CV performance (AUC for this simulation).
  \item \code{gap}: permutation-gap statistic (observed minus permutation mean).
  \item \code{p_value}: permutation p-value for the gap.
  \item \code{leakage}: leakage scenario used.
  \item \code{mode}: CV mode used.
}
Only the permutation-gap summary is returned; fitted models, predictions, and
other audit components are not included.
}
\description{
Simulates synthetic binary classification datasets with optional leakage
mechanisms, fits a model using a leakage-aware cross-validation scheme, and
summarizes the permutation-gap audit for each Monte Carlo seed. The suite is
designed to surface validation failures such as subject overlap across folds,
batch-confounded outcomes, global normalization/summary leakage, and
time-series look-ahead. The output is a per-seed summary of observed CV
performance and its gap versus a label-permutation null; it does not return
fitted models or the full audit object. Results are limited to the built-in
data generator and leakage types implemented here, and should be interpreted
as a simulation-based sanity check rather than a comprehensive leakage
detector for real data.
}
\details{
The generator draws \code{p} standard normal predictors, builds a linear
predictor from the first \code{min(5, p)} features, scales it by
\code{signal_strength}, and samples a binary outcome to achieve the requested
\code{prevalence}. Outcomes are returned as a two-level factor, so the audited
metric is AUC. Simulated metadata include subject, batch, study, and time
fields used by \code{mode} to create leakage-aware splits. Leakage mechanisms
are injected by adding a single extra predictor as described in
\code{leakage}. Parallel execution uses \code{future.apply} when installed and
does not change results.
}
\note{
This function is a general-purpose utility and its data-generation logic
intentionally differs from the custom simulation used in the bioLeak
manuscript (\file{paper/run_simulation.R}). Specific differences:
\itemize{
  \item \strong{peek_norm leakage}: this function uses a z-scored binary
    outcome as the leak feature; the manuscript uses a noisy continuous
    version (\code{as.numeric(y) + rnorm(n, 0, 0.3)}).
  \item \strong{lookahead leakage}: this function shifts the binary outcome
    (\code{c(y[-1], y[n])}); the manuscript shifts a continuous biomarker
    (\code{linpred + noise}).
  \item \strong{signal generation}: this function applies AR correlation to
    predictors via \code{rho}; the manuscript adds AR(1) noise directly to
    the linear predictor.
  \item \strong{audit settings}: the manuscript uses
    \code{perm_refit = FALSE} and \code{perm_stratify = TRUE};
    this function uses \code{perm_refit = "auto"} and the
    \code{perm_stratify} default (\code{FALSE}).
}
Users wishing to reproduce manuscript figures should run
\file{paper/run_simulation.R} directly rather than calling this function.
}
\examples{
\donttest{
  if (requireNamespace("glmnet", quietly = TRUE)) {
    set.seed(1)
    res <- simulate_leakage_suite(
      n = 120, p = 6, prevalence = 0.4,
      mode = "subject_grouped",
      learner = "glmnet",
      leakage = "subject_overlap",
      K = 3, repeats = 1,
      B = 50, seeds = 1,
      parallel = FALSE
    )
    # One row per seed with observed AUC, permutation gap, and p-value
    res
  }
}
}
