ssp.glm: Subsampling for Generalized Linear Models

# `ssp.glm`: Subsampling for Generalized Linear Models

This vignette introduces the usage of the `ssp.glm` using logistic regression as an example of generalized linear models (GLM). The statistical theory and algorithms in this implementation can be found in the relevant reference papers.

The log-likelihood function for a GLM is

\[ \max_{\beta} L(\beta) = \frac{1}{N} \sum_{i=1}^N \left\{y_i u(\beta^{\top} x_i) - \psi \left[ u(\beta^{\top} x_i) \right] \right\}, \] where \(u\) and \(\psi\) are known functions depend on the distribution from the exponential family. For the binomial distribution, the log-likelihood function becomes

\[ \max_{\beta} L(\beta) = \frac{1}{N} \sum_{i=1}^N \left[y_i \beta^{\top} x_i - \log\left(1 + e^{\beta^\top x_i}\right) \right]. \]

The idea of subsampling methods is as follows: instead of fitting the model on the size \(N\) full dataset, a subsampling probability is assigned to each observation and a smaller, informative subsample is drawn. The model is then fitted on the subsample to obtain an estimator with reduced computational cost.

## Installation

You can install the development version of subsampling from GitHub with:

    # install.packages("devtools")
    devtools::install_github("dqksnow/Subsampling")

    library(subsampling)

## Terminology

- Full dataset: The whole dataset used as input.

- Full data estimator: The estimator obtained by fitting the model on the full dataset.

- Subsample: A subset of observations drawn from the full dataset.

- Subsample estimator: The estimator obtained by fitting the model on the subsample.

- Subsampling probability (\(\pi\)): The probability assigned to each observation for inclusion in the subsample.

## Example: Logistic Regression with Simulated Data

We introduce the usage of `ssp.glm` with simulated data. \(X\) contains \(d=6\) covariates drawn from multinormal distribution, and \(Y\) is the binary response variable. The full dataset size is \(N = 1 \times 10^4\).

    set.seed(1)
    N <- 1e4
    beta0 <- rep(-0.5, 7)
    d <- length(beta0) - 1
    corr <- 0.5
    sigmax  <- matrix(corr, d, d) + diag(1-corr, d)
    X <- MASS::mvrnorm(N, rep(0, d), sigmax)
    colnames(X) <- paste("V", 1:ncol(X), sep = "")
    P <- 1 - 1 / (1 + exp(beta0[1] + X %*% beta0[-1]))
    Y <- rbinom(N, 1, P)
    data <- as.data.frame(cbind(Y, X))
    formula <- Y ~ .
    head(data)
    #>   Y         V1         V2          V3          V4          V5         V6
    #> 1 1 -1.0918680 -0.4462684 -0.02250989 -0.19626329 -0.67460551 -0.4392570
    #> 2 0 -0.1591053 -0.4748068  0.46515238  0.88370061 -0.05910325  0.1857218
    #> 3 1 -1.6260754 -0.3394421 -0.68490712 -0.55721107  0.01024563 -0.6319413
    #> 4 0  0.1251949  1.5113247  1.38931519  1.24287417  2.48829727  0.5534888
    #> 5 0  0.1931921 -0.1478401 -0.14788926  0.46973556  0.05205022  1.0907459
    #> 6 0 -0.2560258 -1.6065024  0.32710042 -0.04590727 -0.94748664 -1.2310368

## Key Arguments

The function usage is

    ssp.glm(
      formula,
      data,
      subset = NULL,
      n.plt,
      n.ssp,
      family = "quasibinomial",
      criterion = "optL",
      sampling.method = "poisson",
      likelihood = "weighted",
      control = list(...),
      contrasts = NULL,
      ...
      )

The core functionality of `ssp.glm` revolves around three key questions:

- How are subsampling probabilities computed? (Controlled by the `criterion` argument)

- How is the subsample drawn? (Controlled by the `sampling.method` argument)

- How is the likelihood adjusted to correct for bias? (Controlled by the `likelihood` argument)

### `criterion`

The choices of `criterion` include `optA`, `optL`(default), `LCC` and `uniform`. The optimal subsampling criterion `optA` and `optL` are derived by minimizing the asymptotic covariance of subsample estimator, proposed by Wang, Zhu, and Ma (2018). `LCC` and `uniform` are baseline methods.

### `sampling.method`

The options for the `sampling.method` argument include `withReplacement` and `poisson` (default). `withReplacement` stands for drawing `n.ssp` subsamples from full dataset with replacement, using the specified subsampling probabilities. `poisson` stands for drawing subsamples one by one by comparing the subsampling probability with a realization of uniform random variable \(U(0,1)\). The expected number of drawn samples are `n.ssp`. More details see Wang (2019).

### `likelihood`

The available choices for `likelihood` include `weighted` (default) and `logOddsCorrection`. Both of these likelihood functions can derive an unbiased estimator. Theoretical results indicate that `logOddsCorrection` is more efficient than `weighted` in the context of logistic regression. See Wang and Kim (2022).

## Outputs

After drawing subsample, `ssp.glm` utilizes `survey::svyglm` to fit the model on the subsample, which eventually uses `glm`. Arguments accepted by `svyglm` can be passed through `...` in `ssp.glm`.

Below are two examples demonstrating the use of `ssp.glm` with different configurations.

    n.plt <- 200
    n.ssp <- 600
    ssp.results <- ssp.glm(formula = formula,
                           data = data,
                           n.plt = n.plt,
                           n.ssp = n.ssp,
                           family = "quasibinomial",
                           criterion = "optL",
                           sampling.method = "withReplacement",
                           likelihood = "weighted"
                           )
    summary(ssp.results)
    #> Model Summary
    #>
    #> Call:
    #>
    #> ssp.glm(formula = formula, data = data, n.plt = n.plt, n.ssp = n.ssp,
    #>     family = "quasibinomial", criterion = "optL", sampling.method = "withReplacement",
    #>     likelihood = "weighted")
    #>
    #> Subsample Size:
    #>
    #> 1       Total Sample Size 10000
    #> 2 Expected Subsample Size   600
    #> 3   Actual Subsample Size   600
    #> 4   Unique Subsample Size   561
    #> 5  Expected Subample Rate    6%
    #> 6    Actual Subample Rate    6%
    #> 7    Unique Subample Rate 5.61%
    #>
    #> Coefficients:
    #>
    #>           Estimate Std. Error z value Pr(>|z|)
    #> Intercept  -0.5876     0.1328 -4.4249  <0.0001
    #> V1         -0.4725     0.1623 -2.9120   0.0036
    #> V2         -0.5252     0.1699 -3.0915   0.0020
    #> V3         -0.4789     0.1629 -2.9389   0.0033
    #> V4         -0.6400     0.1688 -3.7926   0.0001
    #> V5         -0.4937     0.1785 -2.7658   0.0057
    #> V6         -0.6226     0.1739 -3.5805   0.0003

    ssp.results <- ssp.glm(formula = formula,
                           data = data,
                           n.plt = n.plt,
                           n.ssp = n.ssp,
                           family = "quasibinomial",
                           criterion = "optA",
                           sampling.method = "poisson",
                           likelihood = "logOddsCorrection"
                           )
    summary(ssp.results)
    #> Model Summary
    #>
    #> Call:
    #>
    #> ssp.glm(formula = formula, data = data, n.plt = n.plt, n.ssp = n.ssp,
    #>     family = "quasibinomial", criterion = "optA", sampling.method = "poisson",
    #>     likelihood = "logOddsCorrection")
    #>
    #> Subsample Size:
    #>
    #> 1       Total Sample Size 10000
    #> 2 Expected Subsample Size   600
    #> 3   Actual Subsample Size   634
    #> 4   Unique Subsample Size   634
    #> 5  Expected Subample Rate    6%
    #> 6    Actual Subample Rate 6.34%
    #> 7    Unique Subample Rate 6.34%
    #>
    #> Coefficients:
    #>
    #>           Estimate Std. Error z value Pr(>|z|)
    #> Intercept  -0.5052     0.0837 -6.0334  <0.0001
    #> V1         -0.4788     0.1063 -4.5052  <0.0001
    #> V2         -0.4893     0.1048 -4.6680  <0.0001
    #> V3         -0.4037     0.1060 -3.8079   0.0001
    #> V4         -0.6308     0.1059 -5.9588  <0.0001
    #> V5         -0.5781     0.1091 -5.3012  <0.0001
    #> V6         -0.4679     0.1085 -4.3133  <0.0001

As recommended by `survey::svyglm`, when working with binomial models, it is advisable to use use `family=quasibinomial()` to avoid a warning issued by `glm`. Refer to svyglm() help documentation Details. The ‘quasi’ version of the family objects provide the same point estimates.

### Returned object

The object returned by `ssp.glm` contains estimation results and indices of the drawn subsample in the full dataset.

    names(ssp.results)
    #>  [1] "model.call"            "coef.plt"              "coef.ssp"
    #>  [4] "coef"                  "cov.plt"               "cov.ssp"
    #>  [7] "cov"                   "index.plt"             "index"
    #> [10] "N"                     "subsample.size.expect" "comp.time"
    #> [13] "terms"

Some key returned variables:

- `index.plt` and `index` are the row indices of drawn pilot subsamples and optimal subsamples in the full data.

- `coef.ssp` is the subsample estimator for \(\beta\) and `coef` is the linear combination of `coef.plt` (pilot estimator) and `coef.ssp`.

- `cov.ssp` and `cov` are estimated covariance matrices of `coef.ssp` and `coef`.

The coefficients and standard errors printed by `summary()` are `coef` and the square root of `diag(cov)`. See the help documentation of `ssp.glm` for details.

## Other Families

We also provide examples for poisson regression and gamma regression in the help documentation of `ssp.glm`. Note that `likelihood = logOddsCorrection` is currently implemented only for logistic regression (family = `binomial` or `quasibonomial`).

## References

Wang, HaiYing. 2019. “More Efficient Estimation for Logistic Regression with Optimal Subsamples.” Journal of Machine Learning Research 20 (132): 1–59.
Wang, HaiYing, and Jae Kwang Kim. 2022. “Maximum Sampled Conditional Likelihood for Informative Subsampling.” Journal of Machine Learning Research 23 (332): 1–50.
Wang, HaiYing, Rong Zhu, and Ping Ma. 2018. “Optimal Subsampling for Large Sample Logistic Regression.” Journal of the American Statistical Association 113 (522): 829–44.