#' Lagrangian Multiplier Smoothing Splines: Mathematical Details
#'
#' @description
#' This document provides the mathematical and implementation details for
#' Lagrangian Multiplier Smoothing Splines as implemented in \pkg{lgspline}.
#'
#' This package provides exhaustive resources for fitting multivariate smoothing
#' smoothing splines with a monomial basis and analytical form cubic smoothing
#' spline penalty.
#'
#' The material is presented such that a programmer or statistician of
#' reasonable experience and background can understand and implement the
#' procedure from scratch, and also potentially critique some of the modelling
#' choices that went into designing this package.
#'
#' Informally, \pkg{lgspline} answers the following question:
#' How can we best adapt a useful functionality of basis splines, under the
#' alternative interpretation of smoothing as explicit external constraints instead?
#'
#' The obvious benefit is a much more flexible and interpretable final model that for
#' non-experienced users is simply easier to understand without post-hoc processing, and
#' for experienced users can be used to customize models more easily.
#'
#' The drawback is that the interpretation of constraints as external adds a new layer of complexity
#' to each step of the model fitting process, whereas for implicit design matrix
#' construction these complications are bypassed.
#'
#' While it is true a B-spline can always be converted back into monomial form,
#' tensor-product splines that generalize this to multiple dimensions often
#' explodes the number and degree of interaction terms, the conversion may not
#' be computationally stable, and it is not available in standard software.
#'
#' @section Statistical Problem Formulation:
#'
#' Consider an \eqn{N \times q} matrix of predictors
#' \eqn{\mathbf{T} = (\mathbf{t}_1, \dots, \mathbf{t}_N)^{\top}} and an
#' \eqn{N \times 1} response vector \eqn{\mathbf{y} = (y_1, \dots, y_N)^{\top}}.
#' We assume the relationship follows a generalized linear model with unknown
#' smooth function \eqn{f}:
#' \deqn{y_i \sim \mathcal{D}(g^{-1}(f(\mathbf{t}_i)),\, \sigma^2)}
#' where \eqn{\mathcal{D}} is a distribution (e.g. exponential family or related) with mean
#' \eqn{\mu_i = g^{-1}(f(\mathbf{t}_i))}, link function \eqn{g(\cdot)}, and
#' dispersion parameter \eqn{\sigma^2}. For Gaussian response with identity
#' link, observations are independently distributed as
#' \eqn{y_i \mid \mathbf{t}_i, \sigma^2 \sim \mathcal{N}(f(\mathbf{t}_i), \sigma^2)}.
#'
#' The objective is to estimate \eqn{f} by:
#' \enumerate{
#'   \item Partitioning the predictor space into \eqn{K+1} mutually exclusive regions.
#'   \item Fitting local polynomial models within each partition.
#'   \item Enforcing smoothness at partition boundaries via Lagrangian multipliers.
#'   \item Penalizing the integrated squared second derivative to discourage roughness.
#' }
#'
#' Unlike other smoothing spline formulations, no post-fitting algebraic
#' rearrangement or disentanglement of a spline basis is needed to obtain
#' interpretable models. The polynomial expansions are homogeneous across
#' partitions, and the relationship between predictor and response is explicit
#' at the coefficient level.
#'
#' To anchor the notation, in the single-predictor cubic case one would write
#' \deqn{\hat{f}(t_i) = \hat{\beta}_{(0)} + \hat{\beta}_{(1)}t_i +
#'   \hat{\beta}_{(2)}t_i^2 + \hat{\beta}_{(3)}t_i^3 =
#'   \mathbf{x}_i^{\top}\hat{\boldsymbol{\beta}},}
#' where \eqn{\mathbf{x}_i = (1, t_i, t_i^2, t_i^3)^{\top}}. The LMSS
#' formulation preserves exactly this kind of polynomial representation, but now
#' does so within each partition and then forces neighboring pieces to agree in
#' the smoothness conditions described below.
#'
#' Core notation used throughout:
#' \itemize{
#'   \item \eqn{\mathbf{y}_{(N \times 1)}}: Response vector.
#'   \item \eqn{\mathbf{T}_{(N \times q)}}: Matrix of predictors.
#'   \item \eqn{\mathbf{X}_{(N \times P)}}: Block-diagonal matrix of polynomial
#'     expansions, with diagonal blocks \eqn{\mathbf{X}_k} of dimension
#'     \eqn{n_k \times p}.
#'   \item \eqn{\boldsymbol{\Lambda}_{(P \times P)}}: Block-diagonal penalty matrix,
#'     with blocks \eqn{\boldsymbol{\Lambda}_k} of dimension \eqn{p \times p}.
#'   \item \eqn{\hat{\boldsymbol{\beta}}_{(P \times 1)}}: Unconstrained penalized
#'     estimate.
#'   \item \eqn{\tilde{\boldsymbol{\beta}}_{(P \times 1)}}: Constrained coefficient
#'     estimates.
#'   \item \eqn{\mathbf{G}_{(P \times P)}}: Block-diagonal matrix with blocks
#'     \eqn{\mathbf{G}_k = (\mathbf{X}_k^{\top}\mathbf{W}_k\mathbf{D}_k\mathbf{X}_k + \boldsymbol{\Lambda}_k)^{-1}},
#'     where \eqn{\mathbf{W}_k} and \eqn{\mathbf{D}_k} are defined below.
#'   \item \eqn{\mathbf{A}_{(P \times r)}}: Constraint matrix encoding smoothness
#'     conditions. Reduced to linearly independent columns via pivoted QR
#'     decomposition.
#'   \item \eqn{\mathbf{U}_{(P \times P)}}:
#'     \eqn{\mathbf{I} - \mathbf{G}\mathbf{A}(\mathbf{A}^{\top}\mathbf{G}\mathbf{A})^{-1}\mathbf{A}^{\top}}.
#'   \item \eqn{\mathbf{D}_{(N \times N)}}: Diagonal matrix of user-supplied
#'     observation weights (\code{observation_weights} or \code{weights}).
#'     Defaults to the identity. These play the role of prior precision on
#'     individual observations: a weight of 2 is equivalent to seeing that
#'     observation twice.
#'   \item \eqn{\mathbf{W}_{(N \times N)}}: Diagonal matrix of GLM working
#'     weights. In the implementation these diagonal entries are whatever is
#'     returned by \code{glm_weight_function}; by default this is
#'     \code{family$variance(mu)}, optionally multiplied by user-supplied
#'     observation weights. For Gaussian response with identity link,
#'     \eqn{\mathbf{W} = \mathbf{I}}. For other families,
#'     \eqn{\mathbf{W}} depends on the current fitted values and is updated at
#'     each Newton-Raphson iteration. For the common canonical families used
#'     by default, this matches the familiar Fisher-scoring weighting role.
#'   \item \eqn{\mathbf{V}_{(N \times N)}}: Correlation matrix of errors.
#'     When no correlation structure is specified, \eqn{\mathbf{V} = \mathbf{I}}.
#'     Otherwise supplied via \code{VhalfInv} or estimated through
#'     \code{VhalfInv_fxn}.
#' }
#'
#' In the Gaussian identity case with unit weights and no correlation,
#' \eqn{\mathbf{G}_k = (\mathbf{X}_k^{\top}\mathbf{X}_k + \boldsymbol{\Lambda}_k)^{-1}}
#' and most formulas simplify accordingly. When \eqn{\mathbf{D}} or
#' \eqn{\mathbf{W}} appear in a formula, the product \eqn{\mathbf{W}\mathbf{D}}
#' means ``GLM working weights times observation weights''; whenever one of
#' them is the identity it drops out.
#'
#' Before these quantities reach the main fitting stage, the user-facing inputs
#' are parsed, standardized, and organized by \code{\link{process_input}}. When
#' the formula interface is used and \code{auto_encode_factors = TRUE}, that
#' preprocessing step also relies on helpers such as \code{\link{create_onehot}}
#' to encode factor levels before the design reaches \code{\link{lgspline.fit}()}.
#' The notation in the remainder of this document therefore refers to the internal
#' objects that actually enter \code{\link{lgspline.fit}()}, not necessarily the raw
#' objects originally supplied by the user.
#'
#' From the user side, many of the arguments that control these internal objects
#' can be supplied either individually or through the grouped lists
#' \code{penalty_args}, \code{tuning_args}, \code{expansion_args},
#' \code{constraint_args}, \code{qp_args}, \code{parallel_args},
#' \code{covariance_args}, \code{return_args}, and \code{glm_args}, as
#' documented in \code{\link{lgspline}}. These grouped lists are unpacked before
#' dispatch into the same fitting pipeline, so they are a convenience layer
#' rather than a separate modeling abstraction. A closely related exploratory
#' mode is \code{dummy_fit = TRUE} in \code{\link{lgspline}} or
#' \code{\link{lgspline.fit}}, which runs the preprocessing, partition
#' construction, expansion building, and penalty setup without solving for
#' nonzero coefficients, making it a practical way to inspect objects such as
#' \code{X}, \code{A}, the returned \code{make_partition_list} from
#' \code{\link{make_partitions}}, and the assembled \code{penalties} from
#' \code{\link{compute_Lambda}} before a full fit.
#'
#'
#' @section Model Formulation and Estimation:
#'
#' \subsection{Piecewise Polynomial Structure}{
#' For \eqn{K} knots (one predictor) or \eqn{K+1} partitions (multiple
#' predictors) there are \eqn{K+1} mutually exclusive partitions
#' \eqn{\mathcal{P}_0, \dots, \mathcal{P}_{K}}. Each observation \eqn{i}
#' belongs to exactly one partition. Within partition \eqn{k}, the function is
#' represented as a polynomial of degree \eqn{p-1} in each predictor:
#' \deqn{\hat{f}_k(\mathbf{t}) = \mathbf{x}^{\top}\tilde{\boldsymbol{\beta}}_k}
#' where \eqn{\mathbf{x}} collects the polynomial basis terms (intercept,
#' linear, quadratic, cubic, and optionally quartic and interaction terms) and
#' \eqn{\tilde{\boldsymbol{\beta}}_k} are the corresponding coefficients. In
#' one predictor, the same idea can be written more explicitly as
#' \deqn{\hat{f}(t_i) = \sum_{k=0}^{K}
#'   \mathbf{x}_{ik}^{\top}\hat{\boldsymbol{\beta}}_k
#'   \mathbf{1}(t_i \in \mathcal{P}_k),}
#' which highlights that the unconstrained problem is just a collection of
#' local polynomial regressions. The expansions are homogeneous across
#' partitions, so coefficients are directly comparable. This is implemented via
#' \code{\link{get_polynomial_expansions}}.
#'
#' The exact contents of \eqn{\mathbf{x}} are controlled by the basis-expansion
#' arguments documented in \code{\link{lgspline}}: \code{include_quadratic_terms},
#' \code{include_cubic_terms}, \code{include_quartic_terms},
#' \code{include_2way_interactions}, \code{include_3way_interactions},
#' \code{include_quadratic_interactions}, \code{exclude_interactions_for},
#' \code{exclude_these_expansions}, and \code{custom_basis_fxn}. Likewise,
#' \code{just_linear_with_interactions} and
#' \code{just_linear_without_interactions} determine which predictors remain
#' structurally linear even though they still participate in the same
#' partition-wise polynomial bookkeeping described here.
#'
#' Letting \eqn{p} denote the number of basis terms per partition,
#' \eqn{P = p(K+1)} is the total number of coefficients. The full design matrix
#' \eqn{\mathbf{X}} and penalty matrix \eqn{\boldsymbol{\Lambda}} are
#' block-diagonal with \eqn{K+1} blocks, so unconstrained estimation reduces to
#' \eqn{K+1} independent penalized regressions, which appears as follows for the identity link case:
#' \deqn{\hat{\boldsymbol{\beta}}_k = \mathbf{G}_k \mathbf{X}_k^{\top} \mathbf{W}_k\mathbf{D}_k\mathbf{y}_k, \quad
#'   \mathbf{G}_k = (\mathbf{X}_k^{\top}\mathbf{W}_k\mathbf{D}_k\mathbf{X}_k + \boldsymbol{\Lambda}_k)^{-1}.}
#' For Gaussian identity with unit weights this reduces to the familiar
#' \eqn{\mathbf{G}_k = (\mathbf{X}_k^{\top}\mathbf{X}_k + \boldsymbol{\Lambda}_k)^{-1}}.
#' The block-diagonal structure means these can be computed in parallel across
#' partitions. In the user-facing interface this is realized by supplying a
#' cluster through \code{cl}, optionally controlling work splitting with
#' \code{chunk_size}, and enabling stages such as \code{parallel_eigen}
#' for the eigendecompositions and, in non-Gaussian Path 3, \code{parallel_unconstrained}
#' for the partition-wise unconstrained fits; nearby stages can likewise use
#' \code{parallel_penalty} and \code{parallel_make_constraint}. The eigenvalue
#' decomposition and matrix square roots of each \eqn{\mathbf{G}_k} are
#' computed by \code{\link{compute_G_eigen}}, and can be returned in the fitted
#' object as \code{G} and \code{Ghalf} when \code{return_G = TRUE} and
#' \code{return_Ghalf = TRUE}.
#'
#' Fitted values for the canonical Gaussian case appear as
#' \eqn{\tilde{\mathbf{y}} = \mathbf{X}\tilde{\boldsymbol{\beta}} = \mathbf{H}\mathbf{y}}
#' for \eqn{\mathbf{H} = \mathbf{X}\mathbf{U}\mathbf{G}\mathbf{X}^{\top}}.
#' }
#'
#' \subsection{Smoothing Constraints and the Constraint Matrix}{
#' Without further intervention the piecewise polynomial will be discontinuous.
#' The central idea of LMSS is that smoothness is not hidden inside a special
#' basis, but instead imposed directly where neighboring partitions meet.
#' At each knot \eqn{t_{k,k+1}} between neighboring partitions \eqn{k} and
#' \eqn{k+1}, up to three smoothing constraints are imposed:
#' \enumerate{
#'   \item Continuity: \eqn{\mathbf{x}_{k,k+1}^{\top}\boldsymbol{\beta}_k = \mathbf{x}_{k,k+1}^{\top}\boldsymbol{\beta}_{k+1}}.
#'   \item First-derivative continuity: \eqn{\mathbf{x}_{k,k+1}^{\prime\top}\boldsymbol{\beta}_k = \mathbf{x}_{k,k+1}^{\prime\top}\boldsymbol{\beta}_{k+1}}.
#'   \item Second-derivative continuity: \eqn{\mathbf{x}_{k,k+1}^{\prime\prime\top}\boldsymbol{\beta}_k = \mathbf{x}_{k,k+1}^{\prime\prime\top}\boldsymbol{\beta}_{k+1}}.
#' }
#' where \eqn{\mathbf{x}^{\prime}} and \eqn{\mathbf{x}^{\prime\prime}} are
#' elementwise first and second derivatives of the basis with respect to
#' \eqn{\mathbf{t}}. For the familiar cubic single-predictor basis
#' \eqn{\mathbf{x} = (1, t, t^2, t^3)^{\top}}, these derivative vectors are
#' \deqn{\mathbf{x}' = (0, 1, 2t, 3t^2)^{\top}, \qquad
#'   \mathbf{x}'' = (0, 0, 2, 6t)^{\top}.}
#' With \eqn{K} knots this yields up to \eqn{3K} scalar
#' constraints (for a single predictor; more for multiple predictors with
#' interactions), collected as linear equations
#' \eqn{\mathbf{A}^{\top}\boldsymbol{\beta} = \mathbf{0}}
#' in a \eqn{P \times r} matrix \eqn{\mathbf{A}}. The constraint matrix is
#' built by \code{\link{make_constraint_matrix}} and returned in the fitted
#' object as \code{A}.
#'
#' In higher dimensions or with many partitions, the constraints can become
#' over-specified and force the model toward a single global polynomial. In these
#' cases it is recommended to drop second-derivative constraints or include quartic
#' terms, allowing the model to fit a richer surface while maintaining
#' perceived smoothness at knots. The appropriate constraint level can be
#' controlled via \code{include_constrain_fitted},
#' \code{include_constrain_first_deriv}, and
#' \code{include_constrain_second_deriv}.
#' The companion flag \code{include_constrain_interactions} determines whether
#' the analogous mixed-partial constraints are imposed for interaction terms,
#' and \code{no_intercept} adds the special homogeneous equality constraint that
#' fixes the intercept at zero (the same behavior triggered by using
#' \code{0 +} in the formula interface).
#'
#' Before computing the projection \eqn{\mathbf{U}}, the constraint matrix is
#' reduced to a linearly independent subset of columns via pivoted QR
#' decomposition. This avoids numerical instability from redundant constraints
#' and ensures \eqn{\mathbf{A}^{\top}\mathbf{G}\mathbf{A}} is invertible.
#' }
#'
#' \subsection{Lagrangian Projection}{
#' The constrained estimate is derived via Lagrangian multipliers. Define the
#' \eqn{P \times P} projection matrix:
#' \deqn{\mathbf{U} = \mathbf{I} - \mathbf{G}\mathbf{A}(\mathbf{A}^{\top}\mathbf{G}\mathbf{A})^{-1}\mathbf{A}^{\top}.}
#' Then the constrained estimate is:
#' \deqn{\tilde{\boldsymbol{\beta}} = \mathbf{U}\hat{\boldsymbol{\beta}}.}
#' The matrix \eqn{\mathbf{U}} has the property that
#' \eqn{\mathbf{U}\mathbf{G}\mathbf{U}^{\top} = \mathbf{U}\mathbf{G}}, which is
#' used extensively in variance estimation and posterior draws. In words, the
#' unconstrained penalized estimate is projected back into the coefficient space
#' that satisfies the smoothness restrictions, and all subsequent uncertainty
#' calculations inherit that same projected geometry.
#' The projection is computed via \code{\link{get_U}} and, when requested,
#' returned in the fitted object as \code{U} through \code{return_U = TRUE}.
#'
#' When the constraints are inhomogeneous
#' (\eqn{\mathbf{A}^{\top}\boldsymbol{\beta} = \mathbf{c}} with
#' \eqn{\mathbf{c} \neq \mathbf{0}}), a particular solution
#' \eqn{\boldsymbol{\beta}_0} satisfying
#' \eqn{\mathbf{A}^{\top}\boldsymbol{\beta}_0 = \mathbf{c}} is added back
#' after projection, yielding the full Lagrangian solution
#' \eqn{\mathbf{U}\hat{\boldsymbol{\beta}} + (\mathbf{I} - \mathbf{U})\boldsymbol{\beta}_0}.
#' In \code{\link{lgspline}} and \code{\link{lgspline.fit}}, users realize
#' this by supplying extra equality columns in \code{constraint_vectors}
#' together with matching right-hand sides in \code{constraint_values};
#' \code{null_constraint} provides the alternate shorthand documented in
#' \code{\link{lgspline}} when \code{constraint_vectors} is supplied and
#' \code{constraint_values} is left empty.
#'
#' In practice \eqn{\mathbf{U}} is never explicitly formed during fitting.
#' The constrained estimate is obtained from a transformed OLS residual problem (the
#' \eqn{\mathbf{G}^{1/2}\mathbf{r}^{*}} trick) in four steps:
#' \enumerate{
#'   \item Obtain the unconstrained partition-wise
#'     unconstrained estimate \eqn{\hat{\boldsymbol{\beta}}}.
#'   \item Set \eqn{\mathbf{y}^{*} = \mathbf{G}^{-1/2}\hat{\boldsymbol{\beta}}} and
#'     \eqn{\mathbf{X}^{*} = \mathbf{G}^{1/2}\mathbf{A}}.
#'   \item Fit the linear model \eqn{\mathbb{E}[\mathbf{y}^{*}] = \mathbf{X}^{*}\boldsymbol{\gamma}}
#'     by OLS using QR decomposition.
#'   \item Compute the residuals \eqn{\mathbf{r}^{*} = \mathbf{y}^{*} - \mathbf{X}^{*}(\mathbf{X}^{*\top}\mathbf{X}^{*})^{-1}\mathbf{X}^{*\top}\mathbf{y}^{*}} from that transformed OLS
#'     fit and recover the constrained estimate by
#'     \eqn{\tilde{\boldsymbol{\beta}} = \mathbf{G}^{1/2}\mathbf{r}^{*}}.
#' }
#'
#' A scaling factor \eqn{1/\sqrt{K+1}} is applied to both
#' \eqn{\mathbf{X}^{*}} and \eqn{\mathbf{y}^{*}} prior to the OLS call
#' and divided out afterward, improving numerical conditioning when the
#' constraint matrix has many rows.
#'
#' The most expensive operation in this approach is the QR decomposition of the
#' \eqn{P \times r} matrix \eqn{\mathbf{X}^{*} = \mathbf{G}^{1/2}\mathbf{A}},
#' which is far cheaper than working with the full \eqn{P \times P} system directly.
#' Without correlation or SQP constraints, \eqn{\mathbf{G}} is stored and operated upon as a
#' list of \eqn{K+1} small \eqn{p \times p} matrices rather than the full
#' \eqn{P \times P} block-diagonal, saving substantial memory when \eqn{K}
#' is large and allowing for parallelism.
#'
#' When correlation is present, \eqn{\mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X}}
#' is no longer block-diagonal, so the full-dimensional system must be handled
#' directly unless the Woodbury acceleration
#' (see \code{\link{.woodbury_decompose_V}}) applies.
#' When additional inequality constraints are present, the code either augments the equality system with a
#' partition-wise active-set refinement (block-separable case) or falls back to
#' dense SQP via the Goldfarb-Idnani dual active-set method implemented in
#' \code{\link[quadprog]{solve.QP}}.
#' }
#'
#' @section GLM Extension and Iterative Updates:
#'
#' \subsection{Working Quantities}{
#' For GLM responses with mean \eqn{\mu_i = g^{-1}(\eta_i)} and working
#' weights \eqn{w_i = [V(\mu_i)\{g'(\mu_i)\}^2]^{-1}}, the penalized
#' information becomes
#' \eqn{\mathbf{G}_k^{-1} = \mathbf{X}_k^{\top}\mathbf{W}_k\mathbf{X}_k
#' + \boldsymbol{\Lambda}_k} with
#' \eqn{\mathbf{W}_k = \mathrm{diag}(w_i : i \in \mathcal{P}_k)}.
#' Because \eqn{\mathbf{W}} is diagonal,
#' \eqn{\mathbf{X}^{\top}\mathbf{W}\mathbf{X}} remains block-diagonal and
#' the four-step procedure carries through with
#' \eqn{\mathbf{G}_k = (\mathbf{X}_k^{\top}\mathbf{W}_k\mathbf{X}_k +
#' \boldsymbol{\Lambda}_k)^{-1}} in place of
#' \eqn{(\mathbf{X}_k^{\top}\mathbf{X}_k + \boldsymbol{\Lambda}_k)^{-1}}.
#' Under the default \code{glm_weight_function}, the diagonal entries reduce
#' to \code{family$variance(mu)} (optionally scaled by observation weights),
#' which for canonical families is the usual Fisher scoring weight.
#'
#' The partition-wise unconstrained estimates are obtained by
#' \code{unconstrained_fit_fxn}, by default
#' \code{\link{unconstrained_fit_default}}, which initializes via an augmented
#' ridge trick (appending \eqn{\boldsymbol{\Lambda}^{1/2}} as
#' pseudo-observations to \code{glm.fit}) then refines using
#' \code{\link{damped_newton_r}} with \code{\link{nr_iterate}}. For
#' non-canonical log-link gamma regression, the \code{keep_weighted_Lambda = TRUE}
#' option correctly returns the augmented ridge estimate directly.
#' }
#'
#' \subsection{Three Fitting Paths}{
#'
#' \strong{Path 1: Correlation structure present.}
#' When a working correlation \eqn{\mathbf{V}} is present, such as for
#' marginal means models or generalized estimating equation (GEE)-like models,
#' \eqn{\mathbf{V}^{-1}} couples observations across partitions, so
#' partition-wise fitting is not directly available. Two sub-paths handle
#' this.
#'
#' \emph{Path 1a} (Gaussian identity + GEE) solves the whitened system
#' directly via
#' \eqn{\tilde{\mathbf{G}} =
#' (\tilde{\mathbf{X}}^{\top}\tilde{\mathbf{X}} +
#' \boldsymbol{\Lambda}_{\mathrm{block}})^{-1}} where
#' \eqn{\tilde{\mathbf{X}} = \mathbf{V}^{-1/2}\mathbf{X}}, then applies
#' the Lagrangian projection in the full \eqn{P}-space.
#' See \code{\link{.get_B_gee_gaussian}}.
#'
#' \emph{Path 1b} (non-Gaussian GEE) uses a damped SQP iteration on the
#' whitened system. The first iterate is a constrained Newton step from the
#' projection matrix \eqn{\mathbf{U}}; subsequent iterates solve the
#' quadratic subproblem via \code{\link[quadprog]{solve.QP}}.
#' See \code{\link{.get_B_gee_glm}}.
#'
#' Both sub-paths have Woodbury-accelerated variants described below.
#'
#' \strong{Path 2: Gaussian identity, no correlation.}
#' The constrained estimate is obtained by a single Lagrangian projection
#' from the per-partition penalized least-squares cross-products (the four
#' steps in closed form). No outer iteration is needed. When \eqn{K = 0} and
#' there are no additional constraints, this reduces to the ordinary
#' penalized closed form
#' \eqn{\hat{\boldsymbol{\beta}} = \mathbf{G}\mathbf{X}^{\top}\mathbf{y}}.
#' Implemented in \code{\link{.get_B_gaussian_nocorr}}.
#'
#' \strong{Path 3: Non-Gaussian GLM, no correlation.}
#' Unconstrained estimates are obtained separately within each partition,
#' then projected onto the constraint space. When the link is non-identity,
#' the information \eqn{\mathbf{G}} depends on the current fitted values
#' through the working weights, so the projection must be iterated. The
#' unconstrained anchor \eqn{\hat{\boldsymbol{\beta}}} is held fixed while
#' \eqn{\mathbf{G}} is recomputed at each iterate's constrained estimate:
#' \deqn{\tilde{\boldsymbol{\beta}}^{(s+1)} =
#'   \mathbf{U}^{(s)}\hat{\boldsymbol{\beta}}, \quad
#'   \mathbf{U}^{(s)} = \mathbf{I} - \mathbf{G}^{(s)}\mathbf{A}
#'   (\mathbf{A}^{\top}\mathbf{G}^{(s)}\mathbf{A})^{-1}\mathbf{A}^{\top}.}
#' Iteration stops when the mean absolute coefficient change falls below
#' \code{tol}, or when the update begins increasing (in which case the
#' previous iterate is restored). The recomputation of weighted Gram
#' matrices, Schur corrections, and square-root information factors at each
#' step is handled by \code{.solver_recompute_G_at_estimate}. Implemented
#' in \code{\link{.get_B_glm_nocorr}}.
#' }
#'
#' \subsection{Woodbury Acceleration for Structured Correlation}{
#' For structured correlation, write the penalized information as
#' \deqn{\mathbf{G}^{-1}
#'   = \mathbf{G}_{\mathrm{on}}^{-1} + \mathbf{G}_{\mathrm{off}}^{-1},}
#' where \eqn{\mathbf{G}_{\mathrm{on}}^{-1}} contains the \eqn{K+1}
#' block-diagonal \eqn{p \times p} pieces of
#' \eqn{\mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X} + \boldsymbol{\Lambda}},
#' and \eqn{\mathbf{G}_{\mathrm{off}}^{-1}} contains the cross-partition part.
#' Without correlation, \eqn{\mathbf{G}_{\mathrm{off}}^{-1} = \mathbf{0}}.
#'
#' When the cross-partition part is low rank, the code factors it as
#' \deqn{\mathbf{G}_{\mathrm{off}}^{-1} = \mathbf{E}\mathbf{J}\mathbf{E}^{\top},}
#' with \eqn{\mathbf{E}} of size \eqn{P \times r} and \eqn{\mathbf{J}}
#' diagonal with entries \eqn{\pm 1}. Then
#' \deqn{\mathbf{G}
#'   = \mathbf{G}_{\mathrm{on}}
#'   - \mathbf{G}_{\mathrm{on}}\mathbf{E}
#'   (\mathbf{J}^{-1} + \mathbf{E}^{\top}\mathbf{G}_{\mathrm{on}}\mathbf{E})^{-1}
#'   \mathbf{E}^{\top}\mathbf{G}_{\mathrm{on}}.}
#' The only dense inverse is the small \eqn{r \times r} matrix
#' \eqn{(\mathbf{J}^{-1} + \mathbf{E}^{\top}\mathbf{G}_{\mathrm{on}}\mathbf{E})^{-1}}.
#'
#' Define \eqn{\mathbf{N} = \mathbf{G}_{\mathrm{on}}^{1/2}\mathbf{E}} and a
#' thin QR decomposition \eqn{\mathbf{N} = \mathbf{Q}\mathbf{R}}. With
#' \eqn{\mathbf{P} = \mathbf{Q}\mathbf{Q}^{\top}} and
#' \eqn{\mathbf{S} = \mathbf{I}_r + \mathbf{R}\mathbf{J}\mathbf{R}^{\top}},
#' the supplement writes
#' \deqn{\mathbf{F}
#'   = (\mathbf{I}_P + \mathbf{N}\mathbf{J}\mathbf{N}^{\top})^{-1}
#'   = \mathbf{I}_P - \mathbf{P} + \mathbf{Q}\mathbf{S}^{-1}\mathbf{Q}^{\top},}
#' so that
#' \deqn{\mathbf{G}^{1/2}
#'   = \mathbf{G}_{\mathrm{on}}^{1/2}\mathbf{F}^{1/2}, \qquad
#'   \mathbf{G}^{-1/2}
#'   = \mathbf{F}^{-1/2}\mathbf{G}_{\mathrm{on}}^{-1/2},}
#' with
#' \deqn{\mathbf{F}^{1/2}
#'   = (\mathbf{I}_P - \mathbf{P}) +
#'   \mathbf{Q}\mathbf{S}^{-1/2}\mathbf{Q}^{\top}, \qquad
#'   \mathbf{F}^{-1/2}
#'   = (\mathbf{I}_P - \mathbf{P}) +
#'   \mathbf{Q}\mathbf{S}^{1/2}\mathbf{Q}^{\top}.}
#' This preserves the same transformed OLS projection used in the
#' independent case:
#' \deqn{\mathbf{y}^{*} = \mathbf{G}^{1/2}\mathbf{w}, \qquad
#'   \mathbf{X}^{*} = \mathbf{G}^{1/2}\mathbf{A}, \qquad
#'   \tilde{\boldsymbol{\beta}} = \mathbf{G}^{1/2}\mathbf{r}^{*},}
#' where \eqn{\mathbf{w} = \mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{y}}.
#' The full-space operations remain the QR on \eqn{\mathbf{X}^{*}} and a
#' rank-\eqn{r} correction through \eqn{\mathbf{Q}}.
#'
#' \strong{Implementation map.} The code uses the same algebra but stores a
#' numerically convenient representation. The correspondence is:
#' \itemize{
#'   \item manuscript \eqn{\mathbf{E}} \eqn{\leftrightarrow} helper output
#'     \code{E} (legacy alias \code{L});
#'   \item manuscript diagonal sign matrix \eqn{\mathbf{J}}
#'     \eqn{\leftrightarrow} helper output \code{J_signs}
#'     (legacy alias \code{S_signs});
#'   \item manuscript inverse
#'     \eqn{(\mathbf{J}^{-1} + \mathbf{E}^{\top}\mathbf{G}_{\mathrm{on}}\mathbf{E})^{-1}}
#'     \eqn{\leftrightarrow} helper output \code{inner_inv}
#'     (legacy alias \code{M});
#'   \item manuscript projector basis \eqn{\mathbf{Q}}
#'     \eqn{\leftrightarrow} internal basis \code{U_Q}
#'     (alias \code{Q_basis});
#'   \item manuscript projector \eqn{\mathbf{P} = \mathbf{Q}\mathbf{Q}^{\top}}
#'     is not stored explicitly, but is represented through rank-\eqn{r}
#'     multiplies involving \code{U_Q};
#'   \item manuscript factors \eqn{\mathbf{F}^{1/2}} and
#'     \eqn{\mathbf{F}^{-1/2}} are stored internally as
#'     \eqn{\mathbf{I} - \mathbf{U}_Q\mathbf{C}\mathbf{U}_Q^{\top}} and
#'     \eqn{\mathbf{I} + \mathbf{U}_Q\mathbf{C}_{\mathrm{inv}}\mathbf{U}_Q^{\top}},
#'     where \code{C} and \code{C_inv} are diagonal.
#' }
#' These are just storage choices; the notation matches the supplement.
#'
#' For the non-Gaussian Woodbury path
#' (\code{\link{.get_B_gee_glm_woodbury}}), the fixed correlation
#' perturbation \eqn{\mathbf{V}^{-1} - \mathbf{I}} and its product with
#' \eqn{\mathbf{X}} are precomputed once and held fixed
#' across Newton iterations. At each step, the weighted perturbation
#' \eqn{\mathbf{X}^{\top}\mathbf{W}(\mathbf{V}^{-1} -
#' \mathbf{I})\mathbf{X}} is formed using the
#' precomputed product, then split and re-factored via
#' \code{\link{.woodbury_redecompose_weighted}}.
#'
#' The Woodbury path is used when \eqn{r < P/3}; otherwise the code falls
#' back to the dense whitened solve. If \eqn{\mathbf{F}} is not positive
#' definite, the dense path is also used.
#' See \code{\link{.get_B_gee_woodbury}} (Gaussian) and
#' \code{\link{.get_B_gee_glm_woodbury}} (non-Gaussian).
#' }
#'
#' \subsection{Step Control}{
#' Different paths use different step-control strategies.
#'
#' In the GEE paths (Paths 1a and 1b), the coefficient update is damped:
#' \deqn{\boldsymbol{\beta}^{(s+1)} = (1 - \alpha_s)\boldsymbol{\beta}^{(s)}
#'   + \alpha_s\boldsymbol{\beta}_{\mathrm{cand}}^{(s)},}
#' with \eqn{\alpha_s = 2^{-d_s}} and \eqn{d_s} increased whenever the
#' candidate gives non-finite or larger deviance. The loop terminates after
#' 10 consecutive rejections or 100 total iterations, and exits early when
#' both coefficient change and deviance improvement fall below \code{tol}
#' after a burn-in period.
#'
#' In Path 3 (non-Gaussian, no correlation), the outer projection loop has
#' no line search. The algorithm recomputes \eqn{\mathbf{G}} at the current
#' constrained estimate and applies a fresh projection. If the mean absolute
#' coefficient change begins increasing, the previous iterate is restored
#' and the loop stops. The partition-wise unconstrained estimates themselves
#' are obtained by damped Newton-Raphson inside
#' \code{\link{unconstrained_fit_default}}, where
#' \code{\link{damped_newton_r}} computes a Newton direction once per
#' iteration and halves the step size until the penalized log-likelihood
#' improves.
#' }
#'
#' @section Accommodating Correlation Structures:
#'
#' \subsection{Parametric Correlation Structures}{
#' Suppose \eqn{\mathrm{Cov}(\mathbf{y}) = \sigma^2\,\mathbf{V}(\boldsymbol{\theta})}
#' for a known parametric family indexed by \eqn{\boldsymbol{\theta}} (e.g.,
#' AR(1) with \eqn{\theta = \rho}, Matern with
#' \eqn{\boldsymbol{\theta} = (\ell, \nu)}, exchangeable with
#' \eqn{\theta = \rho}). The penalized generalized least-squares problem
#' becomes
#' \deqn{\min_{\boldsymbol{\beta}}\;
#'   (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})^{\top}\mathbf{V}^{-1}
#'   (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})
#'   + \boldsymbol{\beta}^{\top}\boldsymbol{\Lambda}\boldsymbol{\beta}
#'   \quad \text{s.t.}\; \mathbf{A}^{\top}\boldsymbol{\beta} = \mathbf{0}.}
#' The correlation matrix \eqn{\mathbf{V}} is supplied through the
#' fitted-object components \code{Vhalf} and \code{VhalfInv}, either
#' directly or via user functions \code{Vhalf_fxn} and \code{VhalfInv_fxn}.
#' When both are non-\code{NULL}, \code{\link{get_B}} dispatches to the GEE
#' paths (Path 1a or 1b). For built-in correlation structures, the required
#' square-root matrices are assembled numerically using
#' \code{\link{matsqrt}}, \code{\link{matinvsqrt}}, and \code{\link{invert}}.
#' }
#'
#' \subsection{Whitening and Permutation}{
#' Because the data are stored in partition ordering (all observations from
#' partition 0, then partition 1, etc.) while \eqn{\mathbf{V}} is in the
#' original observation ordering, a permutation is applied internally:
#' \eqn{\mathbf{V}_{\mathrm{perm}}^{-1/2} =
#' \mathbf{V}^{-1/2}[\boldsymbol{\pi}, \boldsymbol{\pi}]}, where
#' \eqn{\boldsymbol{\pi} = \texttt{unlist(order\_list)}} maps original
#' indices to partition-ordered indices. The whitened design and response are
#' \eqn{\tilde{\mathbf{X}} =
#' \mathbf{V}_{\mathrm{perm}}^{-1/2}\mathbf{X}_{\mathrm{block}}} and
#' \eqn{\tilde{\mathbf{y}} =
#' \mathbf{V}_{\mathrm{perm}}^{-1/2}\mathbf{y}}.
#'
#' The \eqn{\mathbf{X}} and \eqn{\mathbf{y}} inputs to
#' \code{\link{lgspline.fit}} are preserved in their unwhitened form.
#' Whitening is applied inside \code{\link{get_B}} and
#' \code{\link{blockfit_solve}} where the full \eqn{N \times P}
#' block-diagonal design is available, since applying
#' \eqn{\mathbf{V}^{-1/2}} to only the diagonal blocks of the partitioned
#' design would silently discard cross-partition contributions and corrupt
#' the Gram matrix.
#' }
#'
#' \subsection{Loss of Block-Diagonal Structure}{
#' Unlike the independent-errors case,
#' \eqn{\mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X}} is not block-diagonal
#' because \eqn{\mathbf{V}^{-1}} introduces cross-partition coupling. The
#' unconstrained estimator
#' \deqn{\hat{\boldsymbol{\beta}} =
#'   (\mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X} +
#'   \boldsymbol{\Lambda})^{-1}
#'   \mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{y}}
#' requires a full \eqn{P \times P} solve, and the constraint projection
#' proceeds with
#' \eqn{\mathbf{G} = (\mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X} +
#' \boldsymbol{\Lambda})^{-1}}.
#'
#' For structured correlation matrices (AR(1), exchangeable, banded), the
#' perturbation \eqn{\mathbf{V}^{-1} - \mathbf{I}} is sparse and the
#' cross-partition coupling has low effective rank. In these cases the
#' Woodbury-accelerated paths
#' (\code{\link{.get_B_gee_woodbury}},
#' \code{\link{.get_B_gee_glm_woodbury}}) recover the partition-wise
#' computational structure by decomposing the coupling into a
#' block-diagonal correction plus a low-rank remainder, as described in the
#' GLM Extension section. For dense or high-rank
#' \eqn{\mathbf{V}^{-1}}, the code falls back to the full whitened system
#' (\code{\link{.get_B_gee_gaussian}}, \code{\link{.get_B_gee_glm}}).
#' }
#'
#' \subsection{GEE Deviance Monitoring}{
#' For non-Gaussian models with correlation, the deviance used for
#' convergence monitoring is computed in the whitened space by
#' \code{.bf_gee_deviance}. When the family supplies
#' \code{custom_dev.resids}, the raw deviance residuals
#' \eqn{r_i = \mathrm{sign}(d_i)\sqrt{|d_i|}} are divided by
#' \eqn{\sqrt{w_i}} and pre=dmultiplied by
#' \eqn{\mathbf{V}_{\mathrm{perm}}^{-1/2}} before squaring and averaging:
#' \deqn{D_{\mathrm{GEE}} = \frac{1}{N}\left\|
#'   \mathbf{V}_{\mathrm{perm}}^{-1/2}\,
#'   \mathbf{w}^{-1/2}\mathbf{r}\right\|^{2},}
#' where \eqn{\mathbf{w}} is the vector of working weights at the current
#' iterate, clamped below at \eqn{\sqrt{\varepsilon_{\mathrm{mach}}}}. When
#' only \code{dev.resids} is available, the function falls back to the
#' standard mean deviance; otherwise it uses mean squared error.
#' }
#'
#' \subsection{REML Estimation of Correlation Parameters}{
#' Correlation parameters \eqn{\boldsymbol{\theta}} are estimated by minimizing
#' a negative restricted log-likelihood (REML) objective. The criterion
#' implemented in \pkg{lgspline} is a central-limit-theorem-based working
#' approximation to a Laplace-style marginal likelihood criterion,
#' applied here solely to correlation structure estimation rather than
#' penalty parameter selection.
#' Let \eqn{\mathbf{D} = \mathrm{diag}(d_i)} be the observation weight matrix,
#' \eqn{\tilde{\mathbf{W}} = \mathrm{diag}(\tilde{w}_i)} the GLM working weight
#' matrix at the current fitted values, \eqn{\mathbf{V}} the correlation matrix
#' parameterized by \eqn{\boldsymbol{\rho}} (a vector on the unconstrained
#' real line), and \eqn{\tilde{\sigma}^{2}} the dispersion profiled at its
#' restricted maximum likelihood estimate. The negative REML objective
#' implemented in \pkg{lgspline}, scaled by \eqn{1/N}, is
#' \deqn{-\ell_{R}(\boldsymbol{\rho}) = \frac{1}{N}\!\left[
#'   -\log|\mathbf{V}^{-1/2}|
#'   + \frac{N}{2}\log\tilde{\sigma}^{2}
#'   + \frac{1}{2\tilde{\sigma}^{2}}
#'     (\mathbf{y} - \boldsymbol{\mu})^{\top}
#'     \mathbf{D}\tilde{\mathbf{W}}^{-1}\mathbf{V}^{-1}
#'     (\mathbf{y} - \boldsymbol{\mu})
#'   + \frac{1}{2}\log|(\tilde{\sigma}^{2}\mathbf{U}\mathbf{G})^{-1}|^{+}
#' \right],}
#' where \eqn{|\cdot|^{+}} denotes the generalized determinant (product of
#' nonzero eigenvalues), and
#' \eqn{\boldsymbol{\mu} = g^{-1}(\mathbf{X}\tilde{\boldsymbol{\beta}})} are
#' the fitted values on the response scale.
#'
#' Gradients with respect to correlation parameters are available in
#' closed form for all built-in structures except Matern, which uses
#' finite-difference approximation due to the complexity of differentiating
#' the modified Bessel function \eqn{K_{\nu}} with respect to \eqn{\nu}. See
#' \code{\link{reml_grad_from_dV}} for the full gradient derivation and
#' notation. Custom analytic gradients can be supplied through \code{REML_grad},
#' and a fully custom criterion can replace REML through
#' \code{custom_VhalfInv_loss}. The Toeplitz example in \code{\link{lgspline}}
#' demonstrates how to supply custom correlation structures with user-defined
#' gradient functions. Optimization over these working correlation parameters is
#' then carried out by the same quasi-Newton engine used elsewhere in the
#' package, with a finite-difference fallback when needed. In the user-facing interface, this
#' machinery is activated through \code{correlation_structure},
#' \code{correlation_id}, \code{spacetime}, \code{VhalfInv}, \code{Vhalf},
#' \code{VhalfInv_fxn}, \code{Vhalf_fxn}, \code{VhalfInv_par_init},
#' \code{REML_grad}, \code{custom_VhalfInv_loss}, and
#' \code{VhalfInv_logdet}.
#'
#' The gradient of the negative REML has three terms per parameter:
#' \enumerate{
#'   \item \eqn{\frac{1}{2}\mathrm{tr}(\mathbf{V}^{-1}\partial\mathbf{V}/\partial\theta_j)}:
#'     the log-determinant contribution.
#'   \item \eqn{-\frac{1}{2\tilde{\sigma}^{2}}\mathbf{r}^{\top}
#'     (\partial\mathbf{V}/\partial\theta_j)\mathbf{r}}:
#'     the residual quadratic form contribution, where
#'     \eqn{\mathbf{r} = \mathrm{diag}(\sqrt{d_i}/\sqrt{\tilde{w}_i})
#'     \mathbf{V}^{-1/2}(\mathbf{y} - \boldsymbol{\mu})}.
#'   \item \eqn{-\frac{1}{2}\mathrm{tr}\!\left(\mathbf{M}^{+}\mathbf{X}_{*}^{\top}
#'     \mathbf{V}^{-1}(\partial\mathbf{V}/\partial\theta_j)\mathbf{V}^{-1}
#'     \tilde{\mathbf{W}}\mathbf{D}\mathbf{X}_{*}\right)}:
#'     the REML correction, where
#'     \eqn{\mathbf{X}_{*} = \mathbf{X}\mathbf{U}} is the constrained design and
#'     \eqn{\mathbf{M} = \mathbf{X}_{*}^{\top}\mathbf{V}^{-1}
#'     \tilde{\mathbf{W}}\mathbf{D}\mathbf{X}_{*}
#'     + \mathbf{U}^{\top}\boldsymbol{\Lambda}\mathbf{U}} is the projected
#'     penalized information.
#' }
#'
#' For each supported correlation family, the derivatives
#' \eqn{\partial\mathbf{V}/\partial\theta_j} are available in closed form,
#' enabling analytic gradient computation for use with the quasi-Newton
#' optimizer.
#' }
#'
#' \subsection{Connection to Standard Mixed Model REML}{
#' The quadratic penalty \eqn{\boldsymbol{\Lambda}} acts as the inverse prior
#' covariance of a Gaussian random effect on the spline coefficients, with
#' the smoothing parameter satisfying \eqn{\lambda = \tau^{2}/\sigma^{2}};
#' this is the same mixed model representation used by \pkg{mgcv}, and
#' Monte Carlo draws under the resulting Laplace-style approximation are
#' available via \code{\link{generate_posterior}}. For Gaussian responses
#' with identity link and no penalization, the implemented criterion
#' coincides exactly with classical REML. For non-Gaussian responses, the
#' criterion substitutes the Fisher information for the full penalized
#' log-likelihood Hessian, exploiting the CLT approximation that
#' \eqn{\tilde{\mathbf{W}}^{-1/2}(\mathbf{y} - \boldsymbol{\mu})} is
#' approximately Gaussian; this yields a method-of-moments style estimator
#' for the correlation and variance parameters that depends only on
#' mean-variance relationships through \eqn{\mathbf{W}}, and therefore
#' generalizes naturally to quasi-likelihood and other settings where a
#' fully specified log-likelihood is unavailable. The REML correction term
#' \eqn{\log|(\tilde{\sigma}^{2}\mathbf{U}\mathbf{G})^{-1}|^{+}} uses a
#' generalized log-determinant so that only nonzero eigenvalues contribute
#' when rank deficiency arises from smoothness constraints or
#' identifiability conditions in \eqn{\mathbf{A}}. When
#' \eqn{\boldsymbol{\Lambda}} is full rank, the criterion coincides with
#' the exact marginal likelihood from integrating out
#' \eqn{\boldsymbol{\beta}} under its Gaussian prior; when
#' \eqn{\boldsymbol{\Lambda}} is rank-deficient, unpenalized coefficients
#' are projected out in the REML sense while penalized coefficients are
#' integrated through their prior. During penalty tuning, the block-diagonal
#' approximation is retained for GCV criteria and gradients; since GCV is
#' rotation-invariant the practical effect on automatic selection is expected
#' to be negligible, though this is not confirmed and the tuned penalties can
#' always be overridden.
#' }
#'
#' \subsection{Built-In Correlation Structures}{
#' The package provides several built-in correlation structures for modeling
#' spatial and temporal dependence. These are specified via
#' \code{correlation_structure} with group membership in \code{correlation_id}
#' and spatial or temporal coordinates in \code{spacetime} (an \eqn{N}-row
#' matrix). Exchangeable correlation does not require \code{spacetime}.
#'
#' All positive scale parameters are estimated on the log scale,
#' with back-transform \eqn{\exp(\cdot)}. Parameters constrained to
#' \eqn{(0, 1)} use a double-exponential back-transform of the form
#' \eqn{\exp(-\exp(\eta))}, so optimization still occurs on the
#' unconstrained real line while the correlation remains bounded.
#'
#' \describe{
#'   \item{Exchangeable}{
#'     Aliases: \code{'exchangeable'}, \code{'cs'}, \code{'CS'},
#'     \code{'compoundsymmetric'}, \code{'compound-symmetric'}.
#'
#'     A constant correlation \eqn{\nu} between any two observations within
#'     the same cluster. Parameterization: \eqn{\nu = \exp(-\exp(\rho))},
#'     so \eqn{\nu \in (0, 1)}. Only positive within-cluster correlation is
#'     supported under this parameterization.
#'   }
#'   \item{Spatial Exponential}{
#'     Aliases: \code{'spatial-exponential'}, \code{'spatialexponential'},
#'     \code{'exp'}, \code{'exponential'}.
#'
#'     Correlation decays exponentially with distance:
#'     \eqn{\exp(-\omega d)} where \eqn{d} is Euclidean distance and
#'     \eqn{\omega > 0}. Parameterization: \eqn{\omega = \exp(\rho)}.
#'     Mathematically equivalent to the power correlation \eqn{\theta^{d}}
#'     with \eqn{\theta = e^{-\omega}}, but with better numerical properties
#'     during optimization.
#'   }
#'   \item{AR(1)}{
#'     Aliases: \code{'ar1'}, \code{'ar(1)'}, \code{'AR(1)'}, \code{'AR1'}.
#'
#'     Correlation depends on rank difference between observations:
#'     \eqn{\nu^{r}} where \eqn{r} is the rank difference within cluster.
#'     Parameterization: \eqn{\nu = \exp(-\exp(\rho))},
#'     so \eqn{\nu \in (0, 1)}. Only positive autocorrelation is supported.
#'   }
#'   \item{Gaussian / Squared Exponential}{
#'     Aliases: \code{'gaussian'}, \code{'rbf'}, \code{'squared-exponential'}.
#'
#'     Smooth decay with squared distance:
#'     \eqn{\exp(-d^{2}/(2\ell^{2}))} where \eqn{\ell} is the length scale.
#'     Parameterization: \eqn{\ell = \exp(\rho)}.
#'   }
#'   \item{Spherical}{
#'     Aliases: \code{'spherical'}, \code{'Spherical'}, \code{'cubic'},
#'     \code{'sphere'}.
#'
#'     Polynomial decay with a hard cutoff at range \eqn{r}:
#'     \eqn{1 - 1.5(d/r) + 0.5(d/r)^{3}} for \eqn{d \le r}, and \eqn{0}
#'     otherwise. Parameterization: \eqn{r = \exp(\rho)}.
#'   }
#'   \item{Matern}{
#'     Aliases: \code{'matern'}, \code{'Matern'}.
#'
#'     Flexible correlation with adjustable smoothness:
#'     \eqn{(2^{1-\nu}/\Gamma(\nu))(\sqrt{2\nu}\,d/\ell)^{\nu}K_{\nu}(\sqrt{2\nu}\,d/\ell)}.
#'     Two parameters: length scale \eqn{\ell = \exp(\rho_1)} and smoothness
#'     \eqn{\nu = \exp(\rho_2)}. No analytical gradient is available for
#'     \eqn{\nu} due to the difficulty of differentiating the modified Bessel
#'     function \eqn{K_{\nu}} with respect to its order, so finite differences
#'     are used; this makes Matern slower and potentially less stable than
#'     other structures.
#'   }
#'   \item{Gamma-Cosine}{
#'     Aliases: \code{'gamma-cosine'}, \code{'gammacosine'},
#'     \code{'GammaCosine'}.
#'
#'     Oscillatory dependence:
#'     \eqn{(d^{\alpha-1}e^{-\gamma d})/(\Gamma(\alpha)/\gamma^{\alpha})\cdot\cos(\omega d)}.
#'     Three parameters: shape \eqn{\alpha = \exp(\rho_1)}, rate
#'     \eqn{\gamma = \exp(\rho_2)}, frequency \eqn{\omega = \exp(\rho_3)}.
#'     Reduces to exponential when \eqn{\alpha = 1} and \eqn{\omega \approx 0}.
#'   }
#'   \item{Gaussian-Cosine}{
#'     Aliases: \code{'gaussian-cosine'}, \code{'gaussiancosine'},
#'     \code{'GaussianCosine'}.
#'
#'     Smooth oscillatory correlation:
#'     \eqn{\exp(-d^{2}/(2\ell^{2}))\cdot\cos(\omega d)}.
#'     Two parameters: length scale \eqn{\ell = \exp(\rho_1)} and frequency
#'     \eqn{\omega = \exp(\rho_2)}. Reduces to Gaussian when
#'     \eqn{\omega \approx 0}.
#'   }
#' }
#' }
#'
#' \subsection{Interpreting Estimated Correlation Parameters}{
#' Correlation parameters are estimated on transformed scales; they must be
#' back-transformed for interpretation. When \code{\link{confint.lgspline}} is called and the
#' inverse Hessian from BFGS is available, confidence intervals are returned
#' on the untransformed (working) scale and should be back-transformed as
#' described in the examples for \code{\link{lgspline}}.
#' }
#'
#' \subsection{Custom Correlation Structures}{
#' Custom correlation structures can be specified through:
#' \itemize{
#'   \item \code{VhalfInv_fxn}: Creates \eqn{\mathbf{V}^{-1/2}}.
#'   \item \code{Vhalf_fxn}: Creates \eqn{\mathbf{V}^{1/2}}. When omitted,
#'     the code computes it by explicit inversion of \code{VhalfInv}.
#'   \item \code{REML_grad}: Provides the analytical gradient of the REML
#'     objective.
#'   \item \code{VhalfInv_logdet}: Efficient log-determinant computation.
#'   \item \code{custom_VhalfInv_loss}: Replaces the REML objective entirely.
#' }
#' These functions enter \code{\link{lgspline}} through
#' \code{correlation_structure}, \code{VhalfInv_fxn}, \code{Vhalf_fxn},
#' \code{REML_grad}, and \code{custom_VhalfInv_loss}, and the fitted object
#' retains the resulting correlation machinery in components such as
#' \code{VhalfInv_fxn}, \code{Vhalf_fxn}, and
#' \code{VhalfInv_params_estimates}. When \code{VhalfInv} is supplied but
#' \code{Vhalf} is not, \code{Vhalf} is computed unconditionally as the inverse
#' of \code{VhalfInv} for all family/link combinations, since both
#' \code{\link{get_B}} and \code{\link{blockfit_solve}} require it for GEE
#' estimation.
#' }
#'
#' @section Variance and Dispersion Estimation:
#'
#' Once the constrained estimate has been obtained, the next questions are how
#' much flexibility the fitted model effectively used and how uncertainty should
#' be propagated through the same constrained geometry. The quantities in this
#' section are therefore all built on the projected information matrices from
#' the previous sections.
#'
#' \subsection{Effective Degrees of Freedom and Dispersion}{
#' The effective degrees of freedom is the trace of the hat matrix. In the
#' Gaussian identity case with observation weights and correlation, the fitted
#' linear operator is built from the dense GLS analogue
#' \eqn{\mathbf{G}_{\mathrm{correct}}} and can be written schematically as
#' \deqn{\mathbf{H} =
#'   \mathbf{V}^{-1/2}(\mathbf{W}\mathbf{D})^{1/2}
#'   \mathbf{X}\mathbf{U}\mathbf{G}_{\mathrm{correct}}\mathbf{X}^{\top}
#'   (\mathbf{W}\mathbf{D})^{1/2}\mathbf{V}^{-1/2},}
#' where for Gaussian identity \eqn{\mathbf{W} = \mathbf{I}}. In the
#' no-correlation Gaussian case this reduces to the familiar
#' \eqn{\mathbf{H} = \mathbf{X}\mathbf{U}\mathbf{G}\mathbf{X}^{\top}\mathbf{D}}.
#'
#' For Gaussian identity fits, the dispersion estimate is computed as a
#' weighted mean squared residual, optionally scaled by
#' \eqn{N/(N - \mathrm{tr}(\mathbf{H}))} when
#' \code{unbias_dispersion = TRUE}:
#' \deqn{\tilde{\sigma}^{2} =
#'   \frac{1}{N - \mathrm{tr}(\mathbf{H})}\|\mathbf{y} - \mathbf{\tilde{y}}_i \|^{2}.}
#'
#' More generally with weights, a correlation structure and non-linear link function:
#' \deqn{\tilde{\sigma}^{2} =
#'   \frac{1}{N - \mathrm{tr}(\mathbf{H})}\| \mathbf{V}^{-1/2}\mathbf{W}^{-1/2}\mathbf{D}^{1/2}(\mathbf{y} - \tilde{\mathbf{y}})\|^{2}.}
#'
#' This estimated dispersion is returned as \code{sigmasq_tilde}, and the
#' corresponding effective degrees of freedom trace is returned as
#' \code{trace_XUGX}. For non-Gaussian families, the fitting code delegates
#' dispersion estimation to \code{dispersion_function}; thus the package does
#' not assume a single closed-form Pearson-style formula outside the Gaussian
#' identity setting. The hat-matrix trace itself is assembled by
#' \code{\link{compute_trace_H}} in the dense correlation-aware case and by the
#' same blockwise products summarized by \code{trace_XUGX} in the simpler
#' no-correlation paths.
#' A concrete built-in non-Gaussian example is the Weibull AFT path, which pairs
#' \code{\link{weibull_family}} with \code{\link{weibull_dispersion_function}},
#' \code{\link{weibull_glm_weight_function}}, and
#' \code{\link{weibull_schur_correction}}.
#' Users who want these quantities available for downstream inference should
#' keep \code{estimate_dispersion = TRUE} and \code{return_varcovmat = TRUE}
#' (the defaults), since \code{\link{wald_univariate}},
#' \code{\link{confint.lgspline}}, and the prediction-standard-error path in
#' \code{\link{predict.lgspline}} all rely on the post-fit dispersion and
#' covariance components documented here.
#' }
#'
#' \subsection{Variance-Covariance Matrix}{
#' The variance-covariance matrix of \eqn{\tilde{\boldsymbol{\beta}}} is
#' estimated as:
#' \deqn{\mathrm{Var}(\tilde{\boldsymbol{\beta}}) = \tilde{\sigma}^{2}(\mathbf{U}\mathbf{G}^{1/2})(\mathbf{U}\mathbf{G}^{1/2})^{\top}}
#' using the outer-product form for numerical stability. The result is returned
#' as \code{varcovmat} when \code{return_varcovmat = TRUE}. The algebraically
#' equivalent expression \eqn{\tilde{\sigma}^{2}\mathbf{U}\mathbf{G}} is not
#' used because \eqn{\mathbf{G}} is only positive semi-definite when the
#' penalty matrix \eqn{\boldsymbol{\Lambda}} has zero eigenvalues (e.g., the
#' intercept and linear terms under the smoothing spline penalty when
#' \code{flat_ridge_penalty} = 0), which can introduce negative diagonal
#' entries in finite precision arithmetic. The outer-product form also
#' guarantees symmetry.
#'
#' This is the Bayesian posterior covariance, treating the penalty as a
#' Gaussian prior on the coefficients. When \code{exact_varcovmat = TRUE},
#' a frequentist correction is additionally computed:
#' \deqn{\mathrm{Var}_{\mathrm{exact}}(\tilde{\boldsymbol{\beta}}) =
#'   \tilde{\sigma^2}\mathbf{U}\mathbf{G}^{1/2}(\mathbf{X}^{\top}\mathbf{W}\mathbf{D}\mathbf{V}^{-1}\mathbf{X})\mathbf{G}^{1/2}\mathbf{U}^{\top} =
#'   \tilde{\sigma}^{2}\mathbf{U}\mathbf{G}\mathbf{U}^{\top}
#'   - \tilde{\sigma}^{2}\mathbf{U}\mathbf{G}\boldsymbol{\Lambda}\mathbf{G}\mathbf{U}^{\top}.}
#' The first term is the Bayesian posterior covariance; the second is a frequentist
#' correction such that for Gaussian identity link (with or without correlation),
#' this is the exact variance-covariance matrix of the constrained estimator.
#'
#' When a correlation structure is present (\code{VhalfInv} non-\code{NULL}),
#' the block-diagonal \eqn{\mathbf{G}} is replaced by the full weighted GLS
#' analogue
#' \deqn{\mathbf{G}_{\mathrm{correct}} =
#'   \left(\mathbf{X}^{\top}\mathbf{W}\mathbf{D}\mathbf{V}^{-1}\mathbf{X}
#'   + \boldsymbol{\Lambda}\right)^{-1},}
#' where \eqn{\mathbf{W} = \mathbf{I}} in the Gaussian identity case.
#' This dense matrix is what enters the correlation-aware \eqn{\mathbf{U}},
#' \eqn{\mathrm{Var}(\tilde{\boldsymbol{\beta}})}, and
#' \eqn{\mathrm{Var}_{\mathrm{exact}}(\tilde{\boldsymbol{\beta}})}
#' computations.
#'
#' In user-facing terms, \code{return_varcovmat} controls whether this matrix is
#' stored at all, while \code{exact_varcovmat} switches between the default
#' posterior/Laplace approximation and the exact frequentist correction in the
#' Gaussian-identity setting. The stored covariance is what powers
#' \code{\link{wald_univariate}}, \code{\link{confint.lgspline}}, and
#' \code{se.fit = TRUE} in \code{\link{predict.lgspline}}; the
#' \code{critical_value} argument supplied at fit time is carried forward as the
#' default cutoff for those interval-producing helpers.
#' }
#'
#' \subsection{Recomputation of G at Convergence}{
#' At the final iterate, \eqn{\mathbf{G}} is recomputed to reflect the
#' converged working weights and Schur corrections. The implementation
#' computes the weighted design
#' \eqn{\mathbf{X}_{w}^{(k)} = \mathbf{X}_k \cdot \mathrm{diag}(\sqrt{\mathbf{w}_k})},
#' forms the weighted Gram matrix
#' \eqn{\mathbf{X}_{w}^{(k)\top}\mathbf{X}_{w}^{(k)}}, adds the Schur
#' correction, and performs eigendecomposition via
#' \code{\link{compute_G_eigen}} to obtain \eqn{\mathbf{G}_k},
#' \eqn{\mathbf{G}_k^{1/2}}, and \eqn{\mathbf{G}_k^{-1/2}}. The relationship
#' \eqn{\mathbf{G}_k = \mathbf{G}_k^{1/2}(\mathbf{G}_k^{1/2})^{\top}} is
#' enforced exactly by construction, and the fitted object can retain these as
#' \code{G} and \code{Ghalf} when \code{return_G = TRUE} and
#' \code{return_Ghalf = TRUE}. The square-root factors are numerically
#' stabilized through the helper routines \code{\link{matsqrt}} and
#' \code{\link{matinvsqrt}}, which are also used elsewhere in the package when
#' dense analogues of \eqn{\mathbf{G}^{1/2}} or \eqn{\mathbf{G}^{-1/2}} are
#' required.
#' }
#'
#' @section Bayesian Interpretation:
#'
#' The penalty has a natural Gaussian-prior interpretation, so once the
#' constrained estimator and its covariance are available, Bayesian-style
#' posterior simulation follows almost immediately. This section records the
#' interpretation that is already implicit in the fitted object and in the
#' package's posterior simulation helpers.
#'
#' A Bayesian interpretation follows from viewing the penalty as a Gaussian
#' prior on the coefficients. Conditional on the fitted smoothing parameters,
#' the code samples on the coefficient scale from
#' \deqn{\boldsymbol{\beta}^{(m)} =
#'   \tilde{\boldsymbol{\beta}} +
#'   \sqrt{\tilde{\sigma}^{2}},\mathbf{U}\mathbf{G}^{1/2}\mathbf{z}^{(m)},
#'   \qquad \mathbf{z}^{(m)} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}).}
#' The coefficients are then back-transformed to the original response and
#' predictor scales.
#'
#' When inequality constraints are absent, these draws are i.i.d. Gaussian
#' posterior draws around the fitted mode.
#' The underlying coefficient-draw closure also contains an elliptical slice
#' sampling route for active inequality constraints, using the same covariance
#' factor to keep retained draws in the feasible region.
#'
#' At the implementation level, standard Gaussian posterior draws may place
#' positive mass on the infeasible region, so the constrained-draw closure
#' instead targets the truncated posterior
#' \deqn{\pi(\boldsymbol{\beta} \mid \mathbf{y}) \propto
#'   \exp\!\left(-\frac{1}{2}(\boldsymbol{\beta} - \tilde{\boldsymbol{\beta}})^{\top}
#'   \mathbf{G}^{-1}(\boldsymbol{\beta} - \tilde{\boldsymbol{\beta}})\right)
#'   \mathbf{1}(\mathbf{C}^{\top}\boldsymbol{\beta} \succeq \mathbf{c}),}
#' yielding credible intervals that respect the constraint boundaries.
#' The public \code{\link{generate_posterior}} wrapper forwards
#' \code{enforce_qp_constraints} to the stored constrained-draw closure, so
#' constrained draws can be requested directly from the user-facing interface.
#' When a working correlation structure is present, the companion helper
#' \code{\link{generate_posterior_correlation}} extends this idea by propagating
#' uncertainty in the fitted correlation parameters through the same
#' \code{VhalfInv_fxn}/\code{Vhalf_fxn} machinery described in the correlation
#' section, rather than conditioning only on fixed covariance parameters. Correlation
#' parameters are drawn from a multivariate normal distribution centered about their estimates
#' with the inverse approxiamte BFGS Hessian of the REML optimization problem
#' \code{VhalfInv_params_vcov} used by default (or a custom alternative as
#' supplied to the argument \code{correlation_param_vcov_sc}).
#'
#'
#' @section Inequality Constraints via Sequential Quadratic Programming:
#'
#' \subsection{Overview}{
#' Inequality constraints of the form
#' \eqn{\mathbf{C}^{\top}\boldsymbol{\beta} \succeq \mathbf{c}} handle
#' shape restrictions such as monotonicity, convexity, or boundedness. In
#' the monomial basis, these are linear in \eqn{\boldsymbol{\beta}}.
#' Monotonicity at a grid of points requires the first derivative polynomial
#' to be non-negative there; convexity requires the second derivative to be
#' non-negative; range constraints bound the fitted values directly. All of
#' these translate to linear inequality constraints on the coefficient
#' vector.
#'
#' The inequality pieces are assembled by \code{\link{process_qp}}, which
#' returns the \code{qp_Amat}, \code{qp_bvec}, and \code{qp_meq} objects
#' passed to \code{\link[quadprog]{solve.QP}}, together with a
#' \code{quadprog} flag indicating whether any inequality constraints are
#' active. For derivative-sign constraints, \code{\link{process_qp}} calls
#' \code{.build_deriv_qp}, which uses
#' \code{\link{make_derivative_matrix}} on the expansion-standardized
#' design and maps the derivative rows into the full \eqn{P}-dimensional
#' coefficient space partition by partition.
#' }
#'
#' \subsection{Partition-Wise Active-Set Method}{
#' The sparsity pattern of \eqn{\mathbf{C}} is inspected automatically by
#' \code{\link{.detect_qp_global}} (equivalently
#' \code{.solver_detect_qp_global}). When every column of
#' \eqn{\mathbf{C}} has nonzero entries in only a single partition block,
#' the constraint system is block-separable and an active-set method can
#' replace the dense QP.
#'
#' At each iteration, the active set \eqn{\mathcal{A}} (constraints
#' satisfied at equality) is appended to \eqn{\mathbf{A}} as additional
#' equality constraints:
#' \deqn{\mathbf{A}_{\mathrm{aug}} = [\mathbf{A} \mid
#'   \mathbf{C}_{\mathcal{A}}].}
#' The constrained estimate is obtained by the same OLS projection with
#' \eqn{\mathbf{A}_{\mathrm{aug}}} in place of \eqn{\mathbf{A}}, and
#' since \eqn{\mathbf{A}_{\mathrm{aug}}} retains block-diagonal
#' compatibility, all operations remain partition-wise.
#'
#' The active set is updated by checking primal feasibility (adding the
#' most-violated inactive constraint) and dual feasibility (dropping the
#' active constraint with the most negative Lagrange multiplier) until the
#' KKT conditions are satisfied. Lagrange multipliers for active
#' inequalities are recovered from the OLS fit used in the Lagrangian
#' projection: the fitted coefficients on
#' \eqn{\mathbf{X}^* = \mathbf{G}^{1/2}\mathbf{A}_{\mathrm{aug}}} give
#' the multipliers up to sign. Implemented in
#' \code{\link{.active_set_refine}} and
#' \code{\link{.check_kkt_partitionwise}}.
#'
#' When any column of \eqn{\mathbf{C}} spans multiple partition blocks
#' (for example, cross-knot monotonicity constraints), the block-diagonal
#' structure is broken and the dense SQP approach is required. The
#' selection is made automatically. If the active-set method does not
#' converge within its iteration limit (default 50), the code falls back
#' to dense SQP as well.
#' }
#'
#' \subsection{Dense SQP Iteration}{
#' The dense SQP approach, implemented in \code{\link{.qp_refine}},
#' solves a sequence of quadratic subproblems approximating the penalized
#' log-likelihood. At each iteration \eqn{s}:
#' \enumerate{
#'   \item Compute the information matrix
#'     \eqn{\mathbf{M}^{(s)} =
#'     \mathbf{X}^{\top}\mathbf{W}^{(s)}\mathbf{X} +
#'     \boldsymbol{\Lambda}_{\mathrm{block}} + \mathbf{S}^{(s)}},
#'     where \eqn{\mathbf{S}^{(s)}} is the Schur complement correction.
#'   \item Compute the score vector via \code{qp_score_function}.
#'   \item Solve the QP with \code{\link[quadprog]{solve.QP}}, where the
#'     combined constraint matrix is
#'     \eqn{[\mathbf{A} \mid \mathbf{C}]} with the first \eqn{R} columns
#'     treated as equalities.
#'   \item Apply a damped update
#'     \eqn{\boldsymbol{\beta}^{(s+1)} =
#'     (1 - \alpha)\boldsymbol{\beta}^{(s)} +
#'     \alpha\boldsymbol{\beta}_{\mathrm{QP}}}, with
#'     \eqn{\alpha = 2^{-d}} and \eqn{d} incremented upon deviance
#'     increase.
#' }
#' A rescaling factor
#' \eqn{\mathrm{sc} = \sqrt{\mathrm{mean}(|\mathbf{M}^{(s)}|)}} is applied
#' to the Hessian and linear term before calling the QP solver. The
#' equality-constrained estimate from the Lagrangian projection serves as
#' a warm start. The \code{qp_score_function} defaults to the canonical GLM
#' score
#' \eqn{\mathbf{X}^{\top}(\mathbf{y} - \boldsymbol{\mu})}; for custom
#' models a different score can be supplied.
#' }
#'
#' \subsection{Active Set and Lagrange Multipliers}{
#' The active set at the solution identifies binding inequality constraints.
#' The Lagrange multipliers quantify the cost of each: a multiplier of zero
#' means the constraint is not binding. The implementation stores active
#' constraint indices, the corresponding submatrix of the constraint matrix,
#' and the multiplier vector in the \code{qp_info} list returned alongside
#' coefficient estimates, with components \code{lagrangian}, \code{iact},
#' and \code{Amat_active}. When
#' \code{return_lagrange_multipliers = TRUE}, the fitted object also stores
#' the final multiplier vector directly as \code{lagrange_multipliers}.
#'
#' The original assembled inequality data are retained in the fitted
#' object's \code{quadprog_list} component (containing \code{qp_Amat},
#' \code{qp_bvec}, and \code{qp_meq} from \code{\link{process_qp}}) so
#' the final active set can be interpreted relative to the full
#' specification. The final \eqn{\mathbf{U}} used in constructing the
#' posterior variance-covariance matrix is built from both the equality
#' constraints and the active inequality constraints at the solution.
#' }
#'
#' \subsection{Built-In Constraints}{
#' Built-in inequality constraints include:
#' \itemize{
#'   \item Monotonicity: \code{qp_monotonic_increase},
#'     \code{qp_monotonic_decrease}. Enforced by requiring consecutive
#'     fitted values to be non-decreasing (or non-increasing):
#'     \eqn{(\mathbf{x}_i - \mathbf{x}_{i-1})^{\top}\boldsymbol{\beta}
#'     \geq 0}. These are constructed by \code{\link{process_qp}} from
#'     the partition-stacked block design reordered to observation order.
#'   \item Derivative sign: \code{qp_positive_derivative},
#'     \code{qp_negative_derivative}. Enforced through the
#'     first-derivative design matrix from
#'     \code{\link{make_derivative_matrix}}. May be \code{TRUE}/\code{FALSE}
#'     or a character/integer vector selecting specific predictors.
#'   \item Second-derivative sign: \code{qp_positive_2ndderivative},
#'     \code{qp_negative_2ndderivative}. Same construction using the
#'     second-derivative design matrix.
#'   \item Response range: \code{qp_range_lower}, \code{qp_range_upper}.
#'     Bounds on the linear predictor; for non-identity links, the bounds
#'     are transformed to the link scale inside \code{\link{process_qp}}.
#'   \item Custom constraints via \code{qp_Amat_fxn}, \code{qp_bvec_fxn},
#'     and \code{qp_meq_fxn}, which receive the design matrix structure
#'     and return the constraint matrix, bound vector, and number of
#'     equalities. These are commonly paired with a custom
#'     \code{qp_score_function} when the quadratic approximation uses a
#'     non-default likelihood. The low-level objects \code{qp_Amat},
#'     \code{qp_bvec}, and \code{qp_meq} remain documented in
#'     \code{\link{lgspline}} but in the current implementation serve as
#'     activation markers rather than being merged into the constraint set
#'     assembled by \code{\link{process_qp}}.
#' }
#' All constraints can be thinned to a user-specified subset of rows via
#' \code{qp_observations}, which \code{\link{process_qp}} applies before
#' assembly.
#' }
#'
#' @section Blockfit Backfitting for Linear Non-Interactive Effects:
#'
#' \subsection{Motivation}{
#' When a model contains both spline terms (receiving \eqn{K+1}
#' partition-specific coefficient vectors constrained to smoothness) and
#' non-interactive linear terms (``flat'' terms, specified via
#' \code{just_linear_without_interactions}, receiving a single shared
#' coefficient vector \eqn{\mathbf{v}} across all partitions), the standard
#' solver carries \eqn{K+1} copies of \eqn{\mathbf{v}} linked by equality
#' constraints. Backfitting avoids this inflation by solving a
#' lower-dimensional problem at each step. Write the partition-\eqn{k}
#' design as
#' \eqn{\mathbf{X}_k = [\mathbf{Z}_k \mid
#' \mathbf{X}_{\mathrm{flat}}^{(k)}]}, where \eqn{\mathbf{Z}_k} contains
#' the spline columns and \eqn{\mathbf{X}_{\mathrm{flat}}^{(k)}} the flat
#' columns. This is invoked when \code{blockfit = TRUE}, flat columns are
#' non-empty, and \eqn{K > 0}.
#'
#' The design, penalty, and constraint matrices are split into spline and
#' flat components by \code{\link{.bf_split_components}}, which extracts
#' spline rows from \eqn{\mathbf{A}}, drops null columns, rank-reduces via
#' QR, and detects mixed constraints (columns of \eqn{\mathbf{A}} with
#' nonzero entries on both spline and flat rows).
#' }
#'
#' \subsection{Block-Coordinate Descent}{
#' \strong{Spline step.} Holding \eqn{\mathbf{v}} fixed, the code forms
#' the adjusted response
#' \eqn{\mathbf{y}_k - \mathbf{X}_{\mathrm{flat}}^{(k)}\mathbf{v}} and
#' applies the spline-only Lagrangian projection via
#' \code{\link{.bf_lagrangian_project}}:
#' \deqn{\tilde{\boldsymbol{\beta}}_{\mathrm{spline}}^{(k)} =
#'   \mathbf{U}_{\mathrm{spline}}\mathbf{G}_{\mathrm{spline}}
#'   \mathbf{Z}_k^{\top}(\mathbf{y}_k -
#'   \mathbf{X}_{\mathrm{flat}}^{(k)}\mathbf{v}).}
#'
#' \strong{Flat step.} Holding spline coefficients fixed, the shared flat
#' vector is updated by pooled penalized regression on residuals:
#' \deqn{\mathbf{v} = \left(\sum_{k=0}^{K}
#'   \mathbf{X}_{\mathrm{flat}}^{(k)\top}
#'   \mathbf{X}_{\mathrm{flat}}^{(k)} +
#'   \boldsymbol{\Lambda}_{\mathrm{flat}}\right)^{-1}
#'   \sum_{k=0}^{K}
#'   \mathbf{X}_{\mathrm{flat}}^{(k)\top}
#'   (\mathbf{y}_k - \mathbf{Z}_k
#'   \tilde{\boldsymbol{\beta}}_{\mathrm{spline}}^{(k)}).}
#' When the constraint matrix \eqn{\mathbf{A}} has mixed columns (nonzero
#' entries on both spline and flat rows), the flat update instead solves a
#' KKT system enforcing the residual equality constraint
#' \eqn{\mathbf{A}_{\mathrm{flat}}^{\top}\mathbf{v} = \mathbf{c} -
#' \mathbf{A}_{\mathrm{spline}}^{\top}
#' \tilde{\boldsymbol{\beta}}_{\mathrm{spline}}}
#' via \code{\link{.bf_constrained_flat_update}}.
#'
#' Convergence is checked using the maximum absolute change across both
#' spline and flat coefficients. In the weighted inner loop used by the GLM
#' solvers, the flat-block change alone determines stopping.
#' }
#'
#' \subsection{Four Estimation Cases}{
#' \code{\link{blockfit_solve}} dispatches to one of four paths.
#'
#' \strong{Case (a): Gaussian identity + GEE}
#' (\code{\link{.bf_case_gauss_gee}}).
#' Whitening destroys block-diagonal structure, so the code skips
#' backfitting and performs the same full-system Gaussian GEE projection
#' used by \code{\link{get_B}} Path 1a. The result is split back into
#' spline and flat components for downstream assembly.
#'
#' \strong{Case (b): Gaussian identity, no correlation}
#' (\code{\link{.bf_case_gauss_no_corr}}).
#' Standard block-coordinate descent as described above. The spline-only
#' \eqn{\mathbf{G}_{\mathrm{spline}}} factors are precomputed once, and the
#' pooled flat penalized inverse is reused across iterations.
#'
#' \strong{Case (c): GLM + GEE}
#' (\code{\link{.bf_case_glm_gee}}).
#' Two stages. Stage 1 forms a warm start by running damped Newton-Raphson
#' on the unwhitened working response: each outer iteration computes
#' working responses and weights at the current linear predictor, then
#' calls \code{\link{.bf_inner_weighted}} for the inner backfitting loop.
#' Stage 2 refines the warm start on the full whitened system using the
#' damped SQP loop (\code{\link{.bf_sqp_loop}}), replicating the approach
#' used by \code{\link{.get_B_gee_glm}}.
#'
#' \strong{Case (d): GLM without GEE}
#' (\code{\link{.bf_case_glm_no_corr}}).
#' A damped Newton-Raphson outer loop updates working responses and weights,
#' while each inner iteration alternates between weighted spline and
#' weighted flat updates via \code{\link{.bf_inner_weighted}}. Deviance is
#' monitored across outer iterations for convergence and damping.
#' }
#'
#' \subsection{Inequality Constraints and Reassembly}{
#' Because flat coefficients are shared by construction, the corresponding
#' equality constraints are satisfied exactly. Smoothness constraints on the
#' spline block are enforced by the spline-only Lagrangian projection.
#' After backfitting convergence, inequality constraints are enforced via
#' the same partition-wise active-set or dense SQP refinement used by
#' \code{\link{get_B}}. The method is selected automatically by
#' \code{.solver_detect_qp_global}: block-separable constraints use the
#' active-set method through \code{\link{.active_set_refine}}, while
#' cross-partition constraints trigger the dense SQP loop through
#' \code{\link{.bf_sqp_loop}}. For GEE (Case c), inequality handling
#' occurs inside Stage 2 on the whitened system.
#'
#' After convergence, the shared flat vector \eqn{\mathbf{v}} is copied
#' into each partition's coefficient vector, yielding
#' \eqn{\boldsymbol{\beta}_k =
#' [\tilde{\boldsymbol{\beta}}_{\mathrm{spline}}^{(k)\top},
#' \mathbf{v}^{\top}]^{\top}} for compatibility with downstream inference.
#' If \code{\link{blockfit_solve}} throws an error, a warning is issued
#' and the code falls back to \code{\link{get_B}}.
#' }
#'
#' @section Knot Selection and Partitioning:
#'
#' The topic of knot selection is not the main focus of the package, but the
#' partition structure is central because every later design matrix, penalty,
#' and smoothness constraint depends on it. The defaults in \pkg{lgspline} are
#' therefore meant to be practical and transparent rather than theoretically
#' final.
#'
#' \subsection{Univariate Case}{
#' For a single predictor, the default partitioning is now handled by
#' \code{\link{make_partitions}} in the same \eqn{k}-means framework used more
#' generally: \eqn{K+1} centers are fit on an internally standardized copy
#' of the predictor, controlled by \code{standardize_predictors_for_knots}, and
#' then returned on the raw scale. Custom knots can still be supplied via
#' \code{custom_knots}, in which case partition assignment is built directly
#' from those raw-scale breakpoints. The default number of knots \eqn{K} is
#' chosen adaptively based on \eqn{N}, \eqn{p}, \eqn{q}, and the GLM family.
#' For multivariate fits, the resulting partition metadata are returned as
#' \code{make_partition_list} and can be re-used in later calls to
#' \code{\link{lgspline}}. This is particularly useful when one wants to hold
#' the partition geometry fixed across repeated fits, for example while varying
#' penalties, families, or correlation structures.
#' }
#' \subsection{Multivariate Case}{
#' For multiple predictors, \eqn{K+1} cluster centers are identified by
#' \eqn{k}-means on an internally standardized predictor matrix via
#' \code{\link{make_partitions}}. This is the partitioning mechanism used to
#' determine the multivariate spline regions; see MacQueen (1967) for the
#' classical clustering formulation and Kisi et al. (2025) for a recent applied
#' example of \eqn{k}-means-driven partitioning in a nonlinear prediction
#' setting. Midpoints between neighboring centers
#' (those whose midpoint  does not fall into a third cluster) serve as knot
#' locations. Observations are assigned to the nearest cluster center using
#' \code{\link[FNN]{get.knnx}}, and the returned centers and knots are on
#' the original predictor scale. The resulting partition structure is a type
#' of Voronoi diagram and is stored in the fitted object as
#' \code{make_partition_list}. The \code{do_not_cluster_on_these} argument can
#' exclude certain predictors from clustering (e.g., a treatment indicator that
#' should not drive partitioning). The lower-level clustering behavior can be
#' further controlled by \code{cluster_args} and \code{neighbor_tolerance},
#' while \code{cluster_on_indicators} determines whether binary predictors are
#' allowed to influence the partition geometry at all.
#' }
#'
#' \subsection{Standardizing Predictors}{
#' Higher-order polynomial terms can dramatically inflate or deflate the
#' magnitude of basis expansions, introducing numerical instability. All
#' polynomial basis expansions are scaled by
#' \eqn{q_{0.69} - q_{0.31}}, where \eqn{q_{\zeta}} is the \eqn{\zeta}-th
#' quantile of the expansion. For a standard normal distribution this quantity
#' is approximately 1, so the scaling is close to one standard deviation for
#' symmetric distributions. This fitting-stage rescaling is controlled by
#' \code{standardize_expansions_for_fitting}, while knot construction is
#' controlled separately by \code{standardize_predictors_for_knots}. The same
#' scaling is applied to the constraint matrix to maintain smoothness, and
#' coefficients are back-transformed to the original scale after fitting.
#' }
#'
#' @section Smoothing Spline Penalty:
#'
#' \subsection{Penalty Construction}{
#' The penalty matrix \eqn{\boldsymbol{\Lambda}_s} penalizes the integrated
#' squared total curvature of the fitted function over the observed predictor
#' ranges. This is the step that makes the piecewise polynomial fit genuinely
#' behave like a smoothing spline rather than merely a constrained regression
#' spline. The package computes this penalty directly from the monomial
#' structure of the basis rather than by appealing to a pre-tabulated spline
#' basis. For a single partition \eqn{k} with basis expansion
#' \eqn{\mathbf{x}_k = (\phi_1(\mathbf{t}), \ldots, \phi_p(\mathbf{t}))^{\top}}
#' where each \eqn{\phi_i(\mathbf{t}) = \prod_{j=1}^{q} t_j^{\alpha_{ij}}}
#' is a multivariate monomial:
#' \deqn{\boldsymbol{\beta}_k^{\top}\boldsymbol{\Lambda}_s\boldsymbol{\beta}_k
#'   = \int_{\mathbf{a}}^{\mathbf{b}}
#'     \|\tilde{f}_k''(\mathbf{t})\|^{2}\,d\mathbf{t},}
#' where \eqn{\mathbf{a}} and \eqn{\mathbf{b}} are the observed predictor
#' minimums and maximums (computed globally from the data, not
#' partition-specific), and
#' \eqn{\tilde{f}_k(\mathbf{t}) = \mathbf{x}_k^{\top}\boldsymbol{\beta}_k}
#' is the fitted function for partition \eqn{\mathcal{P}_k}.
#'
#' \strong{Total curvature operator.}
#' The integrated squared second derivative decomposes into \eqn{q}
#' curvature operators, one per predictor. For predictor \eqn{v}, the
#' curvature operator \eqn{D_v} is defined as
#' \deqn{D_v = \frac{\partial^{2}}{\partial t_v^{2}}
#'   + \sum_{s \neq v}\frac{\partial^{2}}{\partial t_v\,\partial t_s}.}
#' That is, \eqn{D_v} captures both the pure second derivative with respect
#' to \eqn{t_v} and all mixed second partial derivatives involving \eqn{t_v}.
#' The penalty matrix entries are then
#' \deqn{[\boldsymbol{\Lambda}_s]_{ij}
#'   = \sum_{v=1}^{q}\int_{\mathbf{a}}^{\mathbf{b}}
#'     D_v(\phi_i)\,D_v(\phi_j)\,d\mathbf{t}.}
#'
#' \strong{Monomial derivative rule.}
#' For a monomial \eqn{\phi(\mathbf{t}) = \prod_j t_j^{\alpha_j}}, the
#' derivatives entering \eqn{D_v} have closed forms. The pure second
#' derivative is
#' \deqn{\frac{\partial^{2}}{\partial t_v^{2}}\prod_j t_j^{\alpha_j}
#'   = \alpha_v(\alpha_v - 1)\,t_v^{\alpha_v - 2}\prod_{j \neq v} t_j^{\alpha_j},}
#' which is zero when \eqn{\alpha_v < 2}. The mixed second derivative is
#' \deqn{\frac{\partial^{2}}{\partial t_v\,\partial t_s}\prod_j t_j^{\alpha_j}
#'   = \alpha_v\alpha_s\,t_v^{\alpha_v - 1}t_s^{\alpha_s - 1}
#'   \prod_{j \neq v,s} t_j^{\alpha_j},}
#' which is zero when \eqn{\alpha_v < 1} or \eqn{\alpha_s < 1}. Applying
#' \eqn{D_v} to a monomial \eqn{\phi_i} produces a sum of monomials with
#' known coefficients and exponent vectors.
#'
#' \strong{Factorized integration.}
#' Because every \eqn{D_v(\phi_i)} is polynomial, the product
#' \eqn{D_v(\phi_i)\,D_v(\phi_j)} is also polynomial and the multivariate
#' integral factorizes over predictors:
#' \deqn{\int_{\mathbf{a}}^{\mathbf{b}}\prod_{j=1}^{q} t_j^{e_j}\,d\mathbf{t}
#'   = \prod_{j=1}^{q}\frac{b_j^{e_j+1} - a_j^{e_j+1}}{e_j + 1}.}
#' Crucially, this integral runs over \emph{all} \eqn{q} predictor ranges,
#' including predictors that do not appear in the integrand (for which
#' \eqn{e_j = 0} and the factor reduces to \eqn{b_j - a_j}). This ensures
#' that the penalty is properly scaled relative to the volume of the
#' predictor space.
#'
#' \strong{Single-predictor verification.}
#' For \eqn{q = 1} with expansion
#' \eqn{\mathbf{x} = (1, t, t^{2}, t^{3})^{\top}} on \eqn{[a, b]}, the
#' curvature operator reduces to \eqn{D_1 = \partial^{2}/\partial t^{2}} (no
#' mixed partials exist), and the penalty matrix reduces to
#' \deqn{\boldsymbol{\Lambda}_s
#'   = \int_a^b \mathbf{x}''\mathbf{x}''^{\top}\,dt
#'   = \begin{pmatrix}
#'       0 & 0 & 0 & 0 \\
#'       0 & 0 & 0 & 0 \\
#'       0 & 0 & 4(b - a) & 6(b^{2} - a^{2}) \\
#'       0 & 0 & 6(b^{2} - a^{2}) & 12(b^{3} - a^{3})
#'     \end{pmatrix},}
#' equivalent to the classical cubic smoothing spline penalty originally proposed by Reinsch.
#'
#' \strong{Handling of non-spline predictors.}
#' Predictors specified via \code{just_linear_without_interactions} or
#' \code{just_linear_with_interactions} do not receive higher-order
#' polynomial expansions in the design matrix. To ensure their curvature
#' contributions are still correctly computed (particularly through
#' interaction terms), the implementation temporarily appends phantom
#' higher-order columns (with zero data) for these predictors, computes
#' the full curvature penalty on the augmented basis, and then subsets the
#' result back to the original \eqn{p \times p} dimensions. This ensures
#' that interaction terms involving non-spline predictors receive appropriate
#' penalty contributions without affecting the rest of the estimation
#' pipeline.
#'
#' \strong{Parallel computation.}
#' Because the total penalty is an additive sum over predictors
#' (\eqn{\boldsymbol{\Lambda}_s = \sum_{v=1}^{q}\boldsymbol{\Lambda}_{s,v}}),
#' the computation can be parallelized by distributing the per-predictor
#' curvature matrices across workers via \code{parallel::parLapply} and
#' summing the results. This is controlled by the \code{parallel_penalty}
#' argument and is beneficial when \eqn{q} is large.
#'
#' The penalty is computed by \code{\link{get_2ndDerivPenalty}} (single
#' predictor or subset) and \code{\link{get_2ndDerivPenalty_wrapper}}
#' (full assembly with optional parallelism and non-spline handling).
#'
#' Because the smoothing penalty has zero eigenvalues for the intercept and
#' linear terms (whose second derivatives vanish), an optional ridge penalty on
#' lower-order terms is added for computational stability. The full penalty
#' block for partition \eqn{k} is:
#' \deqn{\boldsymbol{\Lambda}_k = \lambda_w\bigl(\boldsymbol{\Lambda}_s + \lambda_r\boldsymbol{\Lambda}_r + \sum_{m=1}^{M}\xi_{mk}\mathbf{L}_{mk}\bigr)}
#' where \eqn{\lambda_w} is the global wiggle penalty (\code{wiggle_penalty}),
#' \eqn{\lambda_r} is ridge penalty on linear and intercept terms
#' (\code{flat_ridge_penalty}) multiplied by the wiggle penalty, and
#' \eqn{\xi_{mk}} and \eqn{\mathbf{L}_{mk}} denote optional additional
#' penalty multipliers and matrices, including the predictor- and
#' partition-specific components activated through
#' \code{unique_penalty_per_predictor}, \code{unique_penalty_per_partition},
#' \code{predictor_penalties}, and \code{partition_penalties}. This assembly is handled by
#' \code{\link{compute_Lambda}}.
#'
#' The penalty matrix \eqn{\boldsymbol{\Lambda}} is stored as a list
#' of \eqn{K+1} \eqn{p \times p} square, symmetric, positive semi-definite
#' matrices.
#' }
#'
#' \subsection{Penalty Optimization via Leave-One-Out or Generalized Cross-Validation}{
#' After the structural pieces of the model are fixed, the main remaining
#' question is how much smoothing to apply. In \pkg{lgspline}, that tuning is
#' performed with exact leave-one-out by default, or with generalized
#' cross-validation when \code{tuning_criterion = "gcv"}. In either case, the
#' criterion is evaluated using the same constrained estimator that will be
#' used in the final model fit.
#' Penalty parameters are estimated on the log scale via exponential
#' parameterization (\eqn{\lambda = \exp(\theta)},
#' \eqn{\theta \in \mathbb{R}}), ensuring positivity. The chain rule factor
#' \eqn{\partial\exp(\theta)/\partial\theta = \exp(\theta) = \lambda} is
#' applied throughout. User-facing arguments (\code{initial_wiggle},
#' \code{initial_flat}, \code{predictor_penalties},
#' \code{partition_penalties}) accept values on the raw, natural scale;
#' conversion to log scale is handled internally. The final tuned values and
#' assembled penalty pieces are returned in the fitted object's
#' \code{penalties} component.
#'
#' The total penalty matrix \eqn{\boldsymbol{\Lambda}} is constructed as:
#' \deqn{\boldsymbol{\Lambda} = \lambda_w\mathbf{L}_{w} + \lambda_r\mathbf{L}_{r}
#'   + \sum_{j}\nu_j\mathbf{L}_j^{(\mathrm{pred})}
#'   + \sum_{k}\tau_k\mathbf{L}_k^{(\mathrm{part})},}
#' where \eqn{\mathbf{L}_{w}} is the integrated squared second-derivative
#' penalty (i.e., \eqn{\boldsymbol{\Lambda}_s} above), \eqn{\mathbf{L}_{r}} is
#' a ridge penalty on intercept and linear coefficients,
#' \eqn{\mathbf{L}_j^{(\mathrm{pred})}} are predictor-specific penalties, and
#' \eqn{\mathbf{L}_k^{(\mathrm{part})}} are partition-specific penalties. The
#' scalars \eqn{\lambda_w} (\code{wiggle_penalty}), \eqn{\lambda_r}
#' (\code{flat_ridge_penalty}), \eqn{\{\nu_j\}}
#' (\code{predictor_penalties}), and \eqn{\{\tau_k\}}
#' (\code{partition_penalties}) are tuned.
#' With \code{tuning_criterion = "loo"}, the criterion is
#' \deqn{\mathrm{LOO}
#'   = \frac{1}{N}\sum_{i=1}^{N}\left(\frac{r_i}{1-h_{ii}}\right)^2,}
#' where \eqn{h_{ii}} is the diagonal of the tuning hat matrix. For Gaussian
#' identity-link tuning with fixed transformed design, this is exact. With
#' correlation structure and/or non-identity links, \pkg{lgspline} applies the
#' same transformed or linearized tuning problem already used by GCV, so the
#' leave-one-out calculation is carried out on that working scale. In empirical
#' diagnostics, the criterion itself behaves stably, while the fully analytic
#' derivative of the observation-wise leverage term can be numerically delicate
#' in some problems; when that sensitivity matters in practice, the finite-
#' difference optimizer path (\code{use_custom_bfgs = FALSE}) remains
#' available. This exact observation-wise calculation is also the main reason
#' LOO tuning becomes more expensive than GCV as \eqn{N} grows; in routine use,
#' \code{tuning_criterion = "gcv"} is often the more practical choice once the
#' sample size is above about 250,000.
#'
#' With \code{tuning_criterion = "gcv"}, the package instead uses
#' \deqn{\mathrm{GCV}_{u,\gamma}
#'   = \frac{\sum_{i=1}^{N}D_{ii}\,r_i^{2}}
#'           {N(1 - \gamma\bar{W})^{2}}}
#' where \eqn{\bar{W} = \mathrm{tr}(\mathbf{H})/N} is the mean of the
#' hat-matrix diagonal. The symbol \eqn{\gamma} is set by
#' \code{gcv_gamma} and controls optional inflation of the
#' effective-degrees-of-freedom term.
#'
#' For identity link, \eqn{r_i = y_i - \hat{\eta}_i}. For non-identity links,
#' the residuals are
#' \eqn{r_i = g((y_i + \delta)/(1+2\delta)) - (\hat{\eta}_i + \delta)/(1+2\delta)},
#' where \eqn{\delta \geq 0} is a pseudocount that stabilizes the link
#' transformation, automatically tuned within \code{\link{tune_Lambda}} if not supplied
#' to \code{delta}.
#' Non-Gaussian families with observation weights
#' \eqn{\omega_i} have their residuals scaled by \eqn{\omega_i}. When the
#' family provides a custom deviance residual function, that function is used
#' in place of the link-scale residuals.
#'
#' \strong{Pseudocount selection.} The pseudocount \eqn{\delta} is chosen to
#' make the transformed response distribution most closely approximate a
#' \eqn{t}-distribution with \eqn{N-1} degrees of freedom, in the sense of
#' minimizing the (optionally weighted) mean absolute deviation between the
#' sorted standardized transformed responses and the corresponding
#' \eqn{t}-quantiles. This is solved via Brent's method over
#' \eqn{[10^{-64}, 1]}. When the link is identity, or when the response
#' naturally lies in the domain of the link function, \eqn{\delta = 0}.
#' This behavior is exposed through the \code{delta} argument in
#' \code{\link{lgspline}}: supplying a fixed numeric value bypasses the internal
#' search, while leaving it \code{NULL} allows the tuning code to choose the
#' stabilizing pseudocount automatically when needed.
#'
#' \strong{Meta-penalty regularization.} A regularization term pulls the
#' predictor- and partition-specific penalty parameters toward 1 on the raw
#' scale:
#' \deqn{P_{\mathrm{meta}}(\lambda_w, \nu_j, \tau_k)
#'   = \frac{1}{2}c_{\mathrm{meta}}\sum_j(\nu_j - 1)^{2}
#'   + \frac{1}{2}\cdot 10^{-32}(\lambda_w - 1)^{2}}
#' where \eqn{c_{\mathrm{meta}}} is a user-specified coefficient
#' (\code{meta_penalty}). The gradient of \eqn{P_{\mathrm{meta}}} on the log
#' scale, incorporating the exp chain rule, is
#' \eqn{\partial P_{\mathrm{meta}}/\partial\theta_j = c_{\mathrm{meta}}(\nu_j - 1)\nu_j}
#' and
#' \eqn{\partial P_{\mathrm{meta}}/\partial\theta_1 = 10^{-32}(\lambda_w - 1)\lambda_w}.
#' The total objective is the selected tuning criterion plus
#' \eqn{P_{\mathrm{meta}}}.
#' }
#'
#' \subsection{Closed-Form Gradients for Tuning Criteria}{
#' The gradient of the GCV objective with respect to
#' \eqn{\theta_1 = \log\lambda_w} is computed analytically via the quotient
#' rule:
#' \deqn{\frac{\partial\mathrm{GCV}_{u,\gamma}}{\partial\theta_1}
#'   = \frac{1}{D^{2}}\left(\frac{\partial\mathcal{N}}{\partial\theta_1}D
#'   - \mathcal{N}\frac{\partial D}{\partial\theta_1}\right),}
#' where \eqn{\mathcal{N} = \sum r_i^{2}} (numerator) and
#' \eqn{D = N(1 - \gamma\bar{W})^{2}} (denominator). The key intermediates are:
#' \itemize{
#'   \item \eqn{\partial\mathbf{G}/\partial\lambda_w}, computed from the
#'     matrix identity
#'     \eqn{\partial(\mathbf{X}^{\top}\mathbf{X} + \boldsymbol{\Lambda})^{-1}/\partial\lambda = -\mathbf{G}(\partial\boldsymbol{\Lambda}/\partial\lambda)\mathbf{G}}.
#'   \item \eqn{\partial\mathbf{G}^{1/2}/\partial\lambda_w}, derived from
#'     \eqn{\partial\mathbf{G}/\partial\lambda_w} via the eigendecomposition
#'     chain rule.
#'   \item \eqn{\partial\bar{W}/\partial\lambda_w}, the derivative of the
#'     trace of the hat matrix
#'     \eqn{\mathbf{H} = \mathbf{X}\mathbf{U}\mathbf{G}\mathbf{X}^{\top}},
#'     which depends on both
#'     \eqn{\partial\mathbf{G}/\partial\lambda_w} and
#'     \eqn{\partial\mathbf{G}^{1/2}/\partial\lambda_w}.
#'   \item \eqn{\partial\mathcal{N}/\partial\theta_1 = -2\mathbf{r}^{\top}\mathbf{X}(\partial(\mathbf{U}\mathbf{G})/\partial\lambda_w)\mathbf{X}^{\top}\mathbf{y}\cdot\lambda_w},
#'     via the chain rule applied to the residual vector.
#'   \item \eqn{\partial D/\partial\theta_1 =
#'     2(1 - \gamma\bar{W})(-\gamma\,\partial\bar{W}/\partial\lambda_w)\cdot\lambda_w}.
#' }
#' In the implementation, these quantities are assembled by a small set of
#' helper routines: \code{\link{compute_dG_dlambda}} for
#' \eqn{\partial\mathbf{G}/\partial\lambda}, \code{\link{compute_dGhalf}}
#' for \eqn{\partial\mathbf{G}^{1/2}/\partial\lambda},
#' \code{\link{compute_trace_H}} together with the tuning hat-diagonal
#' derivative helpers for the effective-degrees-of-freedom term, and
#' \code{\link{compute_dG_u_dlambda_xy}} for the derivative of the fitted-value
#' quadratic form.
#'
#' For exact leave-one-out, the implementation instead differentiates the
#' per-observation PRESS form directly, requiring both
#' \eqn{\partial r_i / \partial\lambda_w} and
#' \eqn{\partial h_{ii} / \partial\lambda_w}. The latter is assembled
#' partition-wise from the same constrained-\eqn{\mathbf{G}} ingredients used
#' elsewhere in tuning. The exact LOO path does not explicitly form the full
#' projection matrix \eqn{\mathbf{U}} or the full hat matrix \eqn{\mathbf{H}};
#' instead it works with blockwise matrices
#' \eqn{\mathbf{C}_k = \mathbf{G}_k - \mathbf{G}_k\mathbf{A}_k(\mathbf{A}^\top\mathbf{G}\mathbf{A})^{-1}\mathbf{A}_k^\top\mathbf{G}_k}
#' and their derivatives. Empirically, this observation-wise leverage
#' derivative is the most numerically delicate part of the analytic LOO
#' gradient: most observations are typically well behaved, but a diffuse set of
#' small local errors can remain even when the overall LOO objective and the
#' corresponding GCV gradient are stable.
#'
#' The shared wiggle and flat directions are differentiated directly. For
#' predictor- and partition-specific penalties, a lower-cost ratio heuristic is
#' used:
#' \deqn{\frac{\partial\mathrm{GCV}_{u,\gamma}}{\partial\theta_l} \approx
#'   \frac{\mathrm{mean}(\mathrm{diag}(\mathbf{L}_l))}{\mathrm{mean}(\mathrm{diag}(\boldsymbol{\Lambda}))}
#'   \frac{\partial\mathrm{GCV}_{u,\gamma}}{\partial\theta_w},}
#' where \eqn{\theta_l = \log \lambda_l} and
#' \eqn{\mathbf{L}_l} denotes the current contribution of the corresponding
#' predictor- or partition-specific penalty to \eqn{\boldsymbol{\Lambda}}. This
#' keeps the extra penalty directions inexpensive while preserving the direct
#' derivatives for the shared wiggle and flat penalties.
#' }
#'
#' \subsection{Optimization Procedure}{
#' \strong{Grid search initialization.} The selected tuning criterion is
#' evaluated over a grid of candidate values for
#' \eqn{(\lambda_w, \lambda_r)} on the log scale. All combinations of
#' user-supplied candidate vectors (\code{initial_wiggle} and
#' \code{initial_flat}) are formed, and the combination yielding
#' the smallest finite criterion value is selected as the starting
#' point for BFGS optimization. Grid points producing non-finite
#' objective values are discarded. If all grid points fail, an error
#' is raised advising the user to check the data or adjust the grid.
#'
#' \strong{Damped BFGS optimizer.} A custom damped BFGS quasi-Newton optimizer,
#' implemented in \code{\link{tune_Lambda}} through
#' \code{.damped_bfgs()}, minimizes the selected tuning criterion plus
#' \eqn{P_{\mathrm{meta}}}. When analytic gradients are not preferred, or when
#' the exact LOO leverage derivative appears too noisy for a given problem, a
#' base-R \code{\link[stats]{optim}} call with method \code{"BFGS"} provides
#' the finite-difference fallback.
#'
#' \emph{Iterations 1-2: steepest descent.} The first two iterations use
#' steepest descent with a damping factor \eqn{\alpha}:
#' \eqn{\boldsymbol{\phi}^{(t+1)} = \boldsymbol{\phi}^{(t)} - \alpha\nabla_{\boldsymbol{\phi}}}.
#'
#' \emph{Iterations 3+: BFGS.} From iteration 3, an inverse Hessian
#' approximation \eqn{\mathbf{J}^{(t)}} is maintained via the standard secant
#' update. Let
#' \eqn{\mathbf{s}^{(t)} = \boldsymbol{\phi}^{(t)} - \boldsymbol{\phi}^{(t-1)}}
#' and
#' \eqn{\mathbf{v}^{(t)} = \nabla^{(t)} - \nabla^{(t-1)}}. The BFGS update
#' is:
#' \deqn{\mathbf{J}^{(t+1)}
#'   = (\mathbf{I} - \mathbf{u}\mathbf{s}\mathbf{v}^{\top})
#'   \mathbf{J}^{(t)}
#'   (\mathbf{I} - \mathbf{u}\mathbf{v}\mathbf{s}^{\top})
#'   + \mathbf{u}\mathbf{s}\mathbf{s}^{\top},
#'   \qquad \mathbf{u} = (\mathbf{v}^{\top}\mathbf{s})^{-1}.}
#' When \eqn{|\mathbf{v}^{\top}\mathbf{s}| < 10^{-64}}, the approximation is
#' reset to \eqn{\mathbf{I}} and the iteration is flagged for restart. The
#' search direction is
#' \eqn{\mathbf{d}^{(t)} = -\mathbf{J}^{(t)}\nabla^{(t)}}.
#'
#' \emph{Step acceptance.} A step is accepted if the new tuning criterion value
#' is no larger than the old one.
#' On rejection, \eqn{\alpha} is halved. If \eqn{\alpha < 2^{-10}} (early
#' iterations) or \eqn{\alpha < 2^{-12}} (later iterations), the optimizer
#' terminates with the best solution found.
#'
#' \emph{Convergence.} The optimizer terminates when
#' the absolute change in the tuning criterion is less than \eqn{\epsilon}, or
#' \eqn{\|\boldsymbol{\phi}^{(t)} - \boldsymbol{\phi}^{(t-1)}\|_{\infty} < \epsilon},
#' provided at least 10 iterations have elapsed, for penalties
#' \eqn{\boldsymbol{\phi} = \lambda_w, \lambda_r, \nu_j, \tau_j, ...}.
#'
#' \emph{Alternative.} A base-R
#' \code{\link[stats]{optim}} call with method \code{"BFGS"} and
#' finite-difference gradients can be used instead via
#' \code{use_custom_bfgs = FALSE}.
#'
#' \strong{Post-optimization sample-size adjustment.} After optimization, the
#' tuned penalty parameters are divided by \eqn{((N+2)/(N-2))^{2}}
#' (equivalently multiplied by \eqn{((N-2)/(N+2))^{2}}) for both GCV- and
#' LOO-based tuning. This decreases the final penalties at small sample sizes.
#'
#' The tuning loop is implemented in \code{\link{tune_Lambda}}.
#' }
#'
#' @section Incorporating Non-Spline Effects:
#'
#' Multiple fixed effects are accommodated naturally in the LMSS framework
#' because spline effects, linear effects, and many interaction terms all live
#' in the same partition-wise polynomial expansion. The distinction is therefore
#' not whether a term is "allowed" by the solver, but whether it receives full
#' spline treatment or remains structurally linear across partitions.
#'
#' The constrained framework naturally accommodates non-spline terms. If only
#' linear terms are included for a predictor (via
#' \code{just_linear_without_interactions} or
#' \code{just_linear_with_interactions}), the first-derivative smoothing
#' constraint forces the linear coefficient to be identical across all
#' partitions, since the derivative of a linear function is its slope. This
#' is not an algorithmic modification but a natural consequence of the
#' constraint structure.
#'
#' For example, a model with one spline effect and a linear treatment indicator
#' interaction will naturally keep the treatment-time interaction coefficient
#' constant across partitions while allowing the time effect to vary
#' nonlinearly. This conveniently extends to arbitrary combinations of spline
#' and linear terms without requiring special handling.
#'
#' When \code{blockfit = TRUE} is specified alongside
#' \code{just_linear_without_interactions}, the flat-block path provides an
#' alternative enforcement mechanism. Rather than relying on constraint
#' projection, flat coefficients are pooled structurally across partitions
#' during backfitting. The two approaches agree at the point estimate but
#' differ in their uncertainty quantification; see the Blockfit section above.
#'
#' @section Integration:
#'
#' Because the fitted object retains an explicit polynomial representation in
#' each partition, numerical integration can be carried out in a fairly direct
#' way. The package wraps that calculation in a user-facing S3 method so the
#' user does not need to manage knot boundaries or partition membership by hand.
#'
#' In the user-facing interface, numerical integration is applied through
#' \code{\link{integrate.lgspline}}, which applies Gauss-Legendre quadrature to
#' predictions from the fitted model produced by \code{\link{predict.lgspline}}.
#'
#' \subsection{Implementation}{
#' For a user-supplied rectangular domain, \code{\link{integrate.lgspline}} constructs a
#' tensor-product grid of Gauss-Legendre nodes, evaluates the fitted model at
#' those points, and forms the weighted sum. This works for both univariate and
#' multivariate models, respects the fitted partition structure automatically,
#' and avoids requiring the user to keep track of knot boundaries by hand.
#'
#' The \code{vars} argument selects which predictors are integrated over.
#' Predictors not listed in \code{vars} are held fixed at
#' \code{initial_values} when supplied, or otherwise at the midpoint of their
#' observed training range. The optional \code{B_predict} argument makes it
#' possible to integrate posterior draws or other alternate coefficient sets,
#' and \code{n_quad} controls the number of Gauss-Legendre nodes used per
#' integrated dimension.
#'
#' Integration is performed on the response scale by default. Setting
#' \code{link_scale = TRUE} instead integrates the linear predictor
#' \eqn{\eta = f(\mathbf{t})}. For identity-link Gaussian models the two scales
#' coincide.
#' }
#'
#'
#' @section Lagrange Multipliers:
#'
#' When \code{return_lagrange_multipliers = TRUE}, the multiplier vector
#' \deqn{\boldsymbol{\lambda} = (\mathbf{A}^{\top}\mathbf{G}\mathbf{A})^{-1}\mathbf{A}^{\top}\hat{\boldsymbol{\beta}}}
#' is returned. These quantify the sensitivity of the penalized objective to
#' relaxing each smoothness or user-supplied equality constraint. When
#' constraint target values are nonzero
#' (\eqn{\mathbf{A}^{\top}\boldsymbol{\beta}_0 \neq \mathbf{0}}), the modified
#' formulation is used:
#' \deqn{\boldsymbol{\lambda} = (\mathbf{A}^{\top}\mathbf{G}\mathbf{A})^{-1}\mathbf{A}^{\top}(\hat{\boldsymbol{\beta}} - \boldsymbol{\beta}_0)}
#' where \eqn{\mathbf{A}^{\top}\boldsymbol{\beta}_0} is the vector of
#' constraint target values. Multipliers are \code{NULL} when no constraints
#' are active (\eqn{\mathbf{A}} is \code{NULL} or \eqn{K = 0}).
#'
#' For inequality constraints, multipliers are returned as computed by
#' \code{\link[quadprog]{solve.QP}}. The Lagrange multipliers for active
#' inequality constraints can be used diagnostically to identify which shape
#' constraints are most costly in terms of goodness of fit.
#'
#' @section S3 Methods:
#'
#' Standard S3 methods are provided for objects of class \code{lgspline}:
#' \itemize{
#'   \item \code{\link{print.lgspline}} and \code{\link{summary.lgspline}}:
#'     Provide concise model summaries, with
#'     \code{\link{print.summary.lgspline}} formatting coefficient tables in a
#'     familiar regression-style layout.
#'
#'   \item \code{\link{logLik.lgspline}}: Returns a standard \code{logLik}
#'     object. For Gaussian responses with identity link, the exact
#'     log-likelihood is computed. When a correlation structure is present via
#'     \code{VhalfInv}, the log-likelihood includes the
#'     \eqn{\log|\mathbf{V}^{-1/2}|} adjustment and the corresponding
#'     whitened quadratic form. For other families, the method falls back to
#'     \code{family$aic} or a deviance-based approximation. An
#'     \code{include_prior} argument (default \code{TRUE}) optionally adds
#'     the Gaussian prior penalty interpretation of the smoothing spline
#'     penalty
#'     \eqn{-\frac{1}{2\sigma^{2}}\tilde{\boldsymbol{\beta}}^{\top}\boldsymbol{\Lambda}\tilde{\boldsymbol{\beta}}}
#'     to obtain a penalized MAP log-likelihood.
#'
#'   \item \code{\link{predict.lgspline}}: Produces fitted values and related
#'     quantities (e.g., derivatives and standard errors through
#'     \code{se.fit = TRUE}), lets \code{new_predictors} override
#'     \code{newdata}, accepts alternate coefficient lists through
#'     \code{B_predict}, and supports prediction on new predictor matrices
#'     consistent with the original spline expansions.
#'
#'   \item \code{\link{coef.lgspline}}: Extracts partition-specific
#'     coefficient vectors.
#'
#'   \item \code{\link{confint.lgspline}}: Extracts confidence intervals.
#'     When the inverse Hessian from BFGS optimization is available for
#'     correlation parameters, intervals for those correlation parameters are
#'     returned on the working (transformed) scale and should be
#'     back-transformed as described in the correlation section.
#'   \item \code{\link{plot.lgspline}}: For one-dimensional fits, produces
#'     base R graphics showing the fitted function (with optional partition-wise
#'     formulas) and supports overlay via \code{add = TRUE}. For two or more
#'     predictors, an interactive \pkg{plotly}-based visualization is
#'     returned. Specific predictors may be selected via \code{vars}.
#'
#'   \item \code{\link{integrate.lgspline}}: Computes definite integrals of
#'     the fitted surface over rectangular domains by Gauss-Legendre
#'     quadrature.
#' }
#'
#' Additional user-facing helpers include \code{\link{wald_univariate}} for
#' coefficient-wise Wald inference, \code{\link{generate_posterior}} for
#' posterior and posterior-predictive sampling,
#' \code{\link{generate_posterior_correlation}} for correlation-aware posterior
#' simulation, \code{\link{equation}} for closed-form display of the fitted
#' partition formulas, and
#' \code{\link{find_extremum}} for optimizing the fitted surface or a custom
#' acquisition function built from it.
#'
#' @references
#'
#' Buse, A. and Lim, L. (1977). Cubic Splines as a Special Case of
#' Restricted Least Squares. \emph{Journal of the American Statistical
#' Association}, 72, 64-68.
#'
#' Eilers, P. H. and Marx, B. D. (1996). Flexible Smoothing with B-splines
#' and Penalties. \emph{Statistical Science}, 11(2), 89-121.
#'
#' Ezhov, N., Neitzel, F. and Petrovic, S. (2018). Spline Approximation,
#' Part 1: Basic Methodology. \emph{Journal of Applied Geodesy}, 12(2),
#' 139-155.
#'
#' Goldfarb, D. and Idnani, A. (1983). A Numerically Stable Dual Method for
#' Solving Strictly Convex Quadratic Programs. \emph{Mathematical
#' Programming}, 27(1), 1-33.
#'
#' Harville, D. A. (1977). Maximum Likelihood Approaches to Variance
#' Component Estimation and to Related Problems. \emph{Journal of the
#' American Statistical Association}, 72(358), 320-338.
#'
#' Hastie, T. J. and Tibshirani, R. J. (1990). \emph{Generalized Additive
#' Models}. Chapman & Hall/CRC.
#'
#' Kisi, O., Heddam, S., Parmar, K. S., Petroselli, A., Kulls, C. and
#' Zounemat-Kermani, M. (2025). Integration of Gaussian Process Regression
#' and K Means Clustering for Enhanced Short Term Rainfall Runoff Modeling.
#' \emph{Scientific Reports}, 15, 7444.
#'
#' MacQueen, J. B. (1967). Some Methods for Classification and Analysis of
#' Multivariate Observations. In \emph{Proceedings of the Fifth Berkeley
#' Symposium on Mathematical Statistics and Probability}, Volume 1,
#' 281-297. University of California Press.
#'
#' McCullagh, P. and Nelder, J. A. (1989). \emph{Generalized Linear Models}.
#' Chapman & Hall, 2nd edition.
#'
#' Murray, I., Adams, R. P. and MacKay, D. J. C. (2010). Elliptical Slice
#' Sampling. \emph{Proceedings of the 13th International Conference on
#' Artificial Intelligence and Statistics (AISTATS)}, 9, 541-548.
#'
#' Nocedal, J. and Wright, S. J. (2006). \emph{Numerical Optimization}
#' (2nd ed.). Springer.
#'
#' Kim, Y.-J. and Gu, C. (2004). Smoothing Spline Gaussian Regression:
#' More Scalable Computation via Efficient Approximation. \emph{Journal of
#' the Royal Statistical Society: Series B}, 66(2), 337-356.
#'
#' Patterson, H. D. and Thompson, R. (1971). Recovery of Inter-Block
#' Information When Block Sizes Are Unequal. \emph{Biometrika}, 58,
#' 545-554.
#'
#' Pya, N. and Wood, S. N. (2015). Shape Constrained Additive Models.
#' \emph{Statistics and Computing}, 25(3), 543-559.
#'
#' Reinsch, C. H. (1967). Smoothing by Spline Functions. \emph{Numerische
#' Mathematik}, 10, 177-183.
#'
#' Ruppert, D., Wand, M. P. and Carroll, R. J. (2003).
#' \emph{Semiparametric Regression}. Cambridge University Press.
#'
#' Searle, S. R., Casella, G. and McCulloch, C. E. (2006). \emph{Variance
#' Components}. Wiley.
#'
#' Wahba, G. (1990). \emph{Spline Models for Observational Data}. SIAM.
#'
#' Wood, S. N. (2006). On Confidence Intervals for Generalized Additive
#' Models Based on Penalized Regression Splines. \emph{Australian & New
#' Zealand Journal of Statistics}, 48(4), 445-464.
#'
#' Wood, S. N. (2011). Fast Stable Restricted Maximum Likelihood and Marginal
#' Likelihood Estimation of Semiparametric Generalized Linear Models.
#' \emph{Journal of the Royal Statistical Society: Series B}, 73(1), 3-36.
#'
#' Wood, S. N. (2017). \emph{Generalized Additive Models: An Introduction
#' with R}. CRC Press, 2nd edition.
#'
#' @name Details
NULL
