# 5. Agreement and ICC for Wide Data

## Scope

Agreement and reliability are related to correlation, but they are not the same problem. Correlation describes co-movement. Agreement describes similarity on the measurement scale itself. Reliability describes the proportion of variation attributable to stable differences among subjects rather than to measurement error or method disagreement.

This vignette focuses on `ccc()` for wide data and uses related agreement and reliability functions as context:

- `ccc()`
- `ba()`
- `icc()`
- `ccc_rm_reml()` and `ccc_rm_ustat()`
- `icc_rm_reml()`
- `cia()` and `cia_rm()`

## Pairwise concordance and Bland-Altman analysis

Lin’s concordance correlation coefficient combines precision and accuracy in a single number. In `matrixCorr`, `ccc()` computes Lin’s pairwise CCC for numeric wide data and optionally returns large-sample confidence intervals. No formal hypothesis test is implemented; inference is based on the estimate and its confidence interval.

Bland-Altman analysis separates the agreement question into estimated bias and limits of agreement.

    library(matrixCorr)

    set.seed(40)
    ref <- rnorm(50, mean = 100, sd = 10)
    m1 <- ref + rnorm(50, sd = 2)
    m2 <- ref + 1.2 + rnorm(50, sd = 3)

    fit_ba <- ba(m1, m2)
    fit_ccc <- ccc(data.frame(m1 = m1, m2 = m2), ci = TRUE)

    print(fit_ba)
    #> Bland-Altman preview:
    #>   based_on    : 50
    #>   loa_rule    : mean +/- 1.96 * SD
    #>   ci          : 95%
    #>   sd_diff     : 3.722
    #>   width       : 14.589
    #>
    #>  quantity        estimate lwr     upr
    #>  Mean difference -1.290   -2.347  -0.232
    #>  Lower LoA       -8.584   -10.416 -6.752
    #>  Upper LoA       6.005    4.173   7.837
    summary(fit_ccc)
    #> Lin's concordance summary
    #>   dimensions  : 2 x 2
    #>   pairs       : 1
    #>   estimate    : 0.9299
    #>   most_negative: m1-m2 (0.9299)
    #>   most_positive: m1-m2 (0.9299)
    #>   ci          : 95%
    #>   ci_method   : lin_delta_fisher_z
    #>   ci_width    : 0.08
    #>
    #>  item1 item2 estimate n  95% CI
    #>  m1    m2    0.9299   50 [0.88, 0.96]
    estimate(fit_ccc)
    #>           m1        m2
    #> m1 1.0000000 0.9299231
    #> m2 0.9299231 1.0000000
    confint(fit_ccc)
    #>   item1 item2       lwr       upr
    #> 1    m1    m2 0.8804487 0.9593657
    ci(fit_ccc)
    #> $lwr
    #>           m1        m2
    #> m1 1.0000000 0.8804487
    #> m2 0.8804487 1.0000000
    #>
    #> $upr
    #>           m1        m2
    #> m1 1.0000000 0.9593657
    #> m2 0.9593657 1.0000000
    #>
    #> $conf.level
    #> [1] 0.95
    #>
    #> $ci.method
    #> [1] "lin_delta_fisher_z"
    tidy(fit_ccc)
    #>     item1 item2  estimate       lwr       upr
    #> row    m1    m2 0.9299231 0.8804487 0.9593657

The two summaries are complementary rather than redundant. `ccc()` gives a single concordance coefficient, while `ba()` makes the scale of disagreement explicit.

If you have at least 3 methods in wide form, `ba()` can now compute every unordered Bland-Altman contrast directly:

    fit_ba_pairwise <- ba(data.frame(m1 = m1, m2 = m2, m3 = ref - 0.8 + rnorm(50, sd = 2.5)))
    print(fit_ba_pairwise)
    #>
    #> Bland-Altman (row - column) (95% CI)
    #>
    #>  bias   sd_loa loa_low loa_up width  n_obs
    #>  -1.290 3.722  -8.584   6.005 14.589 50
    #>   1.502 3.461  -5.280   8.285 13.565 50
    #>   2.792 4.515  -6.058  11.642 17.700 50
    summary(fit_ba_pairwise)
    #>
    #> Bland-Altman (pairwise row - column) (95% CI)
    #>
    #> Agreement estimates
    #>
    #>  n_obs bias   sd_loa loa_low loa_up width  loa_multiplier
    #>  50    -1.290 3.722  -8.584   6.005 14.589 1.96
    #>  50     1.502 3.461  -5.280   8.285 13.565 1.96
    #>  50     2.792 4.515  -6.058  11.642 17.700 1.96
    #>
    #> Confidence intervals
    #>
    #>  bias_lwr bias_upr lo_lwr  lo_upr up_lwr up_upr
    #>  -2.347   -0.232   -10.416 -6.752 4.173   7.837
    #>   0.519    2.486    -6.984 -3.577 6.582   9.988
    #>   1.509    4.075    -8.281 -3.835 9.419  13.864

## Pairwise ICC

`icc()` extends the wide-data reliability workflow in two directions. It can return a pairwise matrix across method pairs, or it can return the overall classical ICC table for the full set of methods.

    wide_methods <- data.frame(
      J1 = ref + rnorm(50, sd = 1.5),
      J2 = ref + 4.0 + rnorm(50, sd = 1.8),
      J3 = ref - 3.0 + rnorm(50, sd = 2.0),
      J4 = ref + rnorm(50, sd = 1.6)
    )

    fit_icc_pair <- icc(
      wide_methods,
      model = "twoway_random",
      type = "agreement",
      unit = "single",
      scope = "pairwise"
    )

    fit_icc_overall <- icc(
      wide_methods,
      model = "twoway_random",
      type = "agreement",
      unit = "single",
      scope = "overall",
      ci = TRUE
    )

    print(fit_icc_pair, digits = 2)
    #> Intraclass correlation matrix
    #>   method      : Intraclass correlation (two-way random, agreement, single)
    #>   dimensions  : 4 x 4
    #>
    #>      J1   J2   J3   J4
    #> J1 1.00 0.90 0.94 0.98
    #> J2 0.90 1.00 0.80 0.91
    #> J3 0.94 0.80 1.00 0.93
    #> J4 0.98 0.91 0.93 1.00
    summary(fit_icc_pair)
    #> Intraclass correlation summary
    #>   method      : Intraclass correlation (two-way random, agreement, single)
    #>   dimensions  : 4 x 4
    #>   pairs       : 6
    #>   n_complete  : 50
    #>   estimate    : 0.7961 to 0.9817
    #>   most_negative: J2-J3 (0.7961)
    #>   most_positive: J1-J4 (0.9817)
    #>
    #>  item1 item2 estimate n
    #>  J1    J4    0.9817   50
    #>  J1    J3    0.9398   50
    #>  J3    J4    0.9329   50
    #>  J2    J4    0.9136   50
    #>  J1    J2    0.9043   50
    #>  J2    J3    0.7961   50
    print(fit_icc_overall)
    #> Overall intraclass correlation
    #>   method      : Overall intraclass correlation table
    #>   subjects    : 50
    #>   raters      : 4
    #>   selected    : ICC2
    #>
    #> Coefficient table
    #>
    #>  coefficient label            estimate ... upr    selected
    #>  ICC1        Single absolute  0.9050   ... 0.9400 FALSE
    #>  ICC2        Single random    0.9067   ... 0.9647  TRUE
    #>  ICC3        Single fixed     0.9757   ... 0.9850 FALSE
    #>  ICC1k       Average absolute 0.9744   ... 0.9843 FALSE
    #>  ICC2k       Average random   0.9749   ... 0.9909 FALSE
    #>  ICC3k       Average fixed    0.9938   ... 0.9962 FALSE
    #> ... 5 more variables not shown (omitted)
    #> Use as.data.frame()/tidy()/as.matrix() to inspect the full result.

## Pairwise versus overall ICC

This is the most important distinction in the ICC interface.

`scope = "pairwise"` answers: “How reliable is each specific pair of methods?”

`scope = "overall"` answers: “How reliable is the full set of methods when analysed jointly?”

Those are different quantities. The overall ICC cannot, in general, be recovered by averaging the pairwise matrix.

## Consistency versus agreement

This simulation also includes systematic method bias, so it is a natural place to contrast `type = "consistency"` with `type = "agreement"`.

    fit_icc_cons <- icc(
      wide_methods,
      model = "twoway_random",
      type = "consistency",
      unit = "single",
      scope = "overall",
      ci = FALSE
    )

    fit_icc_agr <- icc(
      wide_methods,
      model = "twoway_random",
      type = "agreement",
      unit = "single",
      scope = "overall",
      ci = FALSE
    )

    data.frame(
      type = c("consistency", "agreement"),
      selected_coefficient = c(
        attr(fit_icc_cons, "selected_coefficient"),
        attr(fit_icc_agr, "selected_coefficient")
      ),
      estimate = c(
        attr(fit_icc_cons, "selected_row")$estimate,
        attr(fit_icc_agr, "selected_row")$estimate
      )
    )
    #>          type selected_coefficient  estimate
    #> 1 consistency                 ICC3 0.9757153
    #> 2   agreement                 ICC2 0.9066943

Consistency discounts additive method shifts, whereas agreement penalises them. When methods differ mainly by a systematic offset, consistency can therefore look substantially better than agreement.

## Model, type, and unit

The classical ICC family is controlled by three arguments.

- `model` selects the one-way, two-way random, or two-way mixed formulation.
- `type` selects consistency or agreement.
- `unit` selects single-measure or average-measure reliability.

For pairwise ICC, average-measure output uses `k = 2` because each estimate is based on exactly two methods. For overall ICC, average-measure output uses the full number of analysed columns.

## Agreement indices implemented in matrixCorr

CCC, ICC and CIA address related but distinct agreement questions. CCC quantifies agreement between measurements by combining precision and accuracy (Lin, 1989). ICC expresses reliability or agreement through variance components; its interpretation depends on the study design, model form, and whether agreement or consistency is targeted (Shrout and Fleiss, 1979; McGraw and Wong, 1996). CIA targets individual agreement or interchangeability by comparing disagreement between methods with disagreement within methods (Barnhart, Kosinski, and Haber, 2007; Barnhart et al., 2007).

These indices should not be treated as interchangeable without considering the scientific question, data structure, and implemented estimator.

### CCC functions

`ccc()` is the simple wide-data CCC implementation. It estimates pairwise Lin CCC values and, when `ci = TRUE`, returns Lin delta-method/Fisher-z confidence intervals. It does not report a p-value or test decision.

For repeated-measures data, the package provides two CCC routes. `ccc_rm_reml()` fits pairwise mixed models by REML and converts the fitted variance components and fixed-effect bias term into repeated-measures CCC estimates. `ccc_rm_ustat()` computes a nonparametric U-statistic repeated-measures CCC with optional Fisher-z confidence intervals. These repeated-measures functions estimate agreement parameters and optional confidence intervals; they do not implement a formal hypothesis test of the CCC parameter. `ccc_rm_reml()` can use boundary-aware likelihood-ratio tests for variance-component selection when `vc_select = "auto"`, but those tests are model-selection diagnostics rather than tests of CCC agreement.

### ICC functions

`icc()` computes classical ANOVA ICC forms for wide data. With `scope = "pairwise"` it returns a pairwise matrix; with `scope = "overall"` it returns the standard six-form overall coefficient table. The selected ICC form is controlled by `model`, `type`, and `unit`, so the same numeric value should not be interpreted without those design choices.

Pairwise `icc()` and repeated-measures `icc_rm_reml()` provide estimates and optional confidence intervals without a formal test of a target ICC parameter. For `icc(scope = "overall")`, the package reports ANOVA F statistics, degrees of freedom, and p-values in the overall coefficient table. The implementation and tests verify those reported p-values, but the package documentation does not define a generic ICC agreement hypothesis for users; this vignette therefore does not describe the overall ICC p-values as a generic test of agreement.

### CIA functions

`cia()` and `cia_rm()` are included here only as conceptual comparisons. CIA targets individual agreement or interchangeability rather than the same population concordance question targeted by CCC. In `cia()`, replicated readings within each subject-method cell are required because the estimator compares between-method disagreement with within-method replicate disagreement. The function supports pairwise and overall CIA, optional reference-method scaling, and optional confidence intervals by the selected inference method. No formal hypothesis test is implemented; inference is based on the estimate and its confidence interval.

`cia_rm()` is different from `cia()` because it targets matched repeated measurements under conditions such as visits, raters, laboratories, treatments, or time points, and it is not a technical-replicate estimator. It reports condition-specific CIA estimates, optional confidence intervals, and a homogeneity test for agreement across conditions. As implemented, the reported test statistic is `MS_method_time / MS_error`, with an upper-tail F-test p-value using `df_method_time` and `df_error`. The null hypothesis documented by the function is homogeneous agreement across conditions; the alternative is that agreement changes across conditions.

### Function summary

In this package, the functions differ not only in the target index but also in the supported data structure and inferential procedure. Some functions provide estimates and confidence intervals only. A formal hypothesis test should only be interpreted when the function explicitly implements and reports one.

Function | Index family | Target question | Data/design supported | Estimation approach | Inference reported | Formal hypothesis test implemented? Yes/No/Requires verification
--- | --- | --- | --- | --- | --- | ---
`ccc()` | Lin CCC | Pairwise concordance combining precision and accuracy | Numeric wide data; rows are paired observational units | Moment CCC with Lin delta-method/Fisher-z CI when requested | Estimate; optional confidence interval | No
`ccc_rm_reml()` | Repeated-measures CCC | Pairwise repeated-measures concordance from fitted variance components | Long repeated-measures data; subject plus optional method/time structure | REML mixed-model variance components and fixed-effect bias term | Estimate; optional confidence interval; variance-component diagnostics | No for the CCC parameter; variance-component selection tests are not CCC tests
`ccc_rm_ustat()` | Repeated-measures CCC | Pairwise repeated-measures concordance from U-statistic distances | Long repeated-measures data; balanced method/time coverage per pair | Nonparametric U-statistic estimator with Fisher-z CI when requested | Estimate; optional confidence interval | No
`icc()` | Classical ICC | Reliability or agreement under the selected ICC model, type, and unit | Numeric wide data; pairwise or overall all-column scope | Classical ANOVA mean-square formulas | Pairwise: estimate and optional CI. Overall: coefficient table with F statistic, df, p-value, and optional CI | Requires verification for overall p-values; no pairwise ICC test is documented
`icc_rm_reml()` | Repeated-measures ICC | Pairwise repeated-measures reliability/agreement from fitted variance components | Long repeated-measures data; subject plus optional method/time structure | REML mixed-model variance components and fixed-effect bias term | Estimate; optional confidence interval; variance-component diagnostics | No for the ICC parameter; variance-component selection tests are not ICC tests
`cia()` | CIA | Individual agreement/interchangeability relative to within-method replicate disagreement | Long replicated method-comparison data with replicate identifiers | Method-of-moments disagreement ratios; optional bounded variance-component variant | Estimate; optional confidence interval | No
`cia_rm()` | Repeated-measures CIA | Individual agreement/interchangeability across matched repeated conditions | Long matched repeated-measures data with one observation per subject-method-condition cell | Categorical-condition ANOVA estimator | Estimate; optional confidence interval; homogeneity F statistic and p-value | Yes: homogeneity of agreement across conditions

## Choosing among CCC, BA, ICC, and CIA

In practice these methods answer different questions.

- Use `ccc()` when one concordance coefficient per pair is the main target.
- Use `ccc_rm_ustat()` or `ccc_rm_reml()` when the concordance target is repeated-measures CCC rather than simple paired wide-data CCC.
- Use `ba()` when the size and direction of disagreement should be visible on the original measurement scale. Use `ba_rm()` for the repeated-measures Bland-Altman workflow.
- Use `icc()` when the target is reliability under a classical variance components interpretation.
- Use `icc_rm_reml()` when the target is repeated-measures ICC from the package’s REML variance-component backend.
- Use `cia()` when replicated readings within subject-method cells are available and the target is individual agreement or interchangeability relative to within-method replicate disagreement.
- Use `cia_rm()` when the target is individual agreement across matched repeated conditions rather than technical replicates.

There is overlap in interpretation, but these are not interchangeable estimators.

## References

Barnhart HX, Haber M, Lokhnygina Y, Kosinski AS. (2007). Comparison of concordance correlation coefficient and coefficient of individual agreement in assessing agreement. Journal of Biopharmaceutical Statistics, 17(4), 721-738.

Barnhart HX, Kosinski AS, Haber M. (2007). Assessing individual agreement. Journal of Biopharmaceutical Statistics, 17(4), 697-719.

Carrasco JL, Jover L. (2003). Estimating the concordance correlation coefficient: a new approach. Computational Statistics & Data Analysis, 47(4), 519-539.

Carrasco JL, Phillips BR, Puig-Martinez J, King TS, Chinchilli VM. (2013). Estimation of the concordance correlation coefficient for repeated measures using SAS and R. Computer Methods and Programs in Biomedicine, 109(3), 293-304.

Haber M, Gao J, Barnhart HX. (2010). Evaluation of agreement between measurement methods from data with matched repeated measurements via the coefficient of individual agreement. Journal of Data Science, 8, 457-469.

Lin L. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45, 255-268.

McGraw KO, Wong SP. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Pan Y, Gao J, Haber M, Barnhart HX. (2010). Estimation of coefficients of individual agreement (CIA’s) for quantitative and binary data using SAS and R. Computer Methods and Programs in Biomedicine.

Shrout PE, Fleiss JL. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428.