---
title: "Estimating trial-level effects"
output: rmarkdown::html_vignette
bibliography: citations.bib
csl: apa.csl
link-citations: true
vignette: >
  %\VignetteIndexEntry{Estimating trial-level effects}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

<!-- categorical.Rmd is generated from _categorical.Rmd.src Please edit that file -->

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  dpi = 150, fig.width = 8, fig.height = 6,
  fig.align = "center", out.width = "75%"
)
```


## Introduction
By default, the `hmetad` package uses aggregated data (i.e., counts of the  number of trials with the same stimulus, type 1 response, and type 2 response.) This is because data aggregation makes model fitting and simulation much more efficient. But sometimes researchers will be interested in trial-level effects.

One common example would be what is often called "crossed random effects". For example, in a design where all participants make responses to the same set of items, a researcher might want to estimate both participant-level and item-level effects on their model parameters.

We can simulate data from such a design like so:
```{r, message=FALSE, warning=FALSE}
library(tidyverse)
library(tidybayes)
library(hmetad)

## average model parameters
K <- 3 ## number of confidence levels
mu_log_M <- -0.5
mu_dprime <- 1.5
mu_c <- 0
mu_c2_0 <- rep(-1, K - 1)
mu_c2_1 <- rep(-1, K - 1)

## participant-level standard deviations
sd_log_M_participant <- 0.25
sd_dprime_participant <- 0.5
sd_c_participant <- 0.33
sd_c2_0_participant <- cov_matrix(rep(0.25, K - 1), diag(K - 1))
sd_c2_1_participant <- cov_matrix(rep(0.25, K - 1), diag(K - 1))

## item-level standard deviations
sd_log_M_item <- 0.1
sd_dprime_item <- 0.5
sd_c_item <- 0.75
sd_c2_0_item <- cov_matrix(rep(0.1, K - 1), diag(K - 1))
sd_c2_1_item <- cov_matrix(rep(0.1, K - 1), diag(K - 1))


## simulate data
d <- expand_grid(
  participant = 1:50,
  item = 1:25
) |>
  ## simulate participant-level differences
  group_by(participant) |>
  mutate(
    z_log_M_participant = rnorm(1, sd = sd_log_M_participant),
    z_dprime_participant = rnorm(1, sd = sd_dprime_participant),
    z_c_participant = rnorm(1, sd = sd_c_participant),
    z_c2_0_participant = list(rmulti_normal(1, mu = rep(0, K - 1), Sigma = sd_c2_0_participant)),
    z_c2_1_participant = list(rmulti_normal(1, mu = rep(0, K - 1), Sigma = sd_c2_1_participant))
  ) |>
  ## simulate item-level differences
  group_by(item) |>
  mutate(
    z_log_M_item = rnorm(1, sd = sd_log_M_item),
    z_dprime_item = rnorm(1, sd = sd_dprime_item),
    z_c_item = rnorm(1, sd = sd_c_item),
    z_c2_0_item = list(rmulti_normal(1, mu = rep(0, K - 1), Sigma = sd_c2_0_item)),
    z_c2_1_item = list(rmulti_normal(1, mu = rep(0, K - 1), Sigma = sd_c2_1_item))
  ) |>
  ungroup() |>
  ## compute model parameters
  mutate(
    log_M = mu_log_M + z_log_M_participant + z_log_M_item,
    dprime = mu_dprime + z_dprime_participant + z_dprime_item,
    c = mu_c + z_c_participant + z_c_item,
    c2_0_diff = map2(
      z_c2_0_participant, z_c2_0_item,
      ~ exp(mu_c2_0 + .x + .y)
    ),
    c2_1_diff = map2(
      z_c2_1_participant, z_c2_1_item,
      ~ exp(mu_c2_1 + .x + .y)
    )
  ) |>
  ## simulate two trials per participant/item (stimulus = 0 and stimulus = 1)
  mutate(trial = pmap(list(dprime, c, log_M, c2_0_diff, c2_1_diff), sim_metad, N_trials = 2)) |>
  select(participant, item, trial) |>
  unnest(trial)
```

```{r echo=FALSE}
d
```

Don't worry about the details of the simulation code- what matters is that we have a data set with repeated measures for participants:

```{r count_participants}
count(d, participant)
```
And repeated measures for items:
```{r count_items}
count(d, item)
```

## Standard model with data aggregation
If we would like, we can use the `fit_metad` function on this data with participant-level and item-level effects. However, if we aggregate the data ourselves, we can see that the aggregation doesn't really help us here:

```{r aggregate}
aggregate_metad(d, participant, item)
```

As you can see, the aggregated data set has `r nrow(aggregate_metad(d, participant, item))` rows (with two observations per row), which is not much smaller than the trial-level data that we started with! So, in this case, it will probably be easier *not* to aggregate our data. Nevertheless, there is nothing stopping us from fitting the model like normal:^[Note that in practice, fitting hierarchical models will usually require setting informed priors and adjusting the Stan sampler settings.]
```{r multinomial_model}
# Priors are chosen arbitrarily for this example.
# Please choose your own wisely!
priors <- prior(normal(0, .25), class = Intercept) +
  set_prior(
    "normal(0, .25)",
    class = "Intercept",
    dpar = c("dprime", "c")
  ) +
  set_prior("normal(-0.5, .1)", class = "Intercept", dpar = metac2_parameters(K = 3)) +
  prior(normal(0, 1), class = sd) +
  set_prior("normal(0, 1)", class = "sd", dpar = c("dprime", "c", metac2_parameters(K = 3)))

m.multinomial <- fit_metad(
  bf(
    N ~ 1 + (1 | participant) + (1 | item),
    dprime + c +
      metac2zero1diff + metac2zero2diff +
      metac2one1diff + metac2one2diff ~
      1 + (1 | participant) + (1 | item)
  ),
  data = d, init = 0, cores = 4, prior = priors
)
```

```{r echo=FALSE, warning=FALSE, message=FALSE}
m.multinomial
```


## Data preparation
Fitting the trial-level model does not require data aggregation, however it still requires a small amount of data preparation. To fit the model, we will need two things:

  * a column with the stimulus per trial (`0` or `1`), and
  * a column containing the joint type 1/type 2 responses per trial (between `1` and `2*K`).

Our data already has a `stimulus` column but separate columns for the two responses. So, we can add in a joint response column now:
```{r joint_response}
d <- d |>
  mutate(joint_response = joint_response(response, confidence, K)) |>
  relocate(joint_response, .after = "stimulus")
```

```{r echo=FALSE}
d
```


## Model fitting
Now that we have our data, we can fit the trial-level model using `joint_response` as our response variable, `stimulus` as an extra variable passed to `brms`, and the argument `categorical=TRUE` to tell `fit_metad` not to aggregate the data:
```{r model_fitting}
m.categorical <- fit_metad(
  bf(
    joint_response | vint(stimulus) ~ 1 + (1 | participant) + (1 | item),
    dprime + c +
      metac2zero1diff + metac2zero2diff +
      metac2one1diff + metac2one2diff ~
      1 + (1 | participant) + (1 | item)
  ),
  data = d, categorical = TRUE, init = 0, cores = 4, prior = priors
)
```

```{r echo=FALSE, warning=FALSE, message=FALSE}
m.categorical
```

As you can see, aside from the way that the data is formatted, this model is exactly the same as the multinomial model above.





## Extracting model estimates
Obtaining posterior estimates over model parameters, predictions, and other estimates is very similar to the multinomial model (for details, see `vignette("hmetad")`). So, here we will focus on the type 1 ROC curves, this time using `roc1_rvars` instead of `roc1_draws` for increased efficiency.

To get the posterior estimates, we need to specify a dataset to make predictions for, as well as a random effects formula to use in model predictions. For example, to estimate a ROC averaging over participants and items, we can use an empty data set with `re_formula=NA`:
```{r roc_group}
roc1_rvars(m.multinomial, tibble(.row = 1), re_formula = NA)
```

The process is exactly the same for the categorical model:
```{r roc_group2}
roc1_rvars(m.categorical, tibble(.row = 1), re_formula = NA)
```

Next, to get participant-level ROCs (averaging over items), we can use a dataset with one row per participant and only the participant-level random effects:
```{r roc_participant}
roc1_rvars(m.categorical, distinct(d, participant), re_formula = ~ (1 | participant))
```


We can use a similar process to get item-level ROCs (averaging over participants):
```{r roc_item}
roc1_rvars(m.categorical, distinct(d, item), re_formula = ~ (1 | item))
```


## Other benefits
Aside from representing the data in a more convenient format, the trial-level model should be more useful for things like model comparison using the `loo` package, multivariate models, and mediation models. These features should mostly work out of the box but they are still under active development, so stay tuned!



