Model diagnostics

Code

library(tidyverse)
library(targets)
library(tidybayes)
library(glue)
library(gt)
library(gtExtras)

tar_config_set(
  store = here::here("_targets"),
  script = here::here("_targets.R")
)

tar_load(m_treatment_only)

invisible(list2env(tar_read(graphic_functions), .GlobalEnv))
invisible(list2env(tar_read(diagnostic_functions), .GlobalEnv))
invisible(list2env(tar_read(table_functions), .GlobalEnv))

theme_set(theme_ngo())

MCMC journeys

These multilevel multinomial models are huge and unwieldy—the model with just the treatment variables and respondent-specific intercepts has 3,098 coefficients/parameters (!!).

So here, for the sake of illustration, we show just four parameters: one of the intercepts and a treatment coefficient from each of the µs. These plots show that the chains are stable, well-mixed, and converged.

Traceplots

These should look like hairy caterpillars.

They do.

Code

params_to_show <- c(
  "b_mu1_Intercept", "b_mu1_feat_orgGreenpeace",
  "b_mu2_feat_issueHumanrights", "b_mu3_feat_govtUndergovernmentcrackdown"
)

plot_trace(m_treatment_only, params_to_show)

Trace rank plots (trank plots)

These are histograms of the ranks of the parameter draws across the four chains. If the chains are exploring the same space efficiently, the histograms should be similar and overlapping and no one chain should have a specific rank for a long time (McElreath 2020, 284).

They do.

Code

plot_trank(m_treatment_only, params_to_show)

Posterior predictions

The model should generate predictions that align with the observed outcomes. It does.

Code

plot_pp(m_treatment_only)

References

McElreath, Richard. 2020. Statistical Rethinking: A Bayesian Course with Examples in R and Stan. 2nd ed. Boca Raton, Florida: Chapman and Hall / CRC.

---
title: "Model diagnostics"
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  fig.align = "center", fig.retina = 3,
  fig.width = 6, fig.height = (6 * 0.618),
  out.width = "80%", collapse = TRUE,
  dev = "ragg_png"
)

options(
  digits = 3, width = 120,
  dplyr.summarise.inform = FALSE,
  knitr.kable.NA = ""
)
```

```{r libraries-data, warning=FALSE, message=FALSE}
library(tidyverse)
library(targets)
library(tidybayes)
library(glue)
library(gt)
library(gtExtras)

tar_config_set(
  store = here::here("_targets"),
  script = here::here("_targets.R")
)

tar_load(m_treatment_only)

invisible(list2env(tar_read(graphic_functions), .GlobalEnv))
invisible(list2env(tar_read(diagnostic_functions), .GlobalEnv))
invisible(list2env(tar_read(table_functions), .GlobalEnv))

theme_set(theme_ngo())
```


## MCMC journeys

These multilevel multinomial models are huge and unwieldy—the model with just the treatment variables and respondent-specific intercepts has `r scales::label_comma()(length(get_variables(m_treatment_only)))` coefficients/parameters (!!).

So here, for the sake of illustration, we show just four parameters: one of the intercepts and a treatment coefficient from each of the µs. These plots show that the chains are stable, well-mixed, and converged.

### Traceplots

These should look like hairy caterpillars. 

They do.

```{r mcmc-trace}
params_to_show <- c(
  "b_mu1_Intercept", "b_mu1_feat_orgGreenpeace",
  "b_mu2_feat_issueHumanrights", "b_mu3_feat_govtUndergovernmentcrackdown"
)

plot_trace(m_treatment_only, params_to_show)
```

### Trace rank plots (trank plots)

These are histograms of the ranks of the parameter draws across the four chains. If the chains are exploring the same space efficiently, the histograms should be similar and overlapping and no one chain should have a specific rank for a long time [@McElreath:2020, 284].

They do.

```{r mcmc-trank}
plot_trank(m_treatment_only, params_to_show)
```


## Posterior predictions

The model should generate predictions that align with the observed outcomes. It does.

```{r pp-check}
plot_pp(m_treatment_only)
```