roseetal2019nothingstakeknowledge
/data/papers/roseetal2019nothingstakeknowledge/REPORT.md# Extraction report: roseetal2019nothingstakeknowledge
## Summary
- Pipeline outputs used (preferred sources):
- Numbers: `papers/roseetal2019nothingstakeknowledge/out/tables/` (`tabula_stream_p10_t2.csv`, `tabula_stream_p14_t5.csv`, `tabula_stream_p18_t7.csv`).
- Question wording + study framing: `papers/roseetal2019nothingstakeknowledge/out/tei.xml` (Table 1 caption) and `papers/roseetal2019nothingstakeknowledge/out/text.txt`.
- Output YAML: `papers/roseetal2019nothingstakeknowledge/roseetal2019nothingstakeknowledge.yaml`
- Effect sizes: computed signed SMD `d` and sampling variance `v` for all 57 site×outcome contrasts using Campbell Collaboration’s phi-coefficient conversion from Cramer's `V` + `N`, with sign taken from the paper’s difference plots (Figs. 2/5/8). See `papers/roseetal2019nothingstakeknowledge/analysis/effect_sizes.qmd`.
- Validation: `papers/roseetal2019nothingstakeknowledge/roseetal2019nothingstakeknowledge.yaml` validates against `docs/stakes_meta_schema.json` (via `jsonschema`).
## Recoding decision — study/site/effect structure
- Updated on 2026-04-20: the YAML now treats Rose et al. as one paper-reported cross-cultural bank-case study, not as 19 separate studies.
- The 19 former site-level study entries were moved into effect-level `site_id`, `site_label`, `language`, `language_other`, and `sample` fields.
- All 57 site×outcome effect sizes and effect IDs were retained. Effect subgroup labels are prefixed with the site label to keep exported rows readable.
- This change is semantic/structural only: sites no longer inflate the study count, while site-specific sample metadata remains available to the exporter through effect-level overrides.
## Manual interventions / PDF use (required disclosure)
- No direct PDF/page-image consultation was needed for numeric extraction (tables were available in `out/tables/`).
- PDF figures were consulted to recover effect *direction* by site, because χ² and Cramer's `V` in the tables are unsigned. Signs were read from the “Low − High” difference plots (Figs. 2/5/8), matching the extraction sign convention `d = mean(low) - mean(high)`.
- Note: the low-stakes vignette text is truncated in `out/text.txt`/`out/tei.xml` (it cuts off mid-sentence). If you need the full vignette wording for documentation, consult the PDF and record the page number.
## Effect-size computation (d, v)
The paper reports site-level `N` and Cramer's `V` for 2×2 (stakes × binary outcome) χ² tests. Because the paper does not report the per-condition cell counts/proportions, effect sizes were computed using the Campbell Collaboration Effect Size Calculator equations for the *phi coefficient* (Correlation / phi coefficient, §1.29).
Approach:
- Treat Cramer's `V` as `|phi|` (2×2 case), set `r = |V|`.
- Convert `r` to Cohen’s `d` using `d = 2r / sqrt(1 - r^2)`.
- Compute sampling variance using Campbell’s `v_d = d^2 / chi^2` with `chi^2 = r^2 * N` (equivalently `v_d = 4 / (N * (1 - r^2))`).
- Apply the sign from the paper’s difference plots (Low − High), so the final signed effect matches the extraction convention `d = mean(low) - mean(high)`.
Full derivations and reproducible computations are in `papers/roseetal2019nothingstakeknowledge/analysis/effect_sizes.qmd`.
## Other uncertainties / potential reporting issues
### N total discrepancies (paper text vs site tables)
The paper’s running text reports overall Ns (e.g., 3530; 3522; 2923), but the site-level Ns in extracted tables sum to different totals:
- Table 2 site Ns sum to 3516 (`tabula_stream_p10_t2.csv`)
- Table 4 site Ns sum to 3508 (`tabula_stream_p14_t5.csv`)
- Table 6 site Ns sum to 2911 (`tabula_stream_p18_t7.csv`)
This does not block the computations above (which use the per-site Ns as reported in each table), but it may be worth double-checking whether the paper text contains typos or whether there were additional exclusions/missing responses not reflected in the running totals.
### Compensation details
Table 1 lists Payment as “Volunteers”, “Compensation”, or “Compensation & volunteers”, but does not specify the form of compensation (money vs credits vs gift).
If needed for coding, locate compensation details in the PDF (or supplement) and record the exact wording + page.