roseetal2019nothingstakeknowledge
Back to paper
Download · Edit · History · Raw
/data/papers/roseetal2019nothingstakeknowledge/REPORT.md
Rendered Markdown.

Extraction report: roseetal2019nothingstakeknowledge

Summary

  • Pipeline outputs used (preferred sources):
  • Numbers: papers/roseetal2019nothingstakeknowledge/out/tables/ (tabula_stream_p10_t2.csv, tabula_stream_p14_t5.csv, tabula_stream_p18_t7.csv).
  • Question wording + study framing: papers/roseetal2019nothingstakeknowledge/out/tei.xml (Table 1 caption) and papers/roseetal2019nothingstakeknowledge/out/text.txt.
  • Output YAML: papers/roseetal2019nothingstakeknowledge/roseetal2019nothingstakeknowledge.yaml
  • Effect sizes: computed signed SMD d and sampling variance v for all 57 site×outcome contrasts using Campbell Collaboration’s phi-coefficient conversion from Cramer's V + N, with sign taken from the paper’s difference plots (Figs. 2/5/8). See papers/roseetal2019nothingstakeknowledge/analysis/effect_sizes.qmd.
  • Validation: papers/roseetal2019nothingstakeknowledge/roseetal2019nothingstakeknowledge.yaml validates against docs/stakes_meta_schema.json (via jsonschema).

Recoding decision — study/site/effect structure

  • Updated on 2026-04-20: the YAML now treats Rose et al. as one paper-reported cross-cultural bank-case study, not as 19 separate studies.
  • The 19 former site-level study entries were moved into effect-level site_id, site_label, language, language_other, and sample fields.
  • All 57 site×outcome effect sizes and effect IDs were retained. Effect subgroup labels are prefixed with the site label to keep exported rows readable.
  • This change is semantic/structural only: sites no longer inflate the study count, while site-specific sample metadata remains available to the exporter through effect-level overrides.

Manual interventions / PDF use (required disclosure)

  • No direct PDF/page-image consultation was needed for numeric extraction (tables were available in out/tables/).
  • PDF figures were consulted to recover effect direction by site, because χ² and Cramer's V in the tables are unsigned. Signs were read from the “Low − High” difference plots (Figs. 2/5/8), matching the extraction sign convention d = mean(low) - mean(high).
  • Note: the low-stakes vignette text is truncated in out/text.txt/out/tei.xml (it cuts off mid-sentence). If you need the full vignette wording for documentation, consult the PDF and record the page number.

Effect-size computation (d, v)

The paper reports site-level N and Cramer's V for 2×2 (stakes × binary outcome) χ² tests. Because the paper does not report the per-condition cell counts/proportions, effect sizes were computed using the Campbell Collaboration Effect Size Calculator equations for the phi coefficient (Correlation / phi coefficient, §1.29).

Approach: - Treat Cramer's V as |phi| (2×2 case), set r = |V|. - Convert r to Cohen’s d using d = 2r / sqrt(1 - r^2). - Compute sampling variance using Campbell’s v_d = d^2 / chi^2 with chi^2 = r^2 * N (equivalently v_d = 4 / (N * (1 - r^2))). - Apply the sign from the paper’s difference plots (Low − High), so the final signed effect matches the extraction convention d = mean(low) - mean(high).

Full derivations and reproducible computations are in papers/roseetal2019nothingstakeknowledge/analysis/effect_sizes.qmd.

Other uncertainties / potential reporting issues

N total discrepancies (paper text vs site tables)

The paper’s running text reports overall Ns (e.g., 3530; 3522; 2923), but the site-level Ns in extracted tables sum to different totals: - Table 2 site Ns sum to 3516 (tabula_stream_p10_t2.csv) - Table 4 site Ns sum to 3508 (tabula_stream_p14_t5.csv) - Table 6 site Ns sum to 2911 (tabula_stream_p18_t7.csv)

This does not block the computations above (which use the per-site Ns as reported in each table), but it may be worth double-checking whether the paper text contains typos or whether there were additional exclusions/missing responses not reflected in the running totals.

Compensation details

Table 1 lists Payment as “Volunteers”, “Compensation”, or “Compensation & volunteers”, but does not specify the form of compensation (money vs credits vs gift). If needed for coding, locate compensation details in the PDF (or supplement) and record the exact wording + page.