francis2019stakesscalesskepticism
Back to paper
Download · Edit · History · Raw
/data/papers/francis2019stakesscalesskepticism/REPORT.md
Rendered Markdown.

Francis et al. (2019) Extraction Report

Sources

  • Paper bundle and extracted text in out/bundle.json, out/fulltext.md, and out/text.txt.
  • Public University of Reading dataset: https://researchdata.reading.ac.uk/205/; DOI 10.17864/1947.205.
  • Source ZIPs, unpacked CSVs, and README copied into out/external/.
  • Computations are documented in analysis/effect_sizes.qmd; the rendered audit is analysis/effect_sizes_raw_data.csv.
  • Original Experiment 1 analysis reproduction is documented in analysis/original_analysis_check.qmd; the rendered audit is analysis/original_analysis_check.csv.

Decisions

  • The YAML now treats analysis/effect_sizes.qmd as the source of truth for computed d and v.
  • Scalar studies are extracted as scenario by polarity low-vs-high stakes contrasts, not as six polarity-collapsed omnibus effects.
  • Evidence-seeking studies are included as eligible evidence-seeking contrasts. Negative-polarity prompts are represented as separate effects; they are not coded as skeptical pressure because the vignette does not introduce an explicit doubt or counterconsideration.
  • Non-evidence-seeking effects use d = mean(low) - mean(high).
  • Study 1 negative-polarity evidence-fixed effects reverse-code raw agreement with "doesn't know" (8 - raw) before computing d, so the effect is in the same knowledge-attribution direction as the positive-polarity effects.
  • Evidence-seeking effects use raw low-minus-high d in the YAML; downstream meta-analysis code reverses evidence-seeking effects programmatically.
  • Study 2 registered-replication moderators are coded to match the current Sripada & Stanley (2012) extraction.
  • Possessions/Arson evidence reliability is temporarily coded Medium for harmonization because the relevant evidence is a week-old sprinkler-system check projected forward.

Raw-Data Computation

  • Experiment 1 uses complete paired lowest and highest stakes responses from the post-removal CSVs.
  • Experiment 1 negative-polarity responses are reverse-coded from agreement with denial to knowledge-attribution direction.
  • Evidence-seeking experiments exclude blank, never, and non-positive responses from continuous contrasts, then apply the documented log-MAD outlier rule across the scenario's four stakes cells.
  • The registered replication uses between-group low/high contrasts from the post-removal CSV. The Q2 knowledge scale is reverse-coded from 1 = strongly agree, 7 = strongly disagree to agreement before computing d.

Remaining Cautions

  • The evidence-seeking outlier handling follows the article/data-documentation description as closely as possible from the released files; keep the evidence_seeking_cleaning_rule flag for sensitivity checks.
  • The copied raw CSVs include participant-level Qualtrics/MTurk fields such as IP/location/worker identifiers because those fields are present in the public data release.

Checks

  • Original-analysis diagnostic: using raw agreement scores and complete Study 1 response rows (N = 97: 55 positive-polarity, 42 negative-polarity), analysis/original_analysis_check.qmd reproduces the Appendix I Greenhouse-Geisser corrected mixed ANOVA results to reported rounding.
  • When geepack is available, the same diagnostic also reproduces the Appendix I linear GEE stakes and interaction tests to reported rounding. The only notable discrepancy is the vaccine GEE polarity Wald statistic (207.30 from the released data versus 206.30 reported in the paper); the corresponding ANOVA polarity statistic and GEE stakes/interaction tests match.
  • analysis/effect_sizes.qmd renders successfully and writes analysis/effect_sizes_raw_data.csv, including raw d and YAML-ready d_for_yaml (now identical to raw d).
  • francis2019stakesscalesskepticism.yaml validates against docs/stakes_meta_schema.json.
  • YAML d and v values match analysis/effect_sizes_raw_data.csv: effects use raw low-minus-high d; evidence-seeking harmonization is downstream.
  • All 6 Study 1 negative-polarity evidence-fixed effects carry negative_polarity_reverse_coded.
  • All 36 Francis evidence-seeking effects carry evidence_seeking_cleaning_rule.
  • Effects exporter smoke test returns 54 Francis rows.
  • Extraction HTML rendered to francis2019stakesscalesskepticism.html.
  • invoke/meta-extract were not available on PATH in this environment; HTML was rendered by calling meta_extract.extraction_report.render_extraction_report_html with PYTHONPATH=src.