# Francis et al. (2019) Extraction Report

## Sources

- Paper bundle and extracted text in `out/bundle.json`, `out/fulltext.md`, and `out/text.txt`.
- Public University of Reading dataset: <https://researchdata.reading.ac.uk/205/>; DOI `10.17864/1947.205`.
- Source ZIPs, unpacked CSVs, and README copied into `out/external/`.
- Computations are documented in `analysis/effect_sizes.qmd`; the rendered audit is `analysis/effect_sizes_raw_data.csv`.
- Original Experiment 1 analysis reproduction is documented in `analysis/original_analysis_check.qmd`; the rendered audit is `analysis/original_analysis_check.csv`.

## Decisions

- The YAML now treats `analysis/effect_sizes.qmd` as the source of truth for computed `d` and `v`.
- Scalar studies are extracted as scenario by polarity low-vs-high stakes contrasts, not as six polarity-collapsed omnibus effects.
- Evidence-seeking studies are included as eligible evidence-seeking contrasts. Negative-polarity prompts are represented as separate effects; they are not coded as skeptical pressure because the vignette does not introduce an explicit doubt or counterconsideration.
- Non-evidence-seeking effects use `d = mean(low) - mean(high)`.
- Study 1 negative-polarity evidence-fixed effects reverse-code raw agreement with "doesn't know" (`8 - raw`) before computing `d`, so the effect is in the same knowledge-attribution direction as the positive-polarity effects.
- Evidence-seeking effects use raw low-minus-high `d` in the YAML; downstream meta-analysis code reverses evidence-seeking effects programmatically.
- Study 2 registered-replication moderators are coded to match the current Sripada & Stanley (2012) extraction.
- Possessions/Arson evidence reliability is temporarily coded `Medium` for harmonization because the relevant evidence is a week-old sprinkler-system check projected forward.

## Raw-Data Computation

- Experiment 1 uses complete paired lowest and highest stakes responses from the post-removal CSVs.
- Experiment 1 negative-polarity responses are reverse-coded from agreement with denial to knowledge-attribution direction.
- Evidence-seeking experiments exclude blank, `never`, and non-positive responses from continuous contrasts, then apply the documented log-MAD outlier rule across the scenario's four stakes cells.
- The registered replication uses between-group low/high contrasts from the post-removal CSV. The Q2 knowledge scale is reverse-coded from `1 = strongly agree, 7 = strongly disagree` to agreement before computing `d`.

## Remaining Cautions

- The evidence-seeking outlier handling follows the article/data-documentation description as closely as possible from the released files; keep the `evidence_seeking_cleaning_rule` flag for sensitivity checks.
- The copied raw CSVs include participant-level Qualtrics/MTurk fields such as IP/location/worker identifiers because those fields are present in the public data release.

## Checks

- Original-analysis diagnostic: using raw agreement scores and complete Study 1 response rows (`N = 97`: 55 positive-polarity, 42 negative-polarity), `analysis/original_analysis_check.qmd` reproduces the Appendix I Greenhouse-Geisser corrected mixed ANOVA results to reported rounding.
- When `geepack` is available, the same diagnostic also reproduces the Appendix I linear GEE stakes and interaction tests to reported rounding. The only notable discrepancy is the vaccine GEE polarity Wald statistic (`207.30` from the released data versus `206.30` reported in the paper); the corresponding ANOVA polarity statistic and GEE stakes/interaction tests match.
- `analysis/effect_sizes.qmd` renders successfully and writes `analysis/effect_sizes_raw_data.csv`, including raw `d` and YAML-ready `d_for_yaml` (now identical to raw `d`).
- `francis2019stakesscalesskepticism.yaml` validates against `docs/stakes_meta_schema.json`.
- YAML `d` and `v` values match `analysis/effect_sizes_raw_data.csv`: effects use raw low-minus-high `d`; evidence-seeking harmonization is downstream.
- All 6 Study 1 negative-polarity evidence-fixed effects carry `negative_polarity_reverse_coded`.
- All 36 Francis evidence-seeking effects carry `evidence_seeking_cleaning_rule`.
- Effects exporter smoke test returns 54 Francis rows.
- Extraction HTML rendered to `francis2019stakesscalesskepticism.html`.
- `invoke`/`meta-extract` were not available on PATH in this environment; HTML was rendered by calling `meta_extract.extraction_report.render_extraction_report_html` with `PYTHONPATH=src`.