# REPORT: francisbeaman2023roleconfidenceknowledge

## Pipeline outputs used (in priority order)

- `papers/francisbeaman2023roleconfidenceknowledge/out/tables/camelot_stream_p8_t1.csv` (Table 1: low/high means + SDs by polarity)
- `papers/francisbeaman2023roleconfidenceknowledge/out/fulltext.md` (method description + t(df) for stakes effects by polarity)
- `papers/francisbeaman2023roleconfidenceknowledge/out/external/osf_n8x7q/` (public OSF files downloaded 2026-04-12; see OSF audit below)
- Rendered page images (manual check of page numbers / table text):
  - `papers/francisbeaman2023roleconfidenceknowledge/out/images/pages/p008.png` (Table 1)
  - `papers/francisbeaman2023roleconfidenceknowledge/out/images/figures/p010.png` (positive polarity path c t-stat)
  - `papers/francisbeaman2023roleconfidenceknowledge/out/images/pages/p011.png` (negative polarity path c t-stat)
  - `papers/francisbeaman2023roleconfidenceknowledge/out/images/pages/p013.png` (Figure 6 diagram; c=0.88)

## OSF dataset search (2026-04-12)

The article's data availability statement points to OSF project `https://osf.io/n8x7q/`.
The public project has two components:

- `Data` (`https://osf.io/zefp4/`)
- `Study Materials` (`https://osf.io/ua8wr/`)

Files downloaded locally:

- `out/external/osf_n8x7q/Positive Polarity - Data for Mediation Analysis.csv`
- `out/external/osf_n8x7q/Negative Polarity - Data for Mediation Analysis.csv`
- `out/external/osf_n8x7q/Positive Polarity - Data for Subject Confidence Analysis.csv`
- `out/external/osf_n8x7q/Negative Polarity - Data for Subject Confidence Analysis.csv`
- `out/external/osf_n8x7q/The Role of Confidence in Knowledge Ascriptions - Experimental Materials.pdf`

Assessment:

- The mediation CSVs are participant-level but already collapsed across the six scenarios. They contain only `ES_1`, `ES_2`, `PC_1`, `PC_2`, `SC_1`, and `SC_2`.
- The subject-confidence CSVs do have per-vignette low/high columns (`Para`, `Vacc`, `Game`, `Mount`, `Intro`, `Pval`), but only for protagonist/subject confidence (`*_SC`), not for evidence-seeking (`ES`) or participant confidence (`PC`).
- The materials PDF includes the scenario texts and confirms the six scenario families, but it does not supply raw per-vignette responses.

Conclusion: the OSF files are useful for verifying the aggregate evidence-seeking effects and for a by-vignette check of subject-confidence ratings only. They are not sufficient for the desired by-vignette evidence-seeking/meta-analytic effect extraction analogous to Francis et al. (2019).

### Broader public-data search

Additional checks on 2026-04-12 did not locate a fuller public raw/per-vignette dataset:

- OSF API recursion found only the four CSVs listed above in the `Data` component and the materials PDF in the `Study Materials` component. The mediation CSVs each have a single OSF file version (`revision=1`), so there is no older OSF version exposing more columns.
- OSF project/component history did not reveal a separate raw, Qualtrics, SPSS, R, or per-vignette evidence-seeking file.
- Springer/Crossref/OpenAlex/DataCite/OpenAIRE ScholeXplorer metadata did not expose an additional dataset relation or supplementary data file beyond the article itself and the OSF data page.
- Institutional repository mirrors were checked. CentAUR (University of Reading) provides article PDFs only; White Rose lists the related dataset as the OSF page; Keele Worktribe has the article PDF and an `Open Data Available` tag, but no separate dataset file/link in the public record.
- Internet Archive CDX showed only an archived snapshot of the OSF root project (`n8x7q`) with the same two components (`Data`, `Study Materials`), and no archived component/download pages exposing additional raw data files.

## Computations (reproducible)

- Effect sizes were computed in `papers/francisbeaman2023roleconfidenceknowledge/analysis/effect_sizes.qmd` using `within_smcrp_t` (paired t used to recover within-person r), then copied into `papers/francisbeaman2023roleconfidenceknowledge/francisbeaman2023roleconfidenceknowledge.yaml`.

## Key extraction decisions

1) **Study granularity**
   - The paper reports results separately for positive vs negative polarity groups (random assignment).
   - I represented these as two separate `studies[]` entries (study_id 1 = positive, study_id 2 = negative) to keep the effect sizes independent across studies.

2) **Scenario/moderator coding**
   - Table 1 reports low vs high stakes averages **collapsed across six scenarios**.
   - Therefore `moderators.scenario` and `moderators.evidence` are coded as `null` (cannot code uniquely for an aggregate across multiple scenario types/evidence sources).

## Issues / uncertainties

### NEEDS INPUT 1 — Negative polarity direction check (Table 1 vs narrative text)
The narrative text states: *“Across both positive and negative polarity conditions, participants had higher average evidence-seeking scores for high stakes cases compared to low stakes cases (see Table 1).”*  
However, Table 1 shows (negative polarity; knowledge/evidence-seeking) **Low = 3.31 (3.95)** and **High = 2.43 (3.02)**, i.e., High < Low.

Please confirm one of the following:
1) **Table 1 is correct as printed** (High < Low for negative polarity), OR
2) The negative-polarity low/high values are swapped/misprinted (provide the correct low/high means + SDs).

If (2), I should update `papers/francisbeaman2023roleconfidenceknowledge/francisbeaman2023roleconfidenceknowledge.yaml` and re-run `analysis/effect_sizes.qmd`.

### RESOLVED/PARTIAL — OSF data found, but no per-scenario evidence-seeking outcomes
The public OSF project was found and downloaded locally. It contains aggregate participant-level mediation data and per-scenario subject-confidence data, but not per-scenario evidence-seeking responses. Therefore the current aggregate-across-scenarios effect remains the defensible extraction unless the authors can provide raw/per-vignette evidence-seeking data.