francisbeaman2023roleconfidenceknowledge
Back to paper
Download · Edit · History · Raw
/data/papers/francisbeaman2023roleconfidenceknowledge/REPORT.md
Rendered Markdown.

REPORT: francisbeaman2023roleconfidenceknowledge

Pipeline outputs used (in priority order)

  • papers/francisbeaman2023roleconfidenceknowledge/out/tables/camelot_stream_p8_t1.csv (Table 1: low/high means + SDs by polarity)
  • papers/francisbeaman2023roleconfidenceknowledge/out/fulltext.md (method description + t(df) for stakes effects by polarity)
  • papers/francisbeaman2023roleconfidenceknowledge/out/external/osf_n8x7q/ (public OSF files downloaded 2026-04-12; see OSF audit below)
  • Rendered page images (manual check of page numbers / table text):
  • papers/francisbeaman2023roleconfidenceknowledge/out/images/pages/p008.png (Table 1)
  • papers/francisbeaman2023roleconfidenceknowledge/out/images/figures/p010.png (positive polarity path c t-stat)
  • papers/francisbeaman2023roleconfidenceknowledge/out/images/pages/p011.png (negative polarity path c t-stat)
  • papers/francisbeaman2023roleconfidenceknowledge/out/images/pages/p013.png (Figure 6 diagram; c=0.88)

OSF dataset search (2026-04-12)

The article's data availability statement points to OSF project https://osf.io/n8x7q/. The public project has two components:

  • Data (https://osf.io/zefp4/)
  • Study Materials (https://osf.io/ua8wr/)

Files downloaded locally:

  • out/external/osf_n8x7q/Positive Polarity - Data for Mediation Analysis.csv
  • out/external/osf_n8x7q/Negative Polarity - Data for Mediation Analysis.csv
  • out/external/osf_n8x7q/Positive Polarity - Data for Subject Confidence Analysis.csv
  • out/external/osf_n8x7q/Negative Polarity - Data for Subject Confidence Analysis.csv
  • out/external/osf_n8x7q/The Role of Confidence in Knowledge Ascriptions - Experimental Materials.pdf

Assessment:

  • The mediation CSVs are participant-level but already collapsed across the six scenarios. They contain only ES_1, ES_2, PC_1, PC_2, SC_1, and SC_2.
  • The subject-confidence CSVs do have per-vignette low/high columns (Para, Vacc, Game, Mount, Intro, Pval), but only for protagonist/subject confidence (*_SC), not for evidence-seeking (ES) or participant confidence (PC).
  • The materials PDF includes the scenario texts and confirms the six scenario families, but it does not supply raw per-vignette responses.

Conclusion: the OSF files are useful for verifying the aggregate evidence-seeking effects and for a by-vignette check of subject-confidence ratings only. They are not sufficient for the desired by-vignette evidence-seeking/meta-analytic effect extraction analogous to Francis et al. (2019).

Broader public-data search

Additional checks on 2026-04-12 did not locate a fuller public raw/per-vignette dataset:

  • OSF API recursion found only the four CSVs listed above in the Data component and the materials PDF in the Study Materials component. The mediation CSVs each have a single OSF file version (revision=1), so there is no older OSF version exposing more columns.
  • OSF project/component history did not reveal a separate raw, Qualtrics, SPSS, R, or per-vignette evidence-seeking file.
  • Springer/Crossref/OpenAlex/DataCite/OpenAIRE ScholeXplorer metadata did not expose an additional dataset relation or supplementary data file beyond the article itself and the OSF data page.
  • Institutional repository mirrors were checked. CentAUR (University of Reading) provides article PDFs only; White Rose lists the related dataset as the OSF page; Keele Worktribe has the article PDF and an Open Data Available tag, but no separate dataset file/link in the public record.
  • Internet Archive CDX showed only an archived snapshot of the OSF root project (n8x7q) with the same two components (Data, Study Materials), and no archived component/download pages exposing additional raw data files.

Computations (reproducible)

  • Effect sizes were computed in papers/francisbeaman2023roleconfidenceknowledge/analysis/effect_sizes.qmd using within_smcrp_t (paired t used to recover within-person r), then copied into papers/francisbeaman2023roleconfidenceknowledge/francisbeaman2023roleconfidenceknowledge.yaml.

Key extraction decisions

1) Study granularity - The paper reports results separately for positive vs negative polarity groups (random assignment). - I represented these as two separate studies[] entries (study_id 1 = positive, study_id 2 = negative) to keep the effect sizes independent across studies.

2) Scenario/moderator coding - Table 1 reports low vs high stakes averages collapsed across six scenarios. - Therefore moderators.scenario and moderators.evidence are coded as null (cannot code uniquely for an aggregate across multiple scenario types/evidence sources).

Issues / uncertainties

NEEDS INPUT 1 — Negative polarity direction check (Table 1 vs narrative text)

The narrative text states: “Across both positive and negative polarity conditions, participants had higher average evidence-seeking scores for high stakes cases compared to low stakes cases (see Table 1).”
However, Table 1 shows (negative polarity; knowledge/evidence-seeking) Low = 3.31 (3.95) and High = 2.43 (3.02), i.e., High < Low.

Please confirm one of the following: 1) Table 1 is correct as printed (High < Low for negative polarity), OR 2) The negative-polarity low/high values are swapped/misprinted (provide the correct low/high means + SDs).

If (2), I should update papers/francisbeaman2023roleconfidenceknowledge/francisbeaman2023roleconfidenceknowledge.yaml and re-run analysis/effect_sizes.qmd.

RESOLVED/PARTIAL — OSF data found, but no per-scenario evidence-seeking outcomes

The public OSF project was found and downloaded locally. It contains aggregate participant-level mediation data and per-scenario subject-confidence data, but not per-scenario evidence-seeking responses. Therefore the current aggregate-across-scenarios effect remains the defensible extraction unless the authors can provide raw/per-vignette evidence-seeking data.