Please log in to access this page.
francisbeaman2023roleconfidenceknowledge
/data/papers/francisbeaman2023roleconfidenceknowledge/francisbeaman2023roleconfidenceknowledge.yamlschema_version: '1.2'
paper:
paper_id: francisbeaman2023roleconfidenceknowledge
citation: 'Francis, K. B., & Beaman, C. P. (2023). The role of confidence in knowledge ascriptions: an evidence-seeking
approach. Synthese.'
short_label: Francis & Beaman 2023
doi: 10.1007/s11229-023-04236-w
published: 'Yes'
year: 2023
language: English
language_other: null
research_objective: Test whether stakes (low vs high) affect knowledge judgments in an evidence-seeking paradigm, and whether
participant confidence mediates any stakes effect; includes measures of participant confidence and protagonist confidence.
data_availability:
data_available_online: 'Yes'
url: https://osf.io/n8x7q/
notes: Paper states all measures and collected data are available on the project's OSF page.
notes: null
studies:
- study_id: 1
label: Positive polarity condition ("Know")
language: English
language_other: null
objective: Test whether practical stakes (low vs high) affect evidence-seeking judgments (minimum evidence required for
knowledge) and confidence ratings in the positive polarity prompt condition.
sample:
n_final: 97
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: $1.75
characteristics: MTurk participants (data collected in 2019). Final analyzed sample across both polarity conditions N=187
(69 females, 118 males), ages 19–66 (M=34.70, SD=9.92). Positive polarity group N=97.
mean_age: null
provenance:
page: 5
quote: Participants (N = 249) were recruited from MTurk and paid $1.75 each... leaving a final sample of 187 participants
(69 females, 118 males) between 19 and 66 years old (M = 34.70 years, SD = 9.92 years). Participants were randomly
assigned to the positive polarity condition (N = 97) or the negative polarity condition (N = 90).
design: Within-Subjects
design_other: 'Within-subject stakes manipulation: each participant saw low- and high-stakes versions of six scenarios;
analysis uses per-participant averages for low vs high stakes.'
manipulated_factors:
- 'Stakes: low vs high (within-subjects)'
paradigm: Rating how much evidence is needed for knowledge
paradigm_other: null
scale:
label: numeric/text input
points: null
anchors: 'Numeric free response (# checks); positive prompt: whole number (0,1,2,...) or ''never''.'
direction: Higher numbers indicate more evidence required for knowledge (more checks).
provenance:
page: null
quote: 'Positive polarity enter a whole number: 1, 2, 3 . . . etc. if you think Megan knows without having to check,
write "0". If you think Megan will never know no matter how many times she checks, write "never"'
measures:
knowledge_question_text: What is the minimum number of times Megan needs to check her GPS before she knows that she will
make it to the accident without taking a wrong turn?
knowledge_question_first: null
additional_question_text: "Participant confidence prompt: How confident are you that Megan needs to check a minimum of\
\ that many times in order to know that she will make it to the accident without taking a wrong turn?\r\nProtagonist\
\ confidence prompt: How confident is Megan that she will make it to the accident without taking a wrong turn?"
effects:
- effect_id: s1_e1
subgroup: Knowledge (evidence-seeking) — Positive polarity
subgroup_desc: Average evidence-seeking score (knowledge) for low vs high stakes (collapsed across six scenarios).
design: Within-Subjects
design_other: Same participants provided low- and high-stakes evidence-seeking judgments across six scenarios; Table 1
reports per-participant averages.
moderators:
scenario: other
skeptical_pressure: 'No'
awareness: 'Yes'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: null
quote: The highest-and lowest-stakes versions of six scenarios... were presented to each participant. These scenarios
involved the manipulation of different types of stakes (lives; physical injury; embarrassment; money; damage to
objects of personal value).
tei_id: null
table_ref: null
reason: Effect is aggregated across six different scenarios, so no single scenario code applies.
skeptical_pressure:
provenance:
page: null
quote: Megan is familiar with the surrounding area... and she is traveling on the right route to get to the accident.
tei_id: null
table_ref: null
reason: The vignette examples do not introduce an explicit doubt/counterconsideration; stakes are manipulated via
consequences.
awareness:
provenance:
page: null
quote: Over the radio, Megan is told that there is one person at the scene of the accident... If Megan makes a wrong
turn... nothing terrible will happen. ... If Ottoline makes a wrong turn... the children will die.
tei_id: null
table_ref: null
reason: The stakes are conveyed to the protagonist within the vignette (she is told the consequences).
evidence:
provenance:
page: null
quote: What is the minimum number of times Megan needs to check her GPS... / How many times does Elaine need to
consult her check list before she knows that she is making the vaccine correctly?
tei_id: null
table_ref: null
reason: Because the outcome is averaged across multiple scenarios with different evidence sources (e.g., GPS checks
vs checklist consultation), evidence cannot be coded uniquely.
attribution_person:
provenance:
page: null
quote: What is the minimum number of times Megan needs to check her GPS before she knows that she will make it to
the accident without taking a wrong turn?
tei_id: null
table_ref: null
reason: Participants evaluate a third-person knowledge claim about the protagonist (e.g., 'Megan knows...').
evidence_reliability:
provenance:
page: null
quote: Megan... has GPS on her phone that she can check if necessary...
tei_id: null
table_ref: null
reason: Evidence-source reliability is not explicitly manipulated or characterized in a way that supports High/Medium/Low
coding.
contrast:
group_high: high_stakes
group_low: low_stakes
sign_convention: d = mean(low) - mean(high)
other_notes: Table 1 reports per-participant averages across six scenarios for low vs high stakes.
groups:
- group_id: low_stakes
label: null
n: 97
mean: 2.34
sd: 2.01
se: null
provenance:
page: 8
quote: 'Table 1 (Positive; Knowledge evidence-seeking): Low 2.34 (2.01).'
tei_id: tab_0
table_ref: camelot_stream_p8_t1.csv
- group_id: high_stakes
label: null
n: 97
mean: 4.19
sd: 3.72
se: null
provenance:
page: 8
quote: 'Table 1 (Positive; Knowledge evidence-seeking): High 4.19 (3.72).'
tei_id: tab_0
table_ref: camelot_stream_p8_t1.csv
reported_test:
test: t (mediation path c; stakes → knowledge)
t: -5.9
df1: 96.0
notes: Reported as p < 0.001; 95% CI [-2.47, -1.23].
provenance:
page: 10
quote: For the positive polarity condition... effect of stakes on knowledge via the evidence-seeking path (path c),
t(96) = -5.90, p < 0.001, 95% CI [-2.47, -1.23]
effect_size:
metric: SMD
d: -0.618758512614
v: 0.010411370788
computed_from: groups
needs_review: false
notes: Computed in analysis/effect_sizes.qmd using within_smcrp_t (paired t used to recover within-person r).
quality_flags: []
notes: null
notes: Block one (evidence-seeking prompt followed by the participant confidence prompt) and block two (the protagonist
confidence prompt) were then presently randomly after each scenario to avoid order effects.
- study_id: 2
label: Negative polarity condition ("Don't know")
language: English
language_other: null
objective: Test whether practical stakes (low vs high) affect evidence-seeking judgments (maximum evidence that can be gathered
while still not knowing) and confidence ratings in the negative polarity prompt condition.
sample:
n_final: 89
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: $1.75
characteristics: MTurk participants (data collected in 2019). Participants were assigned to negative polarity N=90, but
1 participant with extreme values in the negative polarity condition was removed prior to analysis (df=88 suggests n=89).
mean_age: 34.7
provenance:
page: 5
quote: Participants were randomly assigned to the positive polarity condition (N = 97) or the negative polarity condition
(N = 90)... One participant reported extreme values in the negative polarity condition... this was removed prior to
further analysis.
mean_age_prov:
page: 5
quote: "final sample of 187 participants (69 females, 118 males) between 19 and\r\n66 years old (M = 34.70 years, SD\
\ = 9.92 years)"
design: Within-Subjects
design_other: 'Within-subject stakes manipulation: each participant saw low- and high-stakes versions of six scenarios;
analysis uses per-participant averages for low vs high stakes.'
manipulated_factors:
- 'Stakes: low vs high (within-subjects)'
paradigm: Rating how much evidence is needed for knowledge
paradigm_other: null
scale:
label: numeric/text input
points: null
anchors: 'Numeric free response (# checks); negative prompt: whole number (1,2,3,...) or ''never''.'
direction: Higher numbers indicate more evidence required for knowledge (more checks).
provenance:
page: 6
quote: 'Negative polarity enter a whole number: 1, 2, 3 . . . etc. if you think Megan will never know no matter how
many times she checks, write "never"'
measures:
knowledge_question_text: "What is the maximum number of times Megan can check her GPS and not\r\nknow that she will make\
\ it to the accident without taking a wrong turn?"
knowledge_question_first: null
additional_question_text: "How confident are you that Megan can check a maximum of that many times\r\nand not know that\
\ she will make it to the accident without taking a wrong turn?\r\n\r\nHow confident is Megan that she will make it\
\ to the accident without taking a wrong turn?"
effects:
- effect_id: s2_e1
subgroup: Knowledge (evidence-seeking) — Negative polarity
subgroup_desc: Average evidence-seeking score (knowledge) for low vs high stakes (collapsed across six scenarios).
design: Within-Subjects
design_other: Same participants provided low- and high-stakes evidence-seeking judgments across six scenarios; Table 1
reports per-participant averages.
moderators:
scenario: other
skeptical_pressure: 'No'
awareness: 'Yes'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: null
quote: The highest-and lowest-stakes versions of six scenarios... were presented to each participant. These scenarios
involved the manipulation of different types of stakes (lives; physical injury; embarrassment; money; damage to
objects of personal value).
tei_id: null
table_ref: null
reason: Effect is aggregated across six different scenarios, so no single scenario code applies.
skeptical_pressure:
provenance:
page: null
quote: Megan is familiar with the surrounding area... and she is traveling on the right route to get to the accident.
tei_id: null
table_ref: null
reason: The vignette examples do not introduce an explicit doubt/counterconsideration; stakes are manipulated via
consequences.
awareness:
provenance:
page: null
quote: Over the radio, Megan is told that there is one person at the scene of the accident... If Megan makes a wrong
turn... nothing terrible will happen. ... If Ottoline makes a wrong turn... the children will die.
tei_id: null
table_ref: null
reason: The stakes are conveyed to the protagonist within the vignette (she is told the consequences).
evidence:
provenance:
page: null
quote: What is the maximum number of times Megan can check her GPS and not know that she will make it to the accident
without taking a wrong turn? / How many times can Elaine consult her check list and still not know that she is
making the vaccine correctly
tei_id: null
table_ref: null
reason: Because the outcome is averaged across multiple scenarios with different evidence sources (e.g., GPS checks
vs checklist consultation), evidence cannot be coded uniquely.
attribution_person:
provenance:
page: null
quote: What is the maximum number of times Megan can check her GPS and not know that she will make it to the accident
without taking a wrong turn?
tei_id: null
table_ref: null
reason: Participants evaluate a third-person knowledge claim about the protagonist (e.g., 'Megan knows...').
evidence_reliability:
provenance:
page: null
quote: Megan... has GPS on her phone that she can check if necessary...
tei_id: null
table_ref: null
reason: Evidence-source reliability is not explicitly manipulated or characterized in a way that supports High/Medium/Low
coding.
contrast:
group_high: high_stakes
group_low: low_stakes
sign_convention: d = mean(low) - mean(high)
other_notes: Table 1 reports per-participant averages across six scenarios for low vs high stakes.
groups:
- group_id: low_stakes
label: null
n: 89
mean: 3.31
sd: 3.95
se: null
provenance:
page: 8
quote: 'Table 1 (Negative; Knowledge evidence-seeking): Low 3.31 (3.95).'
tei_id: tab_0
table_ref: camelot_stream_p8_t1.csv
- group_id: high_stakes
label: null
n: 89
mean: 2.43
sd: 3.02
se: null
provenance:
page: 8
quote: 'Table 1 (Negative; Knowledge evidence-seeking): High 2.43 (3.02).'
tei_id: tab_0
table_ref: camelot_stream_p8_t1.csv
reported_test:
test: t (mediation path c; stakes → knowledge)
t: 2.25
df1: 88.0
notes: Reported as p = 0.027; 95% CI [0.10, 1.65].
provenance:
page: 11
quote: Mediation analyses found a statistically significant effect of stakes on knowledge (evidence-seeking) (path
c), t(88) = 2.25, p = 0.027, 95% CI [0.10, 1.65]
effect_size:
metric: SMD
d: -0.250292573585
v: 0.012222668402
computed_from: groups
needs_review: false
notes: 'Computed in analysis/effect_sizes.qmd using within_smcrp_t (paired t used to recover within-person r). Needs
review: Table 1 suggests mean(high) < mean(low) for negative polarity, which may appear to conflict with narrative
text stating higher evidence-seeking scores for high stakes across both polarities. Effect reversed because of negative
polarity prompt.'
quality_flags: []
notes: null
notes: Block one (evidence-seeking prompt followed by the participant confidence prompt) and block two (the protagonist
confidence prompt) were then presently randomly after each scenario to avoid order effects.