pinillossimpsonndexperimentalevidencesupporting
/data/papers/pinillossimpsonndexperimentalevidencesupporting/pinillossimpsonndexperimentalevidencesupporting.yamlschema_version: '1.2'
paper:
paper_id: pinillossimpsonndexperimentalevidencesupporting
citation: 'Pinillos, N. Á., & Simpson, S. (2014). Experimental Evidence Supporting Anti-intellectualism About Knowledge.
In J. R. Beebe (Ed.), Advances in Experimental Epistemology (Chapter 1, pp. 9–44). Bloomsbury Academic. ISBN: 978-1-4725-0737-2.'
short_label: Pinillos & Simpson 2014
doi: null
published: 'Yes'
year: null
language: English
language_other: null
research_objective: Provide experimental evidence for anti-intellectualism about knowledge by testing whether folk judgments
about when an agent knows are sensitive to practical interests (stakes), using evidence-seeking and agreement paradigms,
and by addressing objections to evidence-seeking probes.
data_availability:
data_available_online: null
url: null
notes: null
notes: Year/venue details were not recoverable from the provided PDF text/metadata; keep as n.d. pending manual bibliographic
completion.
studies:
- study_id: 1
label: Study 1, Experiment 1 (Water purifier; evidence-seeking)
language: English
language_other: null
objective: Test whether participants require more evidence for knowledge under higher stakes (water purifier vignette; numeric
evidence-seeking prompt).
sample:
n_final: 94
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: 15 cents (US)
characteristics: Amazon Turk workers; initial N=141; exclusions reported; computed statistics for N=94. Given 10 minutes
to respond
mean_age: null
provenance:
page: 10
quote: A total of 141 subjects from Amazon Turk were paid 15 cents (US) each to take exactly one of two surveys.
design: Between-Subjects
design_other: 'Two independent conditions: Low Stakes vs High Stakes.'
manipulated_factors: []
paradigm: Rating how much evidence is needed for knowledge
paradigm_other: null
scale:
label: numeric/text input
points: null
anchors: 'Numeric free response (whole number). Instructions: write “0” if already knows; write “never” if never knows.'
direction: Higher numbers indicate more checks/comparisons required (more evidence needed for knowledge).
provenance:
page: 11
quote: 'After how many comparisons will Brian know he has written them down correctly? ... This should be a whole number.
(Note: If you think Brian already knows, write “0.” If you think he’ll never know, no matter how many times he checks,
write “never”).'
measures:
knowledge_question_text: "(Opinion question) Suppose Brian goes back and compares his entire \r\nwritten copy to the instructions\
\ online, and he can do this as many times \r\nas he wants. After how many comparisons will Brian know he has written\
\ \r\nthem down correctly? Please write your answer in the box below. This \r\nshould be a whole number. (Note: If you\
\ think Brian already knows, write \r\n“0.” If you think he’ll never know, no matter how many times he checks, \r\n\
write “never”)."
knowledge_question_first: null
additional_question_text: The reading comprehension checks were placed before the target question and included a question
to see if subjects were aware of what was at stake.
scenarios:
- scenario_code: water_purifier
scenario_type: Installing a water purifier; high-stakes manipulation involves unknown poisoning risk.
high_stakes_text: In High Stakes, it is also the case that the water supply has been poisoned... If Brian fails to assemble
the water purifier properly, he and his family might die. However, Brian is ignorant of these high stakes.
low_stakes_text: 'Low Stakes: Brian installs the purifier because he does not like the taste of tap water.'
provenance:
page: 10
quote: They both concern a protagonist, Brian, who is installing a water purifier... However, in High Stakes, it is
also the case that the water supply has been poisoned... he and his family might die. However, Brian is ignorant of
these high stakes.
effects:
- effect_id: s1_e1
subgroup: Water purifier — evidence-seeking knowledge
subgroup_desc: Number of comparisons required for knowledge
design: Between-Subjects
design_other: null
moderators:
scenario: other
skeptical_pressure: 'No'
awareness: 'No'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 10
quote: Brian... is installing a water purifier at home... in High Stakes... the water supply has been poisoned.
tei_id: null
table_ref: null
reason: The vignette is about a water purifier/poisoning scenario, not a bank/peanuts/bridge/typos vignette.
skeptical_pressure:
provenance:
page: 11
quote: After how many comparisons will Brian know he has written them down correctly?
tei_id: null
table_ref: null
reason: No explicit alternative/error possibility is raised as a live option; the manipulation concerns consequences.
awareness:
provenance:
page: 10
quote: However, Brian is ignorant of these high stakes.
tei_id: null
table_ref: null
reason: High-stakes consequences are explicitly unknown to the protagonist.
evidence:
provenance:
page: 11
quote: Suppose Brian goes back and compares his entire written copy to the instructions online...
tei_id: null
table_ref: null
reason: The evidence is generated by the agent’s own checking/comparing (first-person evidence).
attribution_person:
provenance:
page: 11
quote: After how many comparisons will Brian know...
tei_id: null
table_ref: null
reason: Participants attribute knowledge to Brian (third-person attribution).
evidence_reliability:
provenance:
page: 11
quote: "In order to get a copy, Brian logged onto his computer down the hall and pulled up the instructions from\
\ the manufacturer’s website. He wrote the instructions on a sheet of paper \r\n(he didn’t have a printer). Feeling\
\ confident and believing he wrote them down correctly, he headed back to the faucet."
tei_id: null
table_ref: null
reason: compared the instructions once
contrast:
group_high: WaterPurifier_high
group_low: WaterPurifier_low
sign_convention: d = mean(low) - mean(high)
other_notes: Outcome is number of comparisons/checks; higher scores indicate more evidence needed.
groups:
- group_id: WaterPurifier_low
label: null
n: 46
mean: 0.72
sd: 0.72
se: null
provenance:
page: 11
quote: Low Stakes (N 46, m 0.72, sd 0.72)
tei_id: null
table_ref: null
- group_id: WaterPurifier_high
label: null
n: 48
mean: 1.29
sd: 1.254
se: null
provenance:
page: 11
quote: High Stakes (N 48, m 1.29, sd 1.254)
tei_id: null
table_ref: null
reported_test:
test: t
t: -2.7
df1: 75.54
reported_d: 0.54
notes: Reported as p < 0.01. Paper’s reported Cohen’s d is unsigned/positive; extraction uses sign convention d = mean(low)
- mean(high).
provenance:
page: 11
quote: t(75.54) 2.70, p 0.01. Cohen’s d 0.54 (this is a medium-size effect).
effect_size:
metric: SMD
d: -0.554440139474935
v: 0.0442431369651865
computed_from: groups
needs_review: false
notes: Computed from group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups).
quality_flags: []
notes: null
notes: full texts of the vignettes in the Appendix
- study_id: 2
label: Study 1, Experiment 2 (Airplane; evidence-seeking; stakes × probability)
language: English
language_other: null
objective: Test whether participants require more evidence for knowledge under higher stakes and/or higher probability of
negative outcomes (airline roster vignette; 2×2 between-subjects).
sample:
n_final: 230
recruitment: students
recruitment_other: Volunteer students taking introductory courses at Arizona State University (randomly distributed surveys).
compensation: no compensation
compensation_other: Volunteer students
characteristics: Students taking introductory courses at Arizona State University. Four conditions (LSLP/LSHP/HSLP/HSHP);
initial surveys distributed total=305; exclusions reported; computed statistics for N=230.
mean_age: null
provenance:
page: 13
quote: We discarded 75 surveys... We computed statistics for N 230 subjects. LSLP (N 50, m 1.6, sd 0.969), LSHP
(N 58, m 1.76, sd 0.823), HSLP (N 61, m 1.93, sd 0.944), and HSHP (N 61, m 2.15, sd 1.152) (Figure
1.1).
table_ref: Figure 1.1
design: Between-Subjects
design_other: '2 (stakes: low vs high) × 2 (probability: low vs high) between-subjects.'
manipulated_factors:
- 'Probability of outcome: low vs high'
paradigm: Rating how much evidence is needed for knowledge
paradigm_other: null
scale:
label: numeric/text input
points: null
anchors: 'Numeric free response: number of additional roster surveys (whole number 0,1,2,3...) or “never”.'
direction: Higher numbers indicate more evidence needed for knowledge.
provenance:
page: 13
quote: How many more times do you think Jessie needs to survey the entire roster before he knows the name is not on
the list (enter a whole number... or write “never” if you think Jessie will never know).
measures:
knowledge_question_text: "We are now interested in your opinion about what it would take for Jessie \r\nto know that the\
\ name is not on the roster (the name of the nice guy/\r\nhijacker). Recall that according to the story, Jessie has\
\ already surveyed \r\nthe entire roster once. How many more times do you think Jessie needs to \r\nsurvey the entire\
\ roster before he knows the name is not on the list (enter a \r\nwhole number: 0,1,2,3, . . . etc. or write “never”\
\ if you think Jessie will never \r\nknow)."
knowledge_question_first: null
additional_question_text: "The surveys contained reading comprehension checks. The reading comprehension checks occurred\
\ before the target question and they included \r\nquestions checking to see if subjects knew what was at stake (and\
\ the probabilities)."
scenarios:
- scenario_code: airplane_roster
scenario_type: Airline steward checks passenger roster; stakes and probability of hijacking (high) vs first-class upgrade
acceptance (low).
high_stakes_text: 'High stakes: if wrong, (low/high probability) a criminal would hijack the plane; Jessie is unaware
of hijacker/probability.'
low_stakes_text: 'Low stakes: if wrong, (low/high probability) a nice guy would accept first class; Jessie is unaware
of nice guy/probability.'
provenance:
page: 13
quote: In high stakes–high probability (HSHP)... a criminal, would hijack the plane... In low stakes–high probability
(LSHP)... a nice guy, would accept the invitation to go to first class... Again, Jessie is unaware...
effects:
- effect_id: s2_e1
subgroup: Airplane — Low probability (LSLP vs HSLP)
subgroup_desc: 'Evidence-seeking: roster checks required for knowledge (low probability)'
design: Between-Subjects
design_other: null
moderators:
scenario: airport
skeptical_pressure: 'No'
awareness: 'No'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 13
quote: They are about an airline steward, Jessie, who is assigned to find a name on a roster of 200 passengers before
a flight.
tei_id: null
table_ref: null
reason: The scenario is an airplane/roster vignette; coded airport.
skeptical_pressure:
provenance:
page: 13
quote: How many more times do you think Jessie needs to survey the entire roster before he knows the name is not
on the list...
tei_id: null
table_ref: null
reason: No explicit alternative/error possibility is raised as a conversational challenge; stakes/probability are
manipulated as consequences.
awareness:
provenance:
page: 13
quote: Again, Jessie is unaware that the name belongs to a nice guy/hijacker and unaware of the probability...
tei_id: null
table_ref: null
reason: The protagonist is explicitly unaware of the stake-relevant facts and probabilities.
evidence:
provenance:
page: 13
quote: How many more times do you think Jessie needs to survey the entire roster...
tei_id: null
table_ref: null
reason: Evidence is generated by the agent’s own repeated surveying of the roster (first-person).
attribution_person:
provenance:
page: 13
quote: before he knows the name is not on the list
tei_id: null
table_ref: null
reason: Participants attribute knowledge to Jessie (third-person attribution).
evidence_reliability:
provenance:
page: 13
quote: "After looking through the roster \r\nonce, Jessie thinks this name is not on the list, and in fact it is\
\ not on the list."
tei_id: null
table_ref: null
reason: checked once
contrast:
group_high: HSLP
group_low: LSLP
sign_convention: d = mean(low) - mean(high)
other_notes: Probability held constant at low probability; outcome is number of additional roster surveys required.
groups:
- group_id: LSLP
label: null
n: 50
mean: 1.6
sd: 0.969
se: null
provenance:
page: 13
quote: LSLP (N 50, m 1.6, sd 0.969)
tei_id: null
table_ref: Figure 1.1
- group_id: HSLP
label: null
n: 61
mean: 1.93
sd: 0.944
se: null
provenance:
page: 13
quote: HSLP (N 61, m 1.93, sd 0.944)
tei_id: null
table_ref: Figure 1.1
reported_test:
notes: No simple-effect test for LSLP vs HSLP reported; paper reports an overall main effect for stakes across probability
conditions.
provenance:
page: 14
quote: Comparing all the low-stakes cases against the high-stakes cases reveals a statistically significant main effect
for stakes F(1,224) 7.639, p 0.01. Cohen’s d 0.38 (moderate effect size).
effect_size:
metric: SMD
d: -0.345434178152552
v: 0.0369408039598129
computed_from: groups
needs_review: false
notes: Computed from group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups).
quality_flags: []
notes: null
- effect_id: s2_e2
subgroup: Airplane — High probability (LSHP vs HSHP)
subgroup_desc: 'Evidence-seeking: roster checks required for knowledge (high probability)'
design: Between-Subjects
design_other: null
moderators:
scenario: airport
skeptical_pressure: 'No'
awareness: 'No'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 13
quote: They are about an airline steward, Jessie, who is assigned to find a name on a roster of 200 passengers before
a flight.
tei_id: null
table_ref: null
reason: The scenario is an airplane/roster vignette; coded airport.
skeptical_pressure:
provenance:
page: 13
quote: How many more times do you think Jessie needs to survey the entire roster before he knows the name is not
on the list...
tei_id: null
table_ref: null
reason: No explicit alternative/error possibility is raised as a conversational challenge; stakes/probability are
manipulated as consequences.
awareness:
provenance:
page: 13
quote: Again, Jessie is unaware that the name belongs to a nice guy/hijacker and unaware of the probability...
tei_id: null
table_ref: null
reason: The protagonist is explicitly unaware of the stake-relevant facts and probabilities.
evidence:
provenance:
page: 13
quote: How many more times do you think Jessie needs to survey the entire roster...
tei_id: null
table_ref: null
reason: Evidence is generated by the agent’s own repeated surveying of the roster (first-person).
attribution_person:
provenance:
page: 13
quote: before he knows the name is not on the list
tei_id: null
table_ref: null
reason: Participants attribute knowledge to Jessie (third-person attribution).
evidence_reliability:
provenance:
page: 13
quote: "After looking through the roster \r\nonce, Jessie thinks this name is not on the list, and in fact it is\
\ not on the list."
tei_id: null
table_ref: null
reason: checked once
contrast:
group_high: HSHP
group_low: LSHP
sign_convention: d = mean(low) - mean(high)
other_notes: Probability held constant at high probability; outcome is number of additional roster surveys required.
groups:
- group_id: LSHP
label: null
n: 58
mean: 1.76
sd: 0.823
se: null
provenance:
page: 13
quote: LSHP (N 58, m 1.76, sd 0.823)
tei_id: null
table_ref: Figure 1.1
- group_id: HSHP
label: null
n: 61
mean: 2.15
sd: 1.152
se: null
provenance:
page: 13
quote: HSHP (N 61, m 2.15, sd 1.152)
tei_id: null
table_ref: Figure 1.1
reported_test:
notes: No simple-effect test for LSHP vs HSHP reported; paper reports an overall main effect for stakes across probability
conditions.
provenance:
page: 14
quote: Comparing all the low-stakes cases against the high-stakes cases reveals a statistically significant main effect
for stakes F(1,224) 7.639, p 0.01. Cohen’s d 0.38 (moderate effect size).
effect_size:
metric: SMD
d: -0.387959471896737
v: 0.034278037966776
computed_from: groups
needs_review: false
notes: Computed from group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups).
quality_flags: []
notes: full vignettes not provided in the article
notes: full vignettes in the Appendix
- study_id: 3
label: Study 2 (Coin; agreement with knowledge ascription)
language: English
language_other: null
objective: Test whether stakes affect agreement with a knowledge ascription in a coin-counting vignette.
sample:
n_final: 165
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: null
characteristics: Amazon Turk workers in the United States; coin-counting vignette; Table 1.1 reports group Ns.
mean_age: null
provenance:
page: 15
quote: The surveys were taken by Amazon Turk workers in the United States.
design: Between-Subjects
design_other: 'Two independent conditions: Low stakes vs High stakes.'
manipulated_factors: []
paradigm: Agreement with knowledge claim
paradigm_other: null
scale:
label: Likert 7-point
points: 7
anchors: 0–6; 6 = strongly agree; 3 = neutral
direction: Higher numbers indicate stronger agreement with the knowledge ascription.
provenance:
page: 15
quote: a 7-point Likert scale (0-6) where 6 is "strongly agree" and 3 is "neutral"
measures:
knowledge_question_text: 'To what extent do you agree with the statement: "PETER KNOWS THERE ARE 134 COINS IN THE JAR"'
knowledge_question_first: 'No'
additional_question_text: "For these experiments, we also added a “normative” question before the \r\nknowledge prompt.\
\ For example, in the COIN case we asked whether the \r\nsubject thought Peter should count the pennies again."
scenarios:
- scenario_code: coin
scenario_type: Counting coins in a jar for a contest; stakes manipulation via prize importance and mother’s operation.
high_stakes_text: "Coin High Stakes: Peter is a college student who has entered a contest \r\nsponsored by a local bank.\
\ His task is to count the coins in a jar. The jar contains 134 coins. Peter mistakenly thinks the contest prize is\
\ one hundred dollars. In fact, the prize is $10,000 which Peter really needs. He would use the money to help pay for\
\ a life-saving operation for his mother who is sick and cannot afford healthcare. So the stakes are high for Peter\
\ since if he \r\ndoesn’t win the contest, his mother could die. After counting the coins just once, Peter concludes\
\ there are 134 coins in the jar. His friend, who also thinks the prize is one hundred dollars says to Peter “you only\
\ counted once, even if there are in fact 134 coins in the jar, you don’t know there are 134 coins in the jar. You should\
\ count them again.”"
low_stakes_text: "Coin Low Stakes: Peter is a college student who has entered a contest sponsored by a local bank. His\
\ task is to count the coins in a jar. The jar contains 134 coins. Peter mistakenly thinks the contest prize is one\
\ hundred dollars. In fact, the prize is just a pair of movie passes for this weekend. Peter wouldn’t want them, however,\
\ since he is leaving town this weekend. So \r\nnothing bad would happen if Peter doesn’t win the contest. After counting\
\ the coins just once, Peter concludes there are 134 coins in the jar. His friend, who also thinks the prize is one\
\ hundred dollars says to Peter “you only counted once, even if there are in fact 134 coins in the jar, you don’t know\
\ there are 134 coins in the jar. You should count them again.”"
provenance:
page: 15
quote: Coin Low Stakes... the prize is just a pair of movie passes... So nothing bad would happen... Coin High Stakes...
the prize is $10,000... life-saving operation for his mother... his mother could die.
effects:
- effect_id: s3_e1
subgroup: Coin — knowledge ascription agreement
subgroup_desc: Agreement with knowledge statement (Likert 0–6)
design: Between-Subjects
design_other: null
moderators:
scenario: other
skeptical_pressure: 'Yes'
awareness: 'No'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 15
quote: Coin... count the coins in a jar. The jar contains 134 coins.
tei_id: null
table_ref: null
reason: The vignette concerns coin-counting (not bank/peanuts/bridge/typos).
skeptical_pressure:
provenance:
page: 15
quote: His friend... says to Peter “you only counted once... you don’t know there are 134 coins in the jar... You
should count them again.”
tei_id: null
table_ref: null
reason: A friend explicitly denies knowledge and challenges the adequacy of the evidence.
awareness:
provenance:
page: 15
quote: the protagonist is ignorant of what is at stake... the protagonist is always mistaken about what is at stake
tei_id: null
table_ref: null
reason: The paper states protagonists are ignorant/mistaken about the stakes in both low/high conditions.
evidence:
provenance:
page: 15
quote: After counting the coins just once, Peter concludes there are 134 coins in the jar.
tei_id: null
table_ref: null
reason: Evidence is generated by the agent’s own counting (first-person evidence).
attribution_person:
provenance:
page: 15
quote: '"PETER KNOWS THERE ARE 134 COINS IN THE JAR"'
tei_id: null
table_ref: null
reason: Participants judge a third-person knowledge ascription.
evidence_reliability:
provenance:
page: 15
quote: After counting the coins just once...
tei_id: null
table_ref: null
reason: counted once
contrast:
group_high: Coin_high
group_low: Coin_low
sign_convention: d = mean(low) - mean(high)
other_notes: Outcome is agreement with the knowledge statement; higher scores indicate more agreement.
groups:
- group_id: Coin_low
label: null
n: 87
mean: 3.68
sd: 1.8
se: null
provenance:
page: 17
quote: 'Coin (Likert: 0–6) N 87, m 3.68, sd 1.80'
tei_id: null
table_ref: Table 1.1
- group_id: Coin_high
label: null
n: 78
mean: 3.06
sd: 1.76
se: null
provenance:
page: 17
quote: 'Coin (Likert: 0–6) N 78, m 3.06, sd 1.76'
tei_id: null
table_ref: Table 1.1
reported_test:
test: t
t: 2.23
df1: 161.78
p: 0.023
reported_d: 0.35
provenance:
page: 17
quote: Coin vignettes t(161.78) 2.23, p 0.023, d 0.35 (small effect size)
effect_size:
metric: SMD
d: 0.348076772822079
v: 0.0246864142823528
computed_from: groups
needs_review: false
notes: Computed from Table 1.1 group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups).
quality_flags: []
notes: null
notes: null
- study_id: 4
label: Study 2 (Air; agreement with knowledge ascription)
language: English
language_other: null
objective: Test whether stakes affect agreement with a knowledge ascription in the Air probe.
sample:
n_final: 55
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: null
characteristics: Amazon Turk workers in the United States; Air probe appendix material located in out/external/pinillos
IRI paper with shawn october 2014.pdf; Table 1.1 reports group Ns.
mean_age: null
provenance:
page: 15
quote: The surveys were taken by Amazon Turk workers in the United States.
design: Between-Subjects
design_other: 'Two independent conditions: Low stakes vs High stakes.'
manipulated_factors: []
paradigm: Agreement with knowledge claim
paradigm_other: null
scale:
label: Likert 5-point
points: 5
anchors: 0–4; 4 = strongly agree; 2 = neutral
direction: Higher numbers indicate stronger agreement with the knowledge ascription.
provenance:
page: 15
quote: Air and Bridge, with 5-point Likert scales (0-4) where 4 is "strongly agree" and 2 is "neutral" (See Web Appendix).
measures:
knowledge_question_text: After surveying the roster just that one time, Jason knew that the name was not on the list
knowledge_question_first: 'No'
additional_question_text: "For these experiments, we also added a “normative” question before the \r\nknowledge prompt.\
\ For example, in the COIN case we asked whether the \r\nsubject thought Peter should count the pennies again."
scenarios:
- scenario_code: air
scenario_type: Air probe; appendix includes Air vignettes and the level-of-agreement prompt.
high_stakes_text: Jessie checks a flight roster once, thinks the stakes are low, but the name actually belongs to a dangerous
individual wanted by the FBI; if the individual were on the flight, the plane could be hijacked.
low_stakes_text: Jessie checks a flight roster once, thinks the stakes are low, and the name actually belongs to a nice
passenger who might be bumped up to first class.
provenance:
page: 45
quote: APPENDIX (This will be a WEB Appendix)... AIR (HSHP)... AIR(HSLP)... AIR (LSHP)... AIR(LSLP)... Level of Agreement
Question for Air.
effects:
- effect_id: s4_e1
subgroup: Air — knowledge ascription agreement
subgroup_desc: Agreement with knowledge statement (Likert 0–4)
design: Between-Subjects
design_other: null
moderators:
scenario: other
skeptical_pressure: 'No'
awareness: 'No'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 15
quote: We developed three pairs of vignettes (Coin, Air, and Bridge)
tei_id: null
table_ref: null
reason: Probe is labeled 'Air'; the vignette is not bank/peanuts/bridge/typos.
skeptical_pressure:
provenance:
page: 15
quote: null
tei_id: null
table_ref: null
reason: no skeptical pressure
awareness:
provenance:
page: 15
quote: in both the low-stakes and high-stakes conditions, the protagonist is ignorant of what is at stake.
tei_id: null
table_ref: null
reason: Paper explicitly states protagonists are ignorant of the stakes in Study 2 vignettes.
evidence:
provenance:
page: 32
quote: The normative prompt for AIR was “Do you think Jason should look through the entire roster at least one more
time?”
tei_id: null
table_ref: null
reason: The Air probe involves the agent looking through a roster, indicating first-person evidence from the agent’s
own checking.
attribution_person:
provenance:
page: 15
quote: Subjects are asked about the extent to which they agree or disagree with a statement that says that the protagonist
knows P.
tei_id: null
table_ref: null
reason: The DV is agreement with a third-person knowledge ascription.
evidence_reliability:
provenance:
page: 15
quote: null
tei_id: null
table_ref: null
reason: checked once
contrast:
group_high: Air_high
group_low: Air_low
sign_convention: d = mean(low) - mean(high)
other_notes: Outcome is agreement with the knowledge statement; higher scores indicate more agreement.
groups:
- group_id: Air_low
label: null
n: 25
mean: 2.16
sd: 1.03
se: null
provenance:
page: 17
quote: Air (Likert 0–4) N 25, m 2.16, sd 1.03
tei_id: null
table_ref: Table 1.1
- group_id: Air_high
label: null
n: 30
mean: 2.03
sd: 1.0
se: null
provenance:
page: 17
quote: Air (Likert 0–4) N 30, m 2.03, sd 1.0
tei_id: null
table_ref: Table 1.1
reported_test:
test: t
t: 0.462
df1: 53.0
p: 0.646
provenance:
page: 32
quote: '21 Air: t(53) 0.462, p 0.646.'
effect_size:
metric: SMD
d: 0.128243714633482
v: 0.0734884885252484
computed_from: groups
needs_review: false
notes: Computed from Table 1.1 group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups). Appendix material
located in out/external/pinillos IRI paper with shawn october 2014.pdf; Air vignette/prompt reviewed.
quality_flags: []
notes: null
notes: Appendix material located in out/external/pinillos IRI paper with shawn october 2014.pdf.
- study_id: 5
label: Study 2 (Bridge; agreement with knowledge ascription)
language: English
language_other: null
objective: Test whether stakes affect agreement with a knowledge ascription in the Bridge probe.
sample:
n_final: null
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: null
characteristics: Amazon Turk workers in the United States; Bridge probe appendix material located in out/external/pinillos
IRI paper with shawn october 2014.pdf; Table 1.1 reports group Ns.
mean_age: null
provenance:
page: 15
quote: The surveys were taken by Amazon Turk workers in the United States.
design: Between-Subjects
design_other: 'Two independent conditions: Low stakes vs High stakes.'
manipulated_factors: []
paradigm: Agreement with knowledge claim
paradigm_other: null
scale:
label: Likert 5-point
points: 5
anchors: 0–4; 4 = strongly agree; 2 = neutral
direction: Higher numbers indicate stronger agreement with the knowledge ascription.
provenance:
page: 15
quote: Air and Bridge, with 5-point Likert scales (0-4) where 4 is "strongly agree" and 2 is "neutral" (See Web Appendix).
measures:
knowledge_question_text: 'Assume the bridge is safe enough to cross. We want your sincere opinion on this question: to
what extent do you agree or disagree with the following sentence: "JOHN KNOWS HIS TRUCK WILL MAKE IT ACROSS THE BRIDGE"?'
knowledge_question_first: 'No'
additional_question_text: "“Do you think that John should just cross the bridge.” For BRIDGE and \r\nAIR, subjects were\
\ not given a “Neutral” option in responding, it was just a \r\nbinary “Yes” and “No.”"
scenarios:
- scenario_code: bridge
scenario_type: Bridge probe; full low/high vignettes available in the appendix.
high_stakes_text: John drives a truck over a rickety bridge after hearing that two other trucks crossed safely. He is
unaware that dangerous explosive materials are in his cargo; if the bridge fails, the truck will explode and kill John
and many nearby people.
low_stakes_text: John drives a truck over a rickety bridge after hearing that two other trucks crossed safely. He is
unaware that eggs are in his cargo; if the bridge fails, the eggs will break and can be easily replaced.
provenance:
page: 48
quote: BRIDGE LOW... BRIDGE HIGH... LEVEL OF AGREEMENT QUESTION BRIDGE.
effects:
- effect_id: s5_e1
subgroup: Bridge — knowledge ascription agreement
subgroup_desc: Agreement with knowledge statement (Likert 0–4)
design: Between-Subjects
design_other: null
moderators:
scenario: bridge
skeptical_pressure: 'No'
awareness: 'No'
evidence: External
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 15
quote: We developed three pairs of vignettes (Coin, Air, and Bridge)
tei_id: null
table_ref: null
reason: Probe is explicitly labeled Bridge; coded as the bridge scenario.
skeptical_pressure:
provenance:
page: 15
quote: null
tei_id: null
table_ref: null
reason: no indication
awareness:
provenance:
page: 15
quote: But John is not at all aware that there are dangerous explosive materials delicately arranged in his cargo.
tei_id: null
table_ref: null
reason: Paper explicitly states protagonists are ignorant of the stakes in Study 2 vignettes.
evidence:
provenance:
page: 15
quote: He radios ahead to find out whether other trucks have made it across. He is told that the other two trucks
in the caravan made it over safely.
tei_id: null
table_ref: null
reason: talks to someone
attribution_person:
provenance:
page: 15
quote: "to what extent do you agree or disagree with the following sentence: \"JOHN KNOWS HIS TRUCK \r\nWILL MAKE\
\ IT ACROSS THE BRIDGE\"?"
tei_id: null
table_ref: null
reason: The DV is agreement with a third-person knowledge ascription.
evidence_reliability:
provenance:
page: 15
quote: "He radios ahead to find out whether other trucks \r\nhave made it across. He is told that the other two\
\ trucks in the caravan made it over safely."
tei_id: null
table_ref: null
reason: simple induction
contrast:
group_high: Bridge_high
group_low: Bridge_low
sign_convention: d = mean(low) - mean(high)
other_notes: Outcome is agreement with the knowledge statement; higher scores indicate more agreement.
groups:
- group_id: Bridge_low
label: null
n: 28
mean: 2.32
sd: 1.16
se: null
provenance:
page: 17
quote: Bridge (Likert 0–4) N 28, m 2.32, sd 1.16
tei_id: null
table_ref: Table 1.1
- group_id: Bridge_high
label: null
n: 31
mean: 1.71
sd: 1.13
se: null
provenance:
page: 17
quote: Bridge (Likert 0–4) N 31, m 1.71, sd 1.13
tei_id: null
table_ref: Table 1.1
reported_test:
test: t
t: 2.053
df1: 57.0
reported_d: 0.53
notes: p-value not reported in the extracted text for this t-test.
provenance:
page: 17
quote: Bridge vignettes t(57) 2.053, d 0.53 (medium effect size)
effect_size:
metric: SMD
d: 0.53307299757469
v: 0.0704650416404434
computed_from: groups
needs_review: false
notes: Computed from Table 1.1 group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups). Appendix material
located in out/external/pinillos IRI paper with shawn october 2014.pdf; Bridge vignette/prompt reviewed.
quality_flags: []
notes: null
notes: null
- study_id: 6
label: Modal-reading test (Typo; evidence-seeking; Knows vs Hopes)
language: English
language_other: null
objective: Test whether stakes effects on evidence-seeking prompts can be explained by a ‘modal reading’ by presenting Knows
and Hopes prompts side-by-side; includes a stakes contrast for the Knows question.
sample:
n_final: 70
recruitment: mTurk
recruitment_other: null
compensation: money
compensation_other: Assumed monetary compensation; amount not reported for Study 6.
characteristics: Amazon Turk workers in the United States assumed by continuity with the surrounding Pinillos & Simpson
2014 mTurk studies; Study 6 itself does not explicitly report participant recruitment or compensation. Stakes manipulated
via low vs high Typo vignette; Knows/Hopes prompts asked.
mean_age: null
provenance:
page: 26
quote: The mean for the Knows low-stakes response was 3.31 (N 39, SD 0.86). The mean for the Knows high-stakes response
was 4.42 (N 31, SD 1.36).
design: Between-Subjects
design_other: 'Mixed: stakes manipulated between-subjects (low vs high Typo vignette); Knows vs Hopes prompts asked to the
same participants (order manipulated).'
manipulated_factors:
- 'Question type: Knows vs Hopes (within-subjects)'
- Question order (between-subjects)
paradigm: Rating how much evidence is needed for knowledge
paradigm_other: null
scale:
label: numeric/text input
points: null
anchors: 'Numeric free response: number of proofreads required before the agent knows; whole number.'
direction: Higher numbers indicate more evidence needed for knowledge.
provenance:
page: 22
quote: “How many times do you think Peter has to proofread his paper before he knows that there are no typos? times.”
measures:
knowledge_question_text: How many times do you think Peter has to proofread his paper before he knows that there are no
typos?
knowledge_question_first: null
additional_question_text: How many times do you think Peter has to proofread his paper before he hopes that there are
no typos?
scenarios:
- scenario_code: typos
scenario_type: Proofreading for typos; stakes vary with consequences of a typo.
high_stakes_text: '(Typo-High): John, a good college student has just finished writing a two-page paper for an English
class. The paper is due tomorrow. Even though John is a pretty good speller, he has a dictionary with him that he can
use to check and make sure there are no typos. There is a lot at stake. The teacher is a stickler and guarantees that
no one will get an A for the paper if it has a typo. He demands perfection. John, however, finds himself in an unusual
circumstance. He needs an A for this paper to get an A in the class. And he needs an A in the class to keep his scholarship.
Without the scholarship, he can’t stay in school. Leaving college would be devastating for John and his family who have
sacrificed a lot to help John through school. So it turns out that it is extremely important for John that there are
no typos in this paper. And he is well aware of this.'
low_stakes_text: '(Typo-Low): Peter, a good college student has just finished writing a two-page paper for an English
class. The paper is due tomorrow. Even though Peter is a pretty good speller, he has a dictionary with him that he can
use to check and make sure there are no typos. But very little is at stake. The teacher is just asking for a rough draft
and it won’t matter if there are a few typos. Nonetheless Peter would like to have no typos at all.'
provenance:
page: 22
quote: To test Buckwalter and Schaffer's hypothesis, we ran an experiment where we presented a set of subjects with
the low-stakes Typo vignette and another set, the high-stakes Typo vignette.
effects:
- effect_id: s6_e1
subgroup: Typo — evidence-seeking knowledge (Knows question)
subgroup_desc: Number of proofreads required for knowledge
design: Between-Subjects
design_other: null
moderators:
scenario: typos
skeptical_pressure: 'No'
awareness: 'Yes'
evidence: First Person
attribution_person: Other
evidence_reliability: Medium
moderators_coding:
scenario:
provenance:
page: 22
quote: low stakes Typo vignette and another set, the high stakes Typo vignette
tei_id: null
table_ref: null
reason: The Study 6 text says the experiment reused the low- and high-stakes Typo vignettes; full wording is taken
from the local Pinillos 2012 extraction.
skeptical_pressure:
provenance:
page: 22
quote: How many times do you think Peter has to proofread his paper before he knows that there are no typos?
tei_id: null
table_ref: null
reason: No explicit skeptical challenge/alternative is presented; the manipulation concerns consequences of error.
awareness:
provenance:
page: 8
quote: And he is well aware of this.
tei_id: null
table_ref: null
reason: The reused Pinillos 2012 Typo-High vignette explicitly states that the protagonist is well aware of the high
stakes; Study 6 gives no indication that it used the ignorant high-stakes variant.
evidence:
provenance:
page: 8
quote: Even though Peter is a pretty good speller, he has a dictionary with him that he can use to check and make
sure there are no typos.
tei_id: null
table_ref: null
reason: Evidence is generated by the agent’s own proofreading/checking with a dictionary (first-person evidence);
full wording is taken from the local Pinillos 2012 extraction.
attribution_person:
provenance:
page: 22
quote: before he knows that there are no typos
tei_id: null
table_ref: null
reason: Participants attribute knowledge to Peter (third-person attribution).
evidence_reliability:
provenance:
page: 8
quote: pretty good speller... has a dictionary with him that he can use to check and make sure there are no typos
tei_id: null
table_ref: null
reason: The reused Pinillos 2012 Typo vignette describes a good speller with a dictionary, but the evidence-seeking
task asks how many proofreads are required, so reliability remains coded Medium as in the Pinillos 2012 extraction.
contrast:
group_high: Typo_high
group_low: Typo_low
sign_convention: d = mean(low) - mean(high)
other_notes: Outcome is number of proofreads required; higher scores indicate more evidence needed for knowledge.
groups:
- group_id: Typo_low
label: null
n: 39
mean: 3.31
sd: 0.86
se: null
provenance:
page: 26
quote: The mean for the Knows low-stakes response was 3.31 (N 39, SD 0.86).
tei_id: null
table_ref: null
- group_id: Typo_high
label: null
n: 31
mean: 4.42
sd: 1.36
se: null
provenance:
page: 26
quote: The mean for the Knows high-stakes response was 4.42 (N 31, SD 1.36).
tei_id: null
table_ref: null
reported_test:
test: t
t: 3.95
df1: 48.35
notes: Reported as p < 0.01.
provenance:
page: 26
quote: 'The difference was statistically significant: t(48.35) 3.95, p 0.01.'
effect_size:
metric: SMD
d: -1.0011358164739
v: 0.0652687440029404
computed_from: groups
needs_review: false
notes: Computed from group means/SDs/Ns in analysis/effect_sizes.qmd (method=between_groups). Review resolved by using
the local Pinillos 2012 extraction for the reused Typo vignette wording; participant recruitment/compensation coded
as mTurk/money by continuity with the surrounding Pinillos & Simpson 2014 studies.
quality_flags: []
notes: Study 6 says it presented the low-stakes and high-stakes Typo vignettes. Full Typo wording is not in the located
Pinillos & Simpson appendix; it is taken from the local Pinillos 2012 extraction. There is no indication that Study 6
used the ignorant high-stakes variant.
notes: Full Typo vignette wording not provided in the located Pinillos & Simpson appendix; coded from the local Pinillos
2012 extraction because Study 6 explicitly reused the low-stakes and high-stakes Typo vignettes. Recruitment/compensation
coded as mTurk/money by continuity with the surrounding Pinillos & Simpson 2014 studies; amount not reported.