/data/papers/turrietalndactionabilityjudgmentscause/out/fulltext.mdActionability Judgments Cause Knowledge Judgments
John Turri Wesley Buckwalter David Rose
University of Waterloo, University of Waterloo, Rutgers University
Published on February 10, 2005
Researchers recently demonstrated a strong direct relationship between judgments about what a person knows ("knowledge judgments") and judgments about how a person should act ("actionability judgments"). But it remains unknown whether actionability judgments cause knowledge judgments, or knowledge judgments cause actionability judgments. This paper uses causal modeling to help answer this question. Across two experiments, we found evidence that actionability judgments cause knowledge judgments.
Introduction
A foundational debate in philosophy concerns the relationship between knowledge and action. Rene Descartes claimed that knowledge is fundamentally separable from action and other practical concerns. Descartes cited this assumption in defense of his infamous method of hyperbolic doubt, introduced in the First Meditation (Descartes, 1641). John Locke responded to professed Cartesian doubt by rejecting the assumption that knowledge is fundamentally separable from action. Locke claimed that knowledge was connected to whether you have "assurance enough" to "govern [your] actions" (Locke 1690, book 4.11.8). William James went one step further than Locke. James claimed that knowledge is partly constituted by "practical interests" and that it was essentially "a guide to appropriate action" (James 1879, p. 18). The disagreement between Descartes and Locke was forgotten for most of the 20th century in Anglo-American philosophy, but it has recently been renewed, with many epistemologists adopting Cartesian positions Correspondence to: E-mail: john.turri@gmail.com (e.g., Fumerton 2010;Reed 2013) and others defending Lockean positions (e.g., Fantl andMcGrath 2002, 2009;Hawthorne 2004;Hawthorne and Stanley 2008;Stanley 2005).
In his response to Descartes, Locke suggested that observations about how we talk and act are relevant to a correct theoretical understanding of the relationship between knowledge and practice (Locke 1690(Locke /1975, bk. 4.11.3-8), bk. 4.11.3-8). The emphasis on behavioral evidence plays a crucial role in contemporary defenses of Lockeanism about knowledge (e.g., Fantl and McGrath 2007, p. 562;Hawthorne and Stanley 2008, p. 571). In particular, latter-day Lockeans claim that ordinary knowledge attributions are sensitive to practical factors, such as an increase in stakes or a change in how you should act. Cartesians have largely conceded Lockean claims about our behavior, but they propose that the sensitivity is entirely indirect (e.g., Bach 2005;Ganson 2007;Nagel 2008;Weatherson 2005). More specifically, they argue that practical factors affect knowledge attributions only by influencing our estimation of the factors that, according to Cartesians, constitute knowledge: belief, truth, and quality of evidence.
Until recently, the philosophical debate has relied mainly on introspection and social observation to evaluate the behavioral claims, which is typical in Anglophone analytic philosophy (for reviews, see Turri 2016aTurri , 2016bTurri , 2016c)). But introspection and social observation are subject to well-known limitations (e.g., Lieberman 2013;Milgram 1974, pp. 103-4;Nisbett and Wilson 1977;Wilson 2002), and early experimental attempts to validate the behavioral claims met with mixed results (e.g., Buckwalter 2010;Feltz and Zarpentine 2010;May et al. 2010;Pinillos 2012;Sripada and Stanley 2012). As two commentators recently put it, the evidence from these early attempts "made it easy" to conclude that much of the theoretical debate "was founded on a myth" (Schaffer and Knobe 2012, p. 675). Equally worrisome, moreover, is the fact that Cartesians and Lockeans propose competing psychological models of knowledge judgments that introspection and social observation cannot adjudicate. Even supposing that judgments about how someone should act ("actionability judgments") are connected to knowledge judgments, we cannot introspect or directly observe whether this relationship is direct, as latter-day Lockeans claim, or indirect, as latter-day Cartesians claim.
A recent study addressed this issue (Turri and Buckwalter in press). Using regression and mediation analysis, researchers found a strong direct relationship between actionability judgments and knowledge judgments. Indeed, this relationship was stronger than the relationship between knowledge judgments and evidence judgments, and it was approximately as strong as the relationship between knowledge judgments and truth judgments. These findings rule out Cartesian mediation models and fit very well with Lockean models. But they leave an important question unanswered. Given that there is a strong direct relationship between actionability judgments and knowledge judgments, what is the nature of that link? Which judgment, if either, has priority? Do knowledge judgments cause actionability judgments, do actionability judgments cause knowledge judgments, or do both have a common cause?
This paper reports two experiments designed to answer this question. The experiments are modeled after earlier work that ruled out Cartesian models. But they go beyond earlier work in two crucial ways.
First, they are more closely informed by relevant work in the judgment and decision-making literature. In particular, they are informed by important work that identifies three separate factors that contribute to the perception of heightened stakes (Beach and Mitchell 1978;McAllister, Mitchell, and Beach 1979). The three factors are irreversibility, significance, and accountability. An agent's conduct is irreversible to the extent that it cannot be undone to remove its negative consequences. The more irreversible conduct is, the more likely people are to perceive the situation as high stakes. An agent's conduct is significant to the extent that its likely consequences are extremely good, extremely bad, or have an extremely broad impact. The more significant conduct is, the more likely people are to perceive the situation as high stakes. An agent is accountable for conduct to the extent that she is responsible for its consequences. She will be praised or blamed, rewarded or punished, depending on whether the outcome is positive or negative. The more accountable an agent is for her conduct, the more likely people are to perceive the situation as high stakes. In the experiments that follow, a "high stakes" case is one that is high on all three dimensions of stakes, and a "low stakes" case is one that is low on all three dimensions.
Second, the present research uses a causal search with the Greedy Equivalence Search (GES) algorithm to identify the model that best fits the data. The causal search was conducted with Tetrad 5.0.0 (http://www.phil.cmu.edu/projects/tetrad/). GES considers all possible models available given the different variables. Each variable is treated as a node. GES assigns an information score to the model in which all the nodes are disconnected-the "null model." GES then evaluates adding causal arrows-"edges"-between the nodes (Meek 1997 provides the edge orientation rules). GES adds edges that best improve the model's information score, if such edges exist, until adding more edges does not improve the information score. At this point, GES evaluates whether deleting any edges will further improve the information score, and it deletes any such edges until deleting more edges does not improve the information score. In this way, GES helps us to identify the underlying causal structure among a set of variables, going beyond what regression and mediation analyses can offer. Regression assumes a causal direction, whereas GES does not, and GES is preferable to mediation models because it provides an overall measure of model fit and tends to deliver more accurate models (Iacobucci, Saldanha, and Deng 2007). GES returns the causal model that best fits the data and, given enough data, will return the true causal model (Chickering 2002). (For other applications of GES to theoretical debates, see Rose and Nichols 2013;Rose et al. 2012;Turri 2016d).
Experiment 1
Method
Participants
Two hundred participants (aged 18-68 years, mean age = 31 years; 94% reporting English as a native language; 80 females) were tested. Participants were recruited and tested online using Amazon Mechanical Turk and Qualtrics and compensated $0.40 for approximately 2-3 minutes of their time. Repeat participation was prevented.
Materials and procedure
Participants were randomly assigned to one of two conditions, Low and High, in a between-subjects design. Each participant read a single story about an intelligence agent developing a file on a foreign operative, Ivan. Participants in the Low condition read a story that featured low irreversibility, significance, and accountability. Participants in the High condition read a story that featured high irreversibility, significance, and accountability. The stories for the two conditions thus differed on all three crucial dimensions of stakes identified in the judgment and decision-making literature. Here is the text for the stories, with manipulations in brackets and separated by a slash (Low/High):
Jennifer is an intelligence analyst developing a file on Ivan, an elusive foreign operative. Jennifer has a source who tells her that Ivan stopped [his low-carb diet/selling arms to terrorists] and is no longer [jogging regularly/a threat]. 1 Jennifer must submit a [provisional/final] report on Ivan to her supervisor within the hour. She will [definitely/definitely not] have a chance to revise her [provisional/final] report, and she [will not/will] be held accountable for decisions based on her [provisional/final] report.
After reading the story, participants were asked to rate their agreement or disagreement with eight statements.
1 If Jennifer changes her mind, she can revise her report later. 2 If Ivan still [jogs regularly/is a threat], there will be serious consequences. 3 Jennifer is responsible for decisions made based on her [provisional/final] report. 4 Jennifer thinks that Ivan no longer [jogs regularly/is a threat]. 5 It's true that Ivan no longer [jogs regularly/is a threat]. 6 Jennifer has good evidence that Ivan no longer [jogs regularly/is a threat]. 7 Jennifer should write in the report that Ivan no longer [jogs regularly/is a threat]. 8 Jennifer knows that Ivan no longer [jogs regularly/is a threat].
Statements 1-3 are manipulation checks on the three critical dimensions of stakes: irreversibility, significance, and accountability. Statements 4-8 are the key dependent variables; responses to these statements will count as the participant's belief score, truth score, evidence score, actionability score, and knowledge score, respectively.
Responses were collected on a standard 7-point Likert scale anchored with "strongly disagree," "disagree," "somewhat disagree," "neutral," "somewhat agree," "agree," and "strongly agree," left-to-right on the participant's screen. Responses were coded 1 (strongly disagree) to 7 (strongly agree). Participants never saw the numerical values, only the qualitative anchors. The eight statements were presented in random order and appeared on the participant's screen all at once, while the story remained at the top of the screen. Response options were always presented in the same order. After rating the statements, participants proceeded to a new screen where they completed a brief demographic questionnaire.
Results
Preliminary analysis revealed no main or interaction effects of participant age and gender on any of the eight dependent variables. Preliminary analysis also revealed that the stakes manipulation was extremely effective in all three dimensions (see Table 1). The independent variable stakes (Low/High) and response to the dependent measures were entered into a causal search using the GES algorithm in Tetrad V (see the Introduction section). The model was constrained so that the independent variable stakes could not be caused by any other variable in the model (reflecting the experimental design), and the penalty discount was set to 0.5 (decided a priori). Inspection of normal probability plots revealed no violation of normality assumptions. In line with previous research using similar methodology (Turri and Buckwalter in press), preliminary regression analysis revealed that belief scores did not predict either knowledge scores or actionability scores. GES also revealed that including belief scores resulted in a poor fitting model, so we omitted them. 2 Figure 1 depicts the best fitting model, which fits the data well, χ 2 (3) = 3.44, p = .33, BIC = -12.45. In the model, actionability judgments cause knowledge judgments. Indeed, actionability judgments alone directly cause knowledge judgments, and this is the strongest causal relationship between any two dependent variables in the model.
As a point of comparison, we conducted the causal search in the exact same way except that it forced an edge from knowledge judgments to actionability judgments. This model was rejected as a poor fit for the data, χ 2 (5) = 44.75, p = 0, BIC = 18.27.
Discussion
Our main question was to identity the direction of the relationship between knowledge judgments and actionability judgments. In the present study, actionability judgments caused knowledge judgments and this was the strongest causal relationship between any two dependent variables. The next experiment investigates the robustness of this finding by testing a different cover story and probing for attributions of knowledge and actionability only.
Experiment 2
Method
Participants
Two hundred and five participants (aged 18-72 years, mean age = 32 years; 96% reporting English as a native language; 101 females) were tested. Participants were recruited and tested the same way as in Experiment 1. Repeat participation was prevented.
Materials and procedure
Participants were randomly assigned to one of two conditions, Low and High, in a between-subjects design. Each participant read a single story about an employee charged with updating the menu for a chain of coffee shops. As in Experiment 1, the stories for the two conditions thus differed on all three crucial dimensions of stakes identified in the judgment and decision-making literature. Here is the text for the stories, with manipulations in brackets and separated by a slash (Low/High):
Christina is in charge of updating the menu for a chain of coffee shops each day. To some customers [interested in the history and culture of coffee/with severe nut and she [will not/will] be held accountable for the accuracy of the menu.
After reading the story, participants were asked to rate their agreement or disagreement with five statements. 1 Christina will be held accountable for the accuracy of the menu. 2 Christina will have a chance to revise the menu. 3 If the coffee [is from northern Colombia/contains pine nuts], there could be serious consequences. 4 Christina should write on the menu that the coffee [is from northern Colombia/contains pine nuts]. 5 Christina knows that the coffee [is from northern Colombia/contains pine nuts].
Statements 1-3 are manipulation checks on the three critical dimensions of stakes. Statements 4 and 5 are the key dependent variables of knowledge and actionability. Responses were collected and scored the same way as in Experiment 1, including randomized order for all statements.
Results
Preliminary analysis revealed no main or interaction effects of participant age and gender on any of the dependent variables. Preliminary analysis also revealed that the stakes manipulation was extremely effective in all three dimensions (see Table 2).
We conducted a causal search on the data in the same way as Experiment 1. Figure 2 depicts the best fitting model, which fits the data well, χ 2 (1) = 2.68, p = .11, BIC = -2.64. In the model, actionability judgments cause knowledge judgments.
As a point of comparison, an alternative model in which stakes cause knowledge judgments and, in turn, knowledge judgments cause actionability judgments was rejected as a poor fit for the data, χ 2 (1) = 22.56, p < .00001, BIC = 17.24.
Discussion
The main finding from Experiment 1 was that, in the best fitting causal model, actionability judgments cause knowledge judgments. The present experiment replicated that finding and generalized it to other narrative contexts. It's also worth noting two further points. First, in Experiment 1 higher stakes caused lower actionability scores, whereas in the present experiment higher stakes caused higher actionability scores. This could be because the scenarios for Experiments 1 and 2 differ in whether the riskier option is to act or not act (for related results on knowledge judgments and actionability judgments, see Turri, Friedman, and Keefner in press). Second, in Experiment 1, the stakes manipulation caused (indirectly) a significant difference in knowledge judgments, whereas in the present experiment, stakes did not affect knowledge judgments. Despite these dissimilarities, actionability judgments caused knowledge judgments in both experiments.
Conclusion
Recent work has demonstrated a strong direct relationship between judgments about what someone knows ("knowledge judgments") and how they should act ("actionability judgments"). This result answered a longstanding question in philosophy about whether the relationship between these judgments is direct or instead mediated by other judgments about truth, belief, and evidence. But it left an important question unanswered: do actionability judgments cause knowledge judgments, or do knowledge judgments cause actionability judgments? The present research was designed to help answer this question.
Across two experiments, we found evidence that judgments about how someone should act cause judgments about whether they know. We observed this same basic result in different narrative contexts, when higher stakes caused lower actionability judgments, and when higher stakes caused higher actionability judgments. These results do not show that actionability judgments always cause knowledge judgments, or that knowledge judgments never cause actionability judgments. Further work is required to test how broadly we observe the same basic causal pattern reported above. Nevertheless, the results do provide evidence that in a range of perfectly ordinary cases, actionability judgments cause knowledge judgments.
One assumption of our causal search analyses is that our model does not omit an important latent variable, which might be a common cause of both actionability judgments and knowledge judgments. In Experiment 1, we measured a suite of judgments that theorists commonly assume inform judgments of knowledge or actionability. Our analysis suggests that even when accounting for these other judgments, actionability judgments cause knowledge judgments. Nevertheless, we cannot rule out the theoretical possibility that an unmeasured common cause drove some of the findings. Future work could aim to identify and measure such variables and test whether findings similar to ours persist in that context.
It is worth noting that the best fitting model for Experiment 1 contains some other potentially very interesting features. For instance, according to this model, the first judgment people make concerns actionability, which in turn influences a host of other judgments, including judgments of knowledge, evidence, and truth. Although this pattern is not predicted by any contemporary philosophical theory we are aware of, it was predicted by William James. Wrote James, "The germinal question concerning things brought for the first time before consciousness is … the practical 'What is to be done?'" (James 1879, p. 18). To take another example, belief attributions did not affect knowledge judgments, whereas a mainstream view in epistemology is that belief is an "ingredient" of knowledge. Nevertheless, even if this result does not cohere with mainstream philosophical assumptions, it fits well with recent research on the psychology of knowledge attributions. This research found that people often attribute knowledge while denying belief, or do not base knowledge attributions on belief attributions (e.g., Murray, Sytsma, and Livengood 2012;Myers-Schulz and Schwitzgebel 2013;Turri and Buckwalter in press).
Acknowledgments
For helpful feedback, we thank Angelo Turri and this journal's anonymous referees. This research was supported by the Social Sciences and Humanities Research Council of Canada, the Ontario Ministry of Economic Development and Innovation, a Banting Postdoctoral Fellowship, and the Canada Research Chairs program.
Banting Postdoctoral Fellowship
Note
1 Indicates a paragraph break on the participant's screen. 2 It is perhaps worth noting that the best fitting model that included belief featured an edge from actionability judgments to knowledge judgments. That is, including belief resulted in the same causal relationship between actionability and knowledge judgments reported below.
References
[1] The Emperor's New 'Knows K Bach (2005). [2] A Contingency Model for the Selection of Decision Strategies L Beach and T Mitchell Academy of Management Review (1978) 3 (3). [3] Knowledge Isn't Closed on Saturday: A Study in Ordinary Language W Buckwalter Review of Philosophy and Psychology (2010) 1 (3). [4] Optimal structure identification with greedy search D Chickering Journal of Machine Learning Research (2002) 3 (1). [5] Meditations on first philosophy R Descartes (2006). [6] Advice for Fallibilists: Put Knowledge to Work J Fantl and M Mcgrath Philosophy and Phenomenological Research (2002) 111 (1). [7] Do You Know More When It Matters Less? A Feltz and C Zarpentine Philosophical Psychology (2010) 23 (5). [8] Fencing Out Pragmatic Encroachment R Fumerton Philosophical Perspectives (2010) 24. [9] Evidentialism and Pragmatic Constraints on Outright Belief D Ganson Philosophical Studies (2007) 139 (3). [10] Knowledge and Lotteries J Hawthorne (2004). [11] Knowledge and Action J Hawthorne and J Stanley Journal of Philosophy (2008) 105 (10) pp. 571. [12] A Meditation on Mediation: Evidence That Structural Equations Models Perform Better Than Regressions D Iacobucci et al. Journal of Consumer Psychology (2007) 17 (2). [13] The Sentiment of Rationality W James (1879). [14] Social: Why Our Brains Are Wired to Connect M Lieberman (2013). [15] An Essay Concerning Human Understanding J Locke (1975). [16] Practical Interests, Relevant Alternatives, and Knowledge Attributions: An Empirical Study J May et al. Review of Philosophy and Psychology (2010) 1 (2). [17] The Contingency Model for the Selection of Decision Strategies: An Empirical Test of the Effects of Significance, Accountability, and Reversibility D Mcallister et al. Organizational Behavior and Human Performance (1979) 24. [18] Graphical models: selecting causal and statistical models C Meek (1997). [19] Obedience to Authority: An Experimental View S Milgram (1974). [20] God Knows (But Does God Believe?) D Murray et al. Philosophical Studies (2012) 166 (1). [21] Knowing That P Without Believing That P B Myers-Schulz and E Schwitzgebel Noûs (2013) 47 (2). [22] Knowledge Ascriptions and the Psychological Consequences of Changing Stakes J Nagel Australasian Journal of Philosophy (2008) 86 (2). [23] Telling More Than We Can Know: Verbal Reports on Mental Processes R Nisbett and T Wilson Psychological Review (1977) 84 (3) pp. 231. [24] Knowledge, Experiments and Practical Interests N Pinillos (2012). [25] Practical Matters Do Not Affect Whether You Know B Reed (2013). [26] The Lesson of Bypassing D Rose and S Nichols Review of Philosophy and Psychology (2013) 4. [27] Deep Trouble for the Deep Self D Rose et al. Philosophical Psychology (2012) 25 (5). [28] Contrastive Knowledge Surveyed J Schaffer and J Knobe Noûs (2012) 46 (4). [29] Empirical tests of interest-relative invariantism C Sripada and J Stanley Episteme (2012) 9 (1). [30] Knowledge and Practical Interests J Stanley (2005). [31] How To Do Better: Toward Normalizing Experimentation in Epistemology J Turri (2016) 41 https://doi.org/10.1016/j.concog.2016.01.004. [32] Descartes's Schism, Locke's Reunion: Completing the Pragmatic Turn in Epistemology J Turri and W Buckwalter American Philosophical Quarterly. [33] Knowledge Central: A Central Role for Knowledge Attributions in Social Evaluations J Turri et al. Quarterly Journal of Experimental Psychology. [34] Can se Do Without Pragmatic Encroachment? B Weatherson Philosophical Perspectives (2005) 19 (1). [35] Strangers to Ourselves: Discovering the Adaptive Unconscious T Wilson (2002).