News Quality and Public Opinion: The Impact of Deliberative Quality of News Media on Citizens' Argument Repertoire

News Quality and Public Opinion: The Impact of Deliberative Quality of News Media on Citizens'... Abstract This article investigates the impact of the deliberative quality of online news items on argument repertoire. Participants in an online experiment (N = 557) read a news item that covers a local debate on underground CO2 storage. Two dimensions of deliberative quality are manipulated: justification, or the extent to which arguments in the news item are substantiated, and constructive debate climate, that is, the extent to which stakeholders are reported to work together toward a solution. The findings suggest that public opinion quality would be higher, if news media would better meet deliberative standards. Normative theories expect that citizens are able to express informed opinions on important policy issues. Empirical research, however, offers compelling evidence to the contrary (see Druckman, 2014). Two types of explanations have been advanced. First, citizens might not be able or willing to digest available political information. Second, news media might not adequately perform their role as crucial sources of political information (e.g., Iyengar, Hahn, Bonfadelli, & Marr, 2009). In relation, scholars have argued that normative expectations for public opinion are simply too ambitious (e.g., Graber, 2003). In this article, we take a complementary angle. Rather than contributing to an already extensive body of empirical research on opinion quality as it is, we explore whether it is feasible to improve public opinion quality. As political communication scholars, we focus on the potential contribution of news media. We draw inspiration from deliberative theory and build conceptually on cognitive models of learning from news media. Using an experimental design, we address the question whether quality of public opinion would be higher, if news media would adhere better to normative demands of deliberative theory. Argument Repertoire as Indicator for Opinion Quality Public opinion quality is a multifaceted and complex concept. Druckman (2014) identifies four criteria that are discussed in the literature. Opinions should be informed, internally consistent, coherent with a person's deep values, and not easily swayed by framing efforts. Price and Neijens (1997) even distinguish four sets of quality criteria. These concern public opinion at the individual and the collective level, and address the quality of opinions themselves as well as the process of opinion formation (Price & Neijens, 1997, p. 346). In this article, we have chosen “argument repertoire” (Cappella, Price, & Nir., 2002) as appropriate indicator of opinion quality. Argument repertoire refers to people's knowledge of pro-attitudinal arguments (in favor of their own opinion) and counter-attitudinal arguments (that support the positions of opponents). It is a reliable and valid cognitive measure that gauges the depth of knowledge of citizens on a particular issue (Cappella et al., 2002). As quality criterion, it aligns with the long-standing conviction in public opinion research that opinion quality depends on “the amount of information that supports an opinion” (Price & Neijens, 1997, p. 346; also Druckman, 2014). More specifically, and especially relevant to our study, argument repertoire aligns with the normative demand from deliberative theory (discussed below) that quality opinions are based on the consideration of diverse ideas and views (Cappella et al., 2002; Mutz & Martin, 2001). Of course, knowledge of arguments does not necessarily mean that these are considered in-depth, and values, emotions, and engagement are important components of opinion quality, too (Price & Neijens, 1997). But for a first assessment of the impact of the deliberative quality of news items on opinion quality, argument repertoire presents itself as a relevant, reliable, and valid choice (see also Manosevitch, 2009). Deliberative Criteria for News Media Quality The deliberative turn in democratic theory has made deliberativeness an important ideal for democracy and news media. At the core, we find the conviction that all stakeholders should discuss as many different ideas as possible in a civil and reasoned debate, to come up with commonly agreed solutions for social problems (Burkhalter, Gastil, & Kelshaw, 2002; Ryfe, 2005; Wessler, 2008). Theorists expect, and some empirical studies confirm, that deliberative processes contribute to the quality of public opinion and policy-making (Delli Carpini, Cook, & Jacobs, 2004). For example, participants in deliberative polls—quasi-experimental events, lasting a few days, where citizens discuss a specific policy issue with experts and politicians—have been found to express more informed and reasoned opinions (Andersen & Hansen, 2007; Luskin, Fishkin, & Jowell, 2002). Research also shows that people that are exposed to disagreement in interpersonal political debates can better provide arguments in favor and against their own opinions (Price, Cappella, & Nir, 2002). In modern societies, however, genuine face-to-face deliberation between many stakeholders is not really feasible. Public deliberation instead takes place in mediated public spheres. Scholars have accordingly proposed that (these debates in) news media should meet the standards of deliberativeness, too (Wessler, 2008). The extent to which media meet these criteria varies across situations. The extent to which mediated deliberation subsequently contributes to the quality of opinions and decision-making in society is hardly investigated (see below). Criteria for Deliberativeness Public deliberation, whether face-to-face or in the media, should meet three important criteria (Burkhalter et al., 2002; Stromer-Galley, 2007; Wessler, 2008). The first is diversity: All those affected by an issue should consider all relevant ideas. Next, these different ideas should be justified and discussed in a rational way. Arguments should be based on accurate information, justifications for claims are to be given and demanded, and opposing arguments should be considered on substantial grounds only. Third, deliberation requires a shared, constructive debate climate. Participants should respect each other, listen to diverging views, make an effort to work together, look for shared solutions and the “common good,” and make difficult choices that transcend particular self-interests. A large argument repertoire, a firm grasp of the diverging arguments that are at play in an issue, would be the logical outcome of such deliberation (Cappella et al., 2002). Empirical research suggests that news media tend to adhere to the first criterion of deliberativeness. News media tend to provide citizens, at least to some extent and in comparison with interpersonal networks, with diverse issue information (Graber, 2003; Mutz & Martin, 2001). Research also shows that there are wide variations in the levels of diversity and justification in news media across countries, time periods, and news media (Van der Wurff, Verhoeven, & Gadellaa, 2013; Wessler & Rinke, 2014). Whether news media present policy debates as constructive, is not investigated (for an exception, see Cottle, 2006). The large body of literature on conflict framing, though, seems to point in a different direction (Schuck et al., 2013). Turning to the impact of these characteristics, research suggests that diversity of news media content contributes to public opinion quality. Diversity, or more precisely, competitive framing, motivates people to be accurate and to process information in-depth (Chong & Druckman, 2007a). It reduces the (manipulative) impact of single-issue frames on audience perceptions (Aday, 2006; Chong & Druckman, 2007b; Druckman, 2001) and leads people to take positions that better fit their “true” beliefs and values (Chong & Druckman, 2007a). The impact of justification and a constructive debate climate in news media on public opinion quality has not been investigated. Our study attempts to start filling this gap. Our starting point is the idea that news media can act as a model (Wessler, 2008). Perhaps, news media can stimulate individual citizens to deliberate “from within” (Goodin, 2000)—to conduct imaginative debates, to try to understand the viewpoints and arguments from opponents, and to express one's own reasons in return—even though citizens tend to be observers rather than participants in mediated debates. Such internal deliberation will increase awareness of pro- and counter-attitudinal arguments and hence translate into a larger argument repertoire. To better understand whether and how justification and a constructive debate climate, as deliberative criteria for news media quality, can influence argument repertoire, we turn to cognitive models of learning from news media. Learning from News Media News media are important sources of political information for citizens. How much citizens learn from these news media depends on the interplay between (a) the quality and quantity of news media content and (b) the willingness and ability of citizens to digest that information (Luskin, 1990). Along these lines, we follow Wessler (2008, p. 1) in arguing that deliberative news media can stimulate public opinion quality in two ways: by offering “diverging justifications for political positions” (i.e., relevant content) and by “model[ing] deliberative behavior in audiences” (i.e., triggering audiences to deliberate internally). News Media Content Comparative research shows that differences in news media content—across media types and/or media systems—matter for political knowledge. Some news outlets, like broadsheets and public TV news, pay more attention to hard news than others (tabloids, commercial TV), and thereby enable citizens to acquire more knowledge (Curran, Iyengar, Brink Lund, & Salovaara-Moring, 2009; Fraile & Iyengar, 2014). Moreover, in media systems where political or hard news is more abundant, or citizens have fewer options to choose less-informative news outlets, people learn about politics despite differences in attention. Gaps in political knowledge are consequently smaller (Elenbaas, de Vreese, Schuck, & Boomgaarden, 2014; Iyengar et al., 2009). The cited studies focus on factual knowledge. We assume, as tentatively suggested by David (2009), that the same reasoning applies to knowledge of arguments and hence argument repertoire. News media use has indeed been found to increase knowledge of arguments (Kim, Scheufele, Shanahan, & Choi, 2011). Accordingly, we suggest that when news media meet the criterion of justification (i.e., when news media provide more in-depth reasons for issue positions), users are fostered to learn these arguments and develop larger argument repertoires. We propose to refer to this effect as the informative contribution of news media to public opinion quality. In brief: H1a. Justification of arguments in news media increases argument repertoire. Elaboration of News Media Content How much people learn from available news media depends on cognitive behavior, in particular the level of elaboration of news media content (Beaudoin & Thorson, 2004; Eveland, 2001; Jensen, 2011). Eveland (2001, p. 573) defines elaboration as “the process of connecting new information to other information stored in memory, including prior knowledge, personal experiences, or the connection of two new bits of information together in new ways.” In the elaboration likelihood model (ELM), this is referred to as the “central route”; in cognitive psychology it is known as controlled processing. These concepts reference intentional types of information processing—spending cognitive effort and activating prior cognitions to carefully examine information and arguments. The opposite is known as automatic processing, or (in ELM) as the peripheral route. This involves the superficial processing of information, guided by prior opinions, cues, and heuristics (Petty, Briñol, & Priester, 2008; Shiffrin & Schneider, 1977). Elaboration and controlled processing are predicted to foster learning and long-term attitude change. Instead, automatic processing causes at best incidental learning and superficial changes in attitudes (Petty et al., 2008; Shiffrin & Schneider, 1977). Research confirms that elaboration stimulates learning (Fleming & Thorson, 2008). Accordingly, we predict that users who elaborate news media content, learn more arguments: H1b. Elaboration of news media content by news users increases argument repertoire. Media-Related Antecedents of Elaboration Generally speaking, humans tend to follow the peripheral route when processing information from news media (Druckman & Bolsen, 2011; Hart & Nisbet, 2012). It is therefore important to know whether and how citizens can be motivated to elaborate. Cognitive scholars tend to refer in this respect to personal characteristics (see below). We, however, are first of all interested in whether news media can stimulate elaboration. A few authors point in this direction. As already mentioned, Wessler (2008) suggests that deliberative news media may model deliberation among audiences. Manosevitch (2009) shows more specifically that news media can stimulate elaborate processing by providing “reflective” cues—gentle reminders to the audience about the importance of “thinking about issues.” ELM likewise suggests that news media can stimulate controlled processing by underlining the personal relevance of an issue, or by referencing multiple sources (Petty et al., 2008). Providing diverse or conflicting views in a news item, too, motivates people to be accurate and to process information more in-depth (Chong & Druckman, 2007a; Ryfe, 2005). Reviewing these arguments, we propose that justification in news media triggers citizens to elaborate. If participants in a mediated debate provide reasons and evidence for their views, users might follow their lead and consider an issue more in-depth, too—because news media provide a model, cue elaboration, and/or provide more complex information that triggers in-depth processing. Hence, we propose that justification has an activating effect: H1c. Justification in news media stimulates elaboration by news users. Personal Antecedents of Elaboration Personal characteristics are important determinants of elaboration, too. ELM proposes that people are more motivated to elaborate—and consequently learn more—when messages are perceived to have personal relevance (Petty et al., 2008). In political communication research, personal relevance tends to be discussed in terms of political or issue interest. Citizens who are interested in politics (or a specific issue) consider politics (or that issue) as relevant to them. Political interest is even equated with the internal motivation to learn about politics (Elenbaas et al., 2014). Accordingly, there is strong evidence that interest supports learning (Beaudoin, 2008; Elenbaas et al., 2014; Luskin, 1990). A second personal antecedent of elaboration is need for cognition (Petty et al., 2008; Ho, Peh, & Soh, 2013). Need for cognition is a “dispositional determinant” of central processing (Cacioppo & Petty, 1982, p. 130). It makes people structurally more likely to elaborate. People with a high need for cognition enjoy thinking. They engage spontaneously in effortful cognitive activity. Reviewing these arguments, we predict that issue interest and need for cognition moderate the impact of justification on elaboration. Justification triggers elaboration more strongly, when users are more interested in an issue, have a higher need for cognition, or both: H1d. Justification stimulates elaboration more strongly for news users with a higher issue interest. H1e. Justification stimulates elaboration more strongly for news users with a higher need for cognition. Justification and Argument Repertoire These hypotheses together result in our first model (Figure 1). Justification, the extent to which news media offer substantive reasons for issue positions, is predicted to have two effects on argument repertoire. The first is a direct, informative effect (H1a). Justification increases argument repertoire by providing more in-depth information. The second is indirect. Justification activates elaboration of news media content (H1c), and elaboration causes news users to develop a larger argument repertoire (H1b). The activating impact of justification will be stronger for people who for personal reasons are already more likely to elaborate, that is, for news users with a higher interest in the issue (H1d) or a higher need for cognition (H1e). Figure 1 View largeDownload slide Justification and argument repertoire Figure 1 View largeDownload slide Justification and argument repertoire Considering Opposing Arguments Deliberation involves the consideration of opposing arguments on substantive grounds, with the aim of reaching—if possible—common understanding (Burkhalter et al., 2002). Citizens should, therefore, be aware of various arguments. Their argument repertoire should encompass both pro- and counter-attitudinal arguments. Cognitive models and the literature on selective exposure suggest that the consideration of both pro- and counter-attitudinal arguments is associated with the central processing of information (Holbert, Garrett, & Gleason, 2010; Winter & Krämer, 2012). Elaboration is necessary to deal with the (perceived) complexity of counter-attitudinal information, and controlled effort is required to handle the negative emotions, feelings of dissonance, and threats to one's self-image that arise when one carefully considers arguments that go against one's standpoint (Fischer, Greitemeyer, & Frey, 2008). Automatic processing, in contrast, is associated with selective exposure and the (unconscious) exclusion of counter-attitudinal arguments (Fischer et al., 2008; Hart & Nisbet, 2012; Taber & Lodge, 2006). Elaboration Driven by Accuracy or Partisan Goals Elaboration, however, is not sufficient to foster the even-handed consideration of both pro- and counter-attitudinal arguments. What is equally important, is whether elaboration is driven by “accuracy” or “partisan” goals (Kunda, 1990; Taber & Lodge, 2006). Accuracy goals dominate when people need to make important decisions, or when they expect to be asked to justify their decisions later (Kunda, 1990; Ryfe, 2005). Accuracy goals motivate people “to seek out and carefully consider relevant evidence,” to process both pro- and counter-attitudinal information in a relatively unbiased way, “so as to reach a correct or otherwise best conclusion” (Taber & Lodge, 2006, p. 756). Partisan goals, on the other hand, cause people to “apply their reasoning powers in defense of a prior, specific conclusion” (Taber & Lodge, 2006, p. 756). This is also referred to as motivated reasoning: “the tendency to seek out and/or view new evidence as consistent with one’s prior views, even if this is not objectively accurate” (Druckman & Bolsen, 2011, p. 663). Motivated reasoning requires cognitive effort, to remember and construct beliefs and theories that counter-argue counter-attitudinal information and support a priori conclusions. But it does not contribute to the well-considered balancing of different opinions that is necessary to arrive at well-informed opinions. Hence, it is important to understand when and why citizens elaborate with the aim of forming an accurate opinion (Druckman, 2012, 2014). Personal Antecedents of Motivated Reasoning A personal characteristic that contributes to motivated reasoning is a high need to evaluate. People with a high need to evaluate are “chronically engaged in evaluation” (Bizer et al., 2004, p. 1001). They have more and stronger prior opinions that guide information processing more strongly, than people with a lower need to evaluate (Hart & Nisbet, 2012). Hence, people with a high need to evaluate are more likely to be motivated by directional (rather than accuracy) goals when elaborating media content. Empirically, this translates into a smaller repertoire of counter-attitudinal arguments (Nir, 2011). Thus, need to evaluate moderates the impact of elaboration on argument repertoire bias, too: H2a: Elaboration of news media content by news users results in a stronger pro-attitudinal bias in argument repertoire, if users have a higher need to evaluate. Media-Related Antecedents of the Consideration of Counter-Attitudinal Arguments Our primary interest, though, is again the possible contribution of news media to opinion quality. In this case, we are interested in whether media can trigger a more even-handed consideration of pro- and counter-attitudinal arguments. Our point of departure is the claim that citizens need to be able to imagine themselves, however briefly, in the position of others, to consider counter-attitudinal arguments (Goodin, 2000). Imagining oneself in the position of others requires that one perceives some common ground with these “others.” One needs to share a similar underlying value system or worldview, to be able to see a situation from an opponent's perspective (Ferree, Gamson, Gerhards, & Rucht, 2002). If there is no common ground at all, we cannot reasonably expect people to engage in deliberation (Burkhalter et al., 2002). The consideration of counter-attitudinal arguments is fruitless if differences between stakeholders are rooted in irreconcilable clashes of ideology and fundamental interests. For these reasons, we tentatively propose that news media may model, cue, or facilitate the consideration of counter-attitudinal arguments, when they present, in their coverage, a constructive debate climate: when they report stakeholders as adopting a consensual orientation (as one criterion for deliberation). A constructive debate climate is presented, when news media, for example, report that stakeholders make an effort to listen to each other and search for mutual agreement. Instead, when news media present only conflicting views, or even foreground seemingly irreconcilable differences, an adversarial debate climate is presented. This may induce news users to engage more readily in motivated reasoning, focusing only on like-minded arguments. Thus, we propose that the debate climate (as a characteristic of news media reporting) moderates the impact of elaboration: H2b: Elaboration of news media content by news users results in a better-balanced argument repertoire, if news media present the debate as constructive. Constructive Debate Orientation and Argument Repertoire Bias These hypotheses shape our second model (see Figure 2). Elaboration is a necessary condition to process not only pro- but also counter-attitudinal arguments. But the eventual impact of elaboration is moderated by the goals of information processing. Accuracy goals contribute to an unbiased focus and objective balancing of pro- and counter-attitudinal arguments, which will result in a relatively even-handed argument repertoire. We predict that such objective elaboration is more likely to occur if media present the debate as constructive (H2b). Citizens with a higher need to evaluate, on the other hand, are more likely to engage in motivated reasoning, which will result in a stronger pro-attitudinal bias (H2a). Figure 2 View largeDownload slide Constructive debate climate, need to evaluate, and argument repertoire bias Figure 2 View largeDownload slide Constructive debate climate, need to evaluate, and argument repertoire bias Method To investigate the impact of news media quality on public opinion, we conducted an online experiment with a two-by-two factorial design in the Netherlands. Participants were asked to read a news item on a local debate on underground CO2 storage. We considered CO2 storage an appropriate case for the experiment, because it is a relatively unfamiliar and at the same time contested issue in the Netherlands (European Commission, 2011, pp. 74, 80), that was not salient in the media at the time of our investigation. This enabled us to investigate experimentally to what extent citizens may learn different arguments from news media. Two dimensions of reporting were manipulated: justification and constructive debate climate. The default condition (1) is a news item in which arguments are not substantiated (no justification) and the debate climate is not constructive (but adversarial); the other conditions combine (2) no justification with a constructive debate climate; (3) justification with an adversarial debate climate; and (4) justification with a constructive debate climate. Participants Participants were randomly selected from the online panel of a Dutch market research institute. The sample was stratified according to gender, age, and education, to make it more representative for the Dutch population. After excluding straight-liners, we retained 557 of the 578 people who participated in the experiment. In total, 50.8% are female. The age of participants ranged between 17 and 81 (M = 45.26, SD = 15.24). Educational levels varied between primary school and PhD. The largest groups completed intermediate vocational education (32.1%) or higher professional education (19.4%). Stimulus Material We constructed four different news items, all variations on the same primary text. This text was a news report covering a local meeting on underground CO2 storage, chaired by the mayor, in an imaginary community. The text carefully presented residents and experts making arguments in favor and against CO2 storage. To increase ecological validity, large parts of the text were borrowed from existing news items, and the lay out followed the design of a widely read online news Web site in the Netherlands. For the two justified news items, we added evidence and supporting arguments to the claims made by the actors in the debate. For example, whereas participants in the nonjustified items merely claim that energy efficiency and green energy are better solutions, they add in the justified items that “research from the British Institute for Energy and Climate shows that CO2 storage is an expensive solution to protect the coal industry.” To manipulate the debate climate, the headings and body of the article described, in the constructive conditions, the debate as “constructive,” the actors as “searching for mutual understanding,” and the mayor as confident that “the inhabitants of his town would find a common way out.” In the adversarial conditions, the debate instead was described as “uproarious,” the actors as “diametrically at odds with each other,” and the mayor as concluding that “the differences are irreconcilable.” Draft versions of all articles were pretested on a student population. The results indicated that the items did not differ in readability, nor in the emphasis on arguments pro- and contra-CO2 storage. During the experiment, after reading the news items, participants were asked four questions (on 7-point Likert scale) to judge the level of justification and the debate climate. Results (available on request) show that the manipulations were successful. Dependent Variables Argument repertoire We assess argument repertoire with two open questions (based on Cappella et al., 2002), asking (in random order) for all possible reasons in favor and against underground CO2 storage (see Appendix). A research assistant coded the answers to these questions into 28 possible, relevant arguments. Following Cappella et al. (2002), we did not distinguish between accurate and inaccurate arguments. Per participant, a maximum of three arguments in favor and three arguments against were counted. The number of coded arguments provided the dependent variable argument repertoire count. On average, the participants suggested 2.35 arguments (SD = 1.83). Some participants just provided key words (“dangerous” and “climate change”), others more elaborate descriptions. We, therefore, assessed the level of articulation of arguments, too. We awarded one point for arguments that were just keywords, two points for considerations of at least a half-sentence, and three points for extended considerations of at least two half sentences concerning at least two different aspects. Adding all points resulted in an argument repertoire articulation variable that theoretically ranged from 0 (no arguments provided) to 18 (six extended considerations provided), and in practice from 0 to 16 (M = 4.57; SD = 3.85). To test intercoder reliability, a random sample of 100 answers was recoded by a second coder. The Krippendorff alpha values were excellent to satisfactory: 0.96 for the count of arguments, and 0.71 and 0.78 for the level of articulation arguments pro- and contra-CO2 storage. Argument repertoire bias To differentiate between pro- and counter-attitudinal arguments, we asked participants (before the argument repertoire questions, but after reading the news item) to indicate on a 7-point scale (with 1 = completely opposed and 7 = completely in favor), “considering everything you know about CO2 storage, are you in favour or against underground CO2 storage in the Netherlands?” (M = 3.53; SD = 1.60). In total, 26.0% expressed a neutral opinion. For the others (N = 412), we classified their arguments as either pro- or counter-attitudinal and estimated the pro-attitudinal bias of argument repertoire count as the number of pro-attitudinal minus the number of counter-attitudinal arguments. The variable ranges between −2 and 3, with a higher positive number indicating a stronger positive pro-attitudinal bias (M = 0.31, SD = 0.90). Likewise, we estimated the pro-attitudinal bias of argument repertoire articulation (ranging from −5 to 6, M = 0.71, SD = 1.76). Mediators and Moderators Elaboration was measured as the average score on three items proposed by Borah (2011) to assess whether participants processed information more deeply and consciously. For these and all remaining items, we used 7-point Likert scales (see Appendix). The items form a sufficiently reliable scale (Cronbach's alpha = 0.70), with higher scores indicating more effortful processing (M = 4.50, SD = 1.16). Need to evaluate was assessed at the beginning of the survey with five items previously used by Nir (2011), using the authorized Dutch translations by Hermans (1997). (Cronbach's alpha = .73, M = 4.44, SD = 0.95). Need for cognition was assessed with four items. In selecting these items, we started again from the items proposed by Nir (2011). But after considering the official 15-item Dutch scale (Pieters, Verplanken, & Modde, 1987), we made a partly different choice (see Appendix) for items with higher item-total correlations. The resulting scale is reliable (Cronbach's alpha = .75, M = 4.73, SD = 1.00). Issue interest. To assess participants' pretest issue interest, we asked them (again before reading the news item), among six other issues, to indicate whether they were personally concerned about underground CO2 storage and climate change. These two items make a reliable scale (Cronbach's alpha = 0.73, M = 4.04, SD = 1.39). Analysis To get a look at our first model (H1a–e), we compare argument repertoire across justified and nonjustified conditions with one-way analysis of variance (ANOVA). Next, in line with previous studies in the field (e.g., Blanton, Strauts, & Perez, 2012; Cho, 2013; Stroud & Muddiman, 2013), we use PROCESS, developed by Hayes, to assess the mediating role of elaboration (with PROCESS model 4), the moderating role of need for cognition and issue interest (PROCESS model 2), and finally, the impact of both moderators and the mediator in a model of moderated mediation (PROCESS model 9). For our second model (H2a–b), we follow a similar procedure. We compare the pro-attitudinal bias in argument repertoire across debate climates with one-way ANOVA, and then use PROCESS model 2 to assess more specifically the moderating role of need to evaluate and debate climate. PROCESS generates coefficients using ordinary least squares regression that represent both “direct effects” and “indirect effects” via a mediator and under consideration of possible moderators. To obtain confidence limits for the indirect effects, we use bootstrapping techniques as recommended by Preacher, Rucker, and Hayes (2007) and others. Specifically, we generate 95% bias-corrected accelerated confidence intervals on the basis of 1,000 bootstrap samples for specific indirect effects. If intervals do not include zero, the indirect effect significantly differs from zero. To facilitate interpretation, all independent, moderator, and mediator variables are mean-centered, as recommended by Hayes (2013, p. 282 ff.) when investigating interactions between variables where zero has no natural meaning. Results Justification and Argument Repertoire We predicted that readers of news items in which arguments are justified develop a larger argument repertoire than readers of news media items in which arguments are not justified. Results show that readers of justified news items offer more arguments (M = 2.51, SD = 1.90) than readers of nonjustified items (M = 2.18, SD = 1.75), F(1,555) = 4.60, p < .05). Likewise, readers of justified news items provide more articulated reasons (M = 5.01, SD = 4.12) than readers in the nonjustified conditions (M = 4.165, SD = 3.53), F(1,555) = 6.89, p < .01. We conclude that justification causes a larger argument repertoire. H1a is supported. Next, we argued that the impact of justification on argument repertoire is mediated by elaboration. Justification is predicted to trigger elaboration (H1c), and elaboration is expected to cause people to learn more arguments from news media (H1b). Mediation analysis produces mixed results for these hypotheses. Justification does have a positive effect on elaboration (B = 0.16) but this effect is only significant at p < .1. Elaboration in turn is clearly positively related to argument repertoire count (B = 0.60, p < .001) and articulation (B = 1.34, p < .001). Combined, the indirect path from justification, via elaboration, to argument repertoire is not significant at p < .05. H1b is confirmed but H1c not. The main effect of justification on argument repertoire is direct rather than mediated by elaboration. Moderation analysis adds that the impact of justification on argument repertoire is moderated by issue interest and need for cognition (Table 1). Participants in justified conditions develop larger and more elaborate argument repertoires, if they are interested in the issue and have a moderate to high need for cognition. These findings support H1d and H1e. Table 1 Conditional Effects of Justification on Argument Repertoire at Specific Levels of Issue Interest and Need for Cognition Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 Table 1 Conditional Effects of Justification on Argument Repertoire at Specific Levels of Issue Interest and Need for Cognition Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 For a final test of the hypotheses, we integrate the mediator and moderators in a model of moderated mediation. Qualifying the previous findings, we find that justification has the predicted indirect effect on argument repertoire, but only when issue interest is moderate to high, and when need for cognition is high (Table 2). These findings confirm H1b, H1c, H1d, and H1e. But the mediating role of elaboration is far from perfect. There still is, in the final model, a significant direct effect of justification on argument repertoire (confirming H1a). Table 2 Conditional Indirect Effects of Justification on Argument Repertoire, Via Elaboration, at Specific Values of Issue Interest and Need for Cognition and Direct Effects Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. Number of bootstraps samples: 1000. Level of confidence: 0.95. LLCI = loser limit confidence interval; ULCI = upper limit confidence interval. Table 2 Conditional Indirect Effects of Justification on Argument Repertoire, Via Elaboration, at Specific Values of Issue Interest and Need for Cognition and Direct Effects Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. Number of bootstraps samples: 1000. Level of confidence: 0.95. LLCI = loser limit confidence interval; ULCI = upper limit confidence interval. Debate Climate and Argument Repertoire Bias We ran similar analyses to assess whether news items with a constructive debate climate reduce the pro-attitudinal bias in citizens' argument repertoire. A comparison of means shows the debate climate has no overall effect. Mean values for pro-attitudinal bias are not significantly different across debate climates, regardless of whether we look at argument repertoire count (F(1,410) = 0.06, p = .810) or articulation (F(1,410) = 0.06, p = .811). Table 3 presents the findings from the moderation analyses. These show that elaboration increases the pro-attitudinal bias of argument repertoire when two conditions are met: news items adopt an adversarial debate climate and participants have a moderate to low need to evaluate. The first condition is in line with H2b (although the emphasis is more on the bias-enhancing effect of an adversarial climate than a balancing effect of a constructive climate). Table 3 Conditional Effects of Elaboration on Argument Repertoire Bias at Specific Levels of Need to Evaluate and Debate Climate Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 Table 3 Conditional Effects of Elaboration on Argument Repertoire Bias at Specific Levels of Need to Evaluate and Debate Climate Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 The second condition contradicts H2a. In our sample, a high need to evaluate is associated with a better-balanced argument repertoire, not a stronger bias, as we expected on the basis of Nir (2011). To explore this interesting finding post hoc, we repeated some of Nir's analyses (results not reported in detail). Results show that in our sample especially citizens that both have a higher need for cognition and a higher need to evaluate tend to list fewer pro-attitudinal arguments. One tentative explanation is that these citizens, both for intellectual and evaluative reasons, are especially interested in counter-attitudinal arguments and may take like-minded arguments in news media coverage for granted. The implications of this for opinion quality need to be further explored. Discussion Our findings show that news media do contribute to public opinion quality, if they meet the normative requirements of deliberative theory. When news media provide evidence and substantive arguments to support the positions of actors in debates, citizens develop larger argument repertoires. They are able to provide more articulated reasons why one might be in favor or against a particular course of action. Especially the direct, informative effect of news media on argument repertoire is relatively strong. In addition, justification triggers more elaborate processing, but only for citizens who are already inclined to deliberate—because they have a high need for cognition and an interest in the issue. Third, elaboration contributes to a biased argument repertoire, especially for citizens with a moderate to lower need to evaluate, when the debate climate is adversarial. A constructive debate climate prevents this negative effect on opinion quality. Limitations Before discussing these findings, we review the limitations of our study. The experimental design enabled us to investigate in relative detail the impact of two characteristics of news media content quality on argument repertoire. But we do not know how many citizens would read the news item in normal (i.e., nonexperimental) circumstances. This reduces the external validity of our findings. Related, we acknowledge that our experiment informs us only about short-term effects (Gaines, Kuklinski, & Quirk, 2007) and that our findings might be message-, medium-, issue-, or country-specific (Slater, 1991). There is, therefore, a need for future studies to replicate our findings, to assess how long the established effects last after the experimental treatment, and to extend the analysis to other types of media, issues, and political systems. Second, there are limitations in the extent to which we can draw conclusions in terms of causality. Our experimental design allows us to conclude that justification causes some respondents (namely those with a high need to evaluate and issue interest) to elaborate more, and that justification causes the average respondent to offer more arguments. But the evidence for the relationship between the mediator and the outcome variable is only correlational, not causal. A similar argument applies to our second model. Again, we can conclude that under specific conditions (low to moderate need to evaluate, adversarial debate climate in news item), elaboration and bias are positively associated. More theoretical and empirical work is necessary before any causality can be inferred. Implications While keeping these limitations in mind, our findings have several interesting implications for the political communications literature. First, our study confirms that the impact of news media on opinion quality follows from the interaction between news media content, cognitive behavior, and issue interest (e.g., Elenbaas et al., 2014; Iyengar et al., 2009; Luskin, 1990). Second, our findings support other studies that argue that the emphasis of news media on conflicts may have negative consequences for society. Specifically, our study suggests that an emphasis on irreconcilable differences may hinder internal deliberation. Third, and most importantly, our study supports the normative claim that news media should meet the standard of deliberativeness. Justification and a constructive debate climate, two deliberative qualities of news media elaborated in this article, do matter. To support this claim, we have shown that justification contributes to an increase in argument repertoire, which in turn enables, though not necessarily implies, that opinions are based on better knowledge of conflicting positions and underlying arguments. We also noted that justification and a constructive debate climate have an activating and balancing effect, respectively. This suggests that news media can stimulate deliberation “from within,” if and when these media meet deliberative quality criteria. Combined, these findings provide tentative support for the idea that not only face-to-face but also mediated deliberation can improve the quality of opinions in society. We hope our study contributes to a growing body of literature in which this claim is better justified. Future Research Reflecting on our analysis, we recommend three areas for future research. First, as already indicated above, our findings need to be replicated in different contexts with different news media. One interesting factor to be investigated in this respect, we propose, is news media trust. News media trust varies considerably across media systems. It may be considered both as an outcome of the deliberative quality of news media and a moderator, affecting the impact of news media on opinion quality. Another interesting comparative question is to what extent media and story types, other than the informative online news item investigated here, may stimulate interest or trigger deliberation. What is the potential impact of personalization, of storytelling in journalism, of informative games, of televised documentaries? We see this as an important avenue for further research. Second, our study suggests to study the deliberative quality of existing news media content as well as to explore its determinants (such as competition, journalistic professionalism, public broadcasting, and news cultures). Because it matters whether media provide justifications and a constructive debate climate, we need to explore ways to stimulate news organizations to provide such content. A third promising area of research concerns the noncognitive aspects of opinion quality. Deliberative theory emphasizes the disinterested and rational weighing of arguments, in search for the best possible solution. We accordingly focused our study on media coverage and a cognitive measure of opinion quality. But in reality, engagement, (self-)interest, values, emotions, and political identities play a role in opinion formation, too. Following our theoretical argument, we propose in particular to further explore the notion of a constructive debate climate. The willingness of citizens to adopt a consensual orientation is a crucial component of democracy (Goodin, 2000). We see it as a challenging question for research to what extent a constructive debate climate in media, in interaction with personal characteristics (such as openness), emotions, and engagement, can stimulate citizens to deliberation “from within” and seriously consider the different arguments that quality media offer. Conclusions Research has consistently shown that public opinion does not meet normative democratic criteria, owing to both shortcomings in news media and personal failings. In this article, we propose that these arguments also imply that there are several ways to improve public opinion. Along these lines, our study suggests that when journalists (1) ensure that actors provide justifications for their positions, rather than mere claims, and (2) emphasize the search for common ground rather than fundamental differences, citizens might develop a larger and more balanced argument repertoire. This is an important building block for public opinion quality. These conclusions call, in our view, for more research on how we can strengthen the quality of news media in today's profit-oriented and highly competitive news media industry. Appendix: Overview of Measures Argument Repertoire “Suppose some-one you know is invited to participate in a debate on underground CO2 storage. This person wants to prepare well, but knows little about the issue. He asks you about the arguments. What would you answer? What are the reasons you or other people may have to be in favour of [against] underground CO2 storage. Please list all the reasons in favour of [against] underground CO2 storage that come to mind.” Elaboration “To what extent do you agree or disagree with the following statements?” (1 = disagree completely to 7 = agree completely). “I paid attention to the article; My mind wandered while I was reading the news article (reversed); The article made me think.” Need to Evaluate Same question and scale as above. "I form opinions about everything; It is very important to me to hold strong opinions; It bothers me to remain neutral; I would rather have a strong opinion than no opinion at all; I pay a lot of attention to whether things are good or bad." Need for Cognition Same question and scale as above. "I like to have the responsibility of handling a situation that requires a lot of thinking; I really enjoy a task that involves coming up with new solutions to problems; I would prefer a task that is intellectual, difficult, and important to one that is somewhat important but does not require much thought; I find satisfaction in deliberating hard and for long hours." Issue Interest "Different people consider different issues important. What about you? Which of the following issues [Underground CO2 storage; Climate change] is very important to you personally and which not at all?" (1 = is not at all important to me personally to 7 = is very important to me personally). Acknowledgments We thank two anonymous reviewers and the editors for their valuable comments. References Aday S. ( 2006 ). The framesetting effects of news: An experimental test of advocacy versus objectivist frames . Journalism and Mass Communication Quarterly , 83 , 767 – 784 . doi: 10.1177/107769900608300403 Google Scholar CrossRef Search ADS Andersen V. N. , Hansen K. M. ( 2007 ). How deliberation makes better citizens: The Danish Deliberative Poll on the euro . European Journal of Political Research , 46 , 531 – 556 . doi: 10.1111/j.1475-6765.2007.00699.x Google Scholar CrossRef Search ADS Beaudoin C. E. ( 2008 ). The internet’s impact on international knowledge . New Media and Society , 10 , 455 – 474 . doi: 10.1177/1461444807085327 Google Scholar CrossRef Search ADS Beaudoin C. E. , Thorson E. ( 2004 ). Testing the cognitive mediation model: The roles of news reliance and three gratifications sought . Communication Research , 31 , 446 – 471 . doi: 10.1177/0093650204266098 Google Scholar CrossRef Search ADS Bizer G. Y. , Krosnick J. A. , Holbrook A. L. , Wheeler S. C. , Rucker D. D. , Petty R. E. ( 2004 ). The impact of personality on cognitive, behavioral, and affective political processes: The effects of need to evaluate . Journal of Personality , 72 , 995 – 1027 . doi: 10.1111/j.0022-3506.2004.00288.x Google Scholar CrossRef Search ADS Blanton H. , Strauts E. , Perez M. ( 2012 ). Partisan identification as a predictor of cortisol response to election news . Political Communication , 29 , 447 – 460 . doi: 10.1080/10584609.2012.736239 Google Scholar CrossRef Search ADS Borah P. ( 2011 ). Seeking more information and conversations: Influence of competitive frames and motivated processing . Communication Research , 38 , 303 – 325 . doi: 10.1177/0093650210376190 Google Scholar CrossRef Search ADS Burkhalter S. , Gastil J. , Kelshaw T. ( 2002 ). A conceptual definition and theoretical model of public deliberation in small face-to-face groups . Communication Theory , 12 , 398 – 422 . doi: 10.1111/j.1468-2885.2002.tb00276.x Cacioppo J. , Petty R. ( 1982 ). The need for cognition . Journal of Personality , 42 , 116 – 131 . doi: 10.1037/0022-3514.42.1.116 Cappella J. N. , Price V. , Nir L. ( 2002 ). Argument repertoire as a reliable and valid measure of opinion quality: Electronic dialogue during campaign 2000 . Political Communication , 19 , 73 – 93 . doi: 10.1080/105846002317246498 Google Scholar CrossRef Search ADS Cho J. ( 2013 ). Campaign tone, political affect, and communicative engagement . Journal of Communication , 63 , 1130 – 1152 . doi: 10.1111/jcom.12064 Google Scholar CrossRef Search ADS Chong D. , Druckman J. N. ( 2007a ). Framing public opinion in competitive democracies . American Political Science Review , 101 , 637 – 655 . doi: 10.1017/S0003055407070554 Google Scholar CrossRef Search ADS Chong D. , Druckman J. N. ( 2007b ). Framing theory . Annual Review of Political Science , 10 , 103 – 126 . doi: 10.1146/annurev.polisci.10.072805.103054 Google Scholar CrossRef Search ADS Cottle S. ( 2006 ). Between display and deliberation: Analyzing TV news as communicative architecture . Media, Culture and Society , 28 , 163 – 189 . doi: 10.1177/0163443706061680 Google Scholar CrossRef Search ADS Curran J. , Iyengar S. , Brink Lund A. , Salovaara-Moring I. ( 2009 ). Media system, public knowledge and democracy: A comparative study . European Journal of Communication , 24 , 5 – 26 . doi: 10.1177/0267323108098943 Google Scholar CrossRef Search ADS David C. C. ( 2009 ). Learning political information from the news: A closer look at the role of motivation . Journal of Communication , 59 , 243 – 261 . doi: 10.1111/j.1460-2466.2009.01414.x Google Scholar CrossRef Search ADS Delli Carpini M. X. , Cook F. L. , Jacobs L. R. ( 2004 ). Public deliberation, discursive participation, and citizen engagement: A review of the empirical literature . Annual Review of Political Science , 7 , 315 – 344 . doi: 10.1146/annurev.polisci.7.121003.091630 Google Scholar CrossRef Search ADS Druckman J. N. ( 2001 ). The implications of framing effects for citizen competence . Political Behavior , 23 , 225 – 256 . doi: 10.1023/A:1015006907312 Google Scholar CrossRef Search ADS Druckman J. N. ( 2012 ). The politics of motivation . Critical Review , 24 , 199 – 216 . doi: 10.1080/08913811.2012.711022 Google Scholar CrossRef Search ADS Druckman J. N. ( 2014 ). Pathologies of studying public opinion, political communication, and democratic responsiveness . Political Communication , 31 , 467 – 492 . doi: 10.1080/10584609.2013.852643 Google Scholar CrossRef Search ADS Druckman J. N. , Bolsen T. ( 2011 ). Framing, motivated reasoning, and opinions about emergent technologies . Journal of Communication , 61 , 659 – 688 . doi: 10.1111/j.1460-2466.2011.01562.x Google Scholar CrossRef Search ADS Elenbaas M. , de Vreese C. , Schuck A. , Boomgaarden H. ( 2014 ). Reconciling passive and motivated learning: The saturation-conditional impact of media coverage and motivation on political information . Communication Research , 41 , 481 – 504 . doi: 10.1177/0093650212467032 Google Scholar CrossRef Search ADS European Commission . ( 2011 ). Special Eurobarometer 364. Public awareness and acceptance of CO2 capture and storage (Wave 75.1) . Brussels : European Commission . Retrieved from http://ec.europa.eu/public_opinion/archives/ebs/ebs_364_en.pdf Eveland W. P. ( 2001 ). The cognitive mediation model of learning from the news: Evidence from nonelection, off-year election, and presidential election contexts . Communication Research , 28 , 571 – 601 . doi: 10.1177/009365001028005001 Google Scholar CrossRef Search ADS Ferree M. M. , Gamson W. A. , Gerhards J. , Rucht D. ( 2002 ). Four models of the public sphere in modern democracies . Theory and Society , 31 , 289 – 324 . doi: 10.1023/A:1016284431021 Google Scholar CrossRef Search ADS Fischer P. , Greitemeyer T. , Frey D. ( 2008 ). Self-regulation and selective exposure: The impact of depleted self-regulation resources on confirmatory information processing . Journal of Personality and Social Psychology , 94 , 382 – 395 . doi: 10.1037/0022-3514.94.3.382 Google Scholar CrossRef Search ADS Fleming K. , Thorson E. ( 2008 ). Assessing the role of information-processing strategies in learning from local news media about sources of social capital . Mass Communication and Society , 11 , 398 – 419 . doi: 10.1080/15205430801950643 Google Scholar CrossRef Search ADS Fraile M. , Iyengar S. ( 2014 ). Not all news sources are equally informative: A cross-national analysis of political knowledge in Europe . The International Journal of Press/Politics , 19 , 275 – 294 . doi: 10.1177/1940161214528993 Google Scholar CrossRef Search ADS Gaines B. J. , Kuklinski J. H. , Quirk P. J. ( 2007 ). The logic of the survey experiment reexamined . Political Analysis , 15 , 1 – 20 . doi: 10.1093/pan/mpl008 Google Scholar CrossRef Search ADS Goodin R. E. ( 2000 ). Democratic deliberation within . Philosophy and Public Affairs , 29 , 81 – 109 . doi: 10.1111/j.1088-4963.2000.00081.x Google Scholar CrossRef Search ADS Graber D. ( 2003 ). The media and democracy: Beyond myths and stereotypes . Annual Review of Political Science , 6 , 139 – 160 . doi: 10.1146/annurev.polisci.6.121901.085707 Google Scholar CrossRef Search ADS Hart P. S. , Nisbet E. C. ( 2012 ). Boomerang effects in science communication: How motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies . Communication Research , 39 , 701 – 723 . doi: 10.1177/0093650211416646 Google Scholar CrossRef Search ADS Hayes A. F. ( 2013 ). Introduction to mediation: A regression-based approach. New York, NY : Guilford Publications . Hermans D. ( 1997 ). N.E.S. – Dutch version. Geautoriseerde Nederlandse vertaling van de Need to Evaluate Scale. Retrieved from http://www.ekgp.ugent.be/pages/nl/vragenlijsten/NES.pdf Holbert R. L. , Garrett R. K. , Gleason L. S. ( 2010 ). A new era of minimal effects? A response to Bennett and Iyengar . Journal of Communication , 60 , 15 – 34 . doi: 10.1111/j.1460-2466.2009.01470.x Google Scholar CrossRef Search ADS Ho S. S. , Peh X. , Soh V. W. L. ( 2013 ). The cognitive mediation model: Factors influencing public knowledge of the H1N1 pandemic and intention to take precautionary behaviors . Journal of Health Communication , 18 , 773 – 794 . doi: 10.1080/10810730.2012.743624 Google Scholar CrossRef Search ADS Iyengar S. , Hahn K. S. , Bonfadelli H. , Marr M. ( 2009 ). “Dark areas of ignorance” revisited: Comparing international affairs knowledge in Switzerland and the United States . Communication Research , 36 , 341 – 358 . doi: 10.1177/0093650209333024 Google Scholar CrossRef Search ADS Jensen J. D. ( 2011 ). Knowledge acquisition following exposure to cancer news articles: A test of the cognitive mediation model . Journal of Communication , 61 , 514 – 534 . doi: 10.1111/j.1460-2466.2011.01549.x Google Scholar CrossRef Search ADS Kim S.-H. , Scheufele D. A. , Shanahan J. , Choi D.-H. ( 2011 ). Deliberation in spite of controversy? News media and the public’s evaluation of a controversial issue in South Korea . Journalism and Mass Communication Quarterly , 88 , 320 – 336 . doi: 10.1177/107769901108800206 Google Scholar CrossRef Search ADS Kunda Z. ( 1990 ). The case for motivated reasoning . Psychological Bulletin , 108 , 480 – 98 . doi:10.1037/0033-2909.108.3.480 Google Scholar CrossRef Search ADS Luskin R. C. ( 1990 ). Explaining political sophistication . Political Behavior , 12 , 331 – 361 . doi: 10.1007/BF00992793 Google Scholar CrossRef Search ADS Luskin R. C. , Fishkin J. S. , Jowell R. ( 2002 ). Considered opinions: Deliberative polling in Britain . British Journal of Political Science , 32 , 455 – 487 . doi: 10.1017/S0007123402000194 Google Scholar CrossRef Search ADS Manosevitch E. ( 2009 ). The reflective cue: Prompting citizens for greater consideration of reasons . International Journal of Public Opinion Research , 21 , 187 – 203 . doi:10.1093/ijpor/edp013 Google Scholar CrossRef Search ADS Mutz D. C. , Martin P. S. ( 2001 ). Facilitating communication across lines of political difference: The role of mass media . American Political Science Review , 95 , 97 – 114 . Nir L. ( 2011 ). Motivated reasoning and public opinion perception . Public Opinion Quarterly , 75 , 504 – 532 . doi: 10.1093/poq/nfq076 Google Scholar CrossRef Search ADS Petty R. E. , Briñol P. , Priester J. R. ( 2008 ). Mass media attitude change: Implications of the elaboration likelihood model of persuasion. In Bryant J. , Oliver M. B. (Eds.), Media effects: Advances in theory and research (pp. 125 – 164 ). New York, NY : Taylor & Francis . Pieters R. G. M. , Verplanken B. , Modde J. M. ( 1987 ). “Neiging tot nadenken”: Samenhang met beredeneerd gedrag . Nederlands Tijdschrift Voor de Psychologie , 42 , 62 – 70 . Preacher K. J. , Rucker D. D. , Hayes A. F. ( 2007 ). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions . Multivariate Behavioral Research , 42 , 185 – 227 . doi: 10.1080/00273170701341316 Google Scholar CrossRef Search ADS Price V. , Cappella J. , Nir L. ( 2002 ). Does disagreement contribute to more deliberative opinion? Political Communication , 19 , 95 – 112 . doi: 10.1080/105846002317246506 Google Scholar CrossRef Search ADS Price V. , Neijens P. ( 1997 ). Opinion quality in public opinion research . International Journal of Public Opinion Research , 9 , 336 – 360 . doi: 10.1093/ijpor/9.4.336 Google Scholar CrossRef Search ADS Ryfe D. M. ( 2005 ). Does deliberative democracy work? Annual Review of Political Science , 8 , 49 – 71 . doi: 10.1146/annurev.polisci.8.032904.154633 Google Scholar CrossRef Search ADS Schuck A. , Vliegenthart R. , Boomgaarden H. G. , Elenbaas M. , Azrout R. , van Spanje J. , Vreese C. H. D. ( 2013 ). Explaining campaign news coverage: How medium, time, and context explain variation in the media framing of the 2009 European parliamentary elections . Journal of Political Marketing , 12 , 8 – 28 . doi: 10.1080/15377857.2013.752192 Google Scholar CrossRef Search ADS Shiffrin R. M. , Schneider W. ( 1977 ). Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory . Psychological Review , 84 , 127 – 190 . doi: 10.1037/0033-295X.84.2.127 Google Scholar CrossRef Search ADS Slater M. D. ( 1991 ). Use of message stimuli in mass communication experiments: A methodological assessment and discussion . Journalism Quarterly , 68 , 412 – 421 . doi: 10.1177/107769909106800312 Google Scholar CrossRef Search ADS Stromer-Galley J. ( 2007 ). Measuring deliberation’s content: A coding scheme . Journal of Public Deliberation , 3 ( 1 ), 1 – 35 . http://services.bepress.com/jpd/vol3/iss1/art12 Stroud N. J. , Muddiman A. ( 2013 ). Selective exposure, tolerance, and satirical news . International Journal of Public Opinion Research , 25 , 271 – 290 . doi: 10.1093/ijpor/edt013 Google Scholar CrossRef Search ADS Taber C. , Lodge M. ( 2006 ). Motivated skepticism in the evaluation of political beliefs . American Journal of Political Science , 50 , 755 – 769 . doi: 10.1111/j.1540-5907.2006.00214.x Google Scholar CrossRef Search ADS Van der Wurff R. , Verhoeven P. , Gadellaa M. ( 2013 ). Scientists and deliberativeness of European public television news . JCOM , 12 , A02 . Wessler H. ( 2008 ). Investigating deliberativeness comparatively . Political Communication , 25 , 1 – 22 . doi: 10.1080/10584600701807752 Google Scholar CrossRef Search ADS Wessler H. , Rinke E. M. ( 2014 ). Deliberative performance of television news in three types of democracy: Insights from the United States, Germany, and Russia . Journal of Communication , 64 , 827 – 851 . doi: 10.1111/jcom.12115 Google Scholar CrossRef Search ADS Winter S. , Krämer N. C. ( 2012 ). Selecting science information in web 2.0: How source cues, message sidedness, and need for cognition influence users’ exposure to blog posts . Journal of Computer-Mediated Communication , 18 , 80 – 96 . doi: 10.1111/j.1083-6101.2012.01596.x Google Scholar CrossRef Search ADS © The Author 2016. Published by Oxford University Press on behalf of The World Association for Public Opinion Research. All rights reserved. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Public Opinion Research Oxford University Press

News Quality and Public Opinion: The Impact of Deliberative Quality of News Media on Citizens' Argument Repertoire

Loading next page...
 
/lp/ou_press/news-quality-and-public-opinion-the-impact-of-deliberative-quality-of-cirnhukTdS
Publisher
Oxford University Press
Copyright
© The Author 2016. Published by Oxford University Press on behalf of The World Association for Public Opinion Research. All rights reserved.
ISSN
0954-2892
eISSN
1471-6909
D.O.I.
10.1093/ijpor/edw024
Publisher site
See Article on Publisher Site

Abstract

Abstract This article investigates the impact of the deliberative quality of online news items on argument repertoire. Participants in an online experiment (N = 557) read a news item that covers a local debate on underground CO2 storage. Two dimensions of deliberative quality are manipulated: justification, or the extent to which arguments in the news item are substantiated, and constructive debate climate, that is, the extent to which stakeholders are reported to work together toward a solution. The findings suggest that public opinion quality would be higher, if news media would better meet deliberative standards. Normative theories expect that citizens are able to express informed opinions on important policy issues. Empirical research, however, offers compelling evidence to the contrary (see Druckman, 2014). Two types of explanations have been advanced. First, citizens might not be able or willing to digest available political information. Second, news media might not adequately perform their role as crucial sources of political information (e.g., Iyengar, Hahn, Bonfadelli, & Marr, 2009). In relation, scholars have argued that normative expectations for public opinion are simply too ambitious (e.g., Graber, 2003). In this article, we take a complementary angle. Rather than contributing to an already extensive body of empirical research on opinion quality as it is, we explore whether it is feasible to improve public opinion quality. As political communication scholars, we focus on the potential contribution of news media. We draw inspiration from deliberative theory and build conceptually on cognitive models of learning from news media. Using an experimental design, we address the question whether quality of public opinion would be higher, if news media would adhere better to normative demands of deliberative theory. Argument Repertoire as Indicator for Opinion Quality Public opinion quality is a multifaceted and complex concept. Druckman (2014) identifies four criteria that are discussed in the literature. Opinions should be informed, internally consistent, coherent with a person's deep values, and not easily swayed by framing efforts. Price and Neijens (1997) even distinguish four sets of quality criteria. These concern public opinion at the individual and the collective level, and address the quality of opinions themselves as well as the process of opinion formation (Price & Neijens, 1997, p. 346). In this article, we have chosen “argument repertoire” (Cappella, Price, & Nir., 2002) as appropriate indicator of opinion quality. Argument repertoire refers to people's knowledge of pro-attitudinal arguments (in favor of their own opinion) and counter-attitudinal arguments (that support the positions of opponents). It is a reliable and valid cognitive measure that gauges the depth of knowledge of citizens on a particular issue (Cappella et al., 2002). As quality criterion, it aligns with the long-standing conviction in public opinion research that opinion quality depends on “the amount of information that supports an opinion” (Price & Neijens, 1997, p. 346; also Druckman, 2014). More specifically, and especially relevant to our study, argument repertoire aligns with the normative demand from deliberative theory (discussed below) that quality opinions are based on the consideration of diverse ideas and views (Cappella et al., 2002; Mutz & Martin, 2001). Of course, knowledge of arguments does not necessarily mean that these are considered in-depth, and values, emotions, and engagement are important components of opinion quality, too (Price & Neijens, 1997). But for a first assessment of the impact of the deliberative quality of news items on opinion quality, argument repertoire presents itself as a relevant, reliable, and valid choice (see also Manosevitch, 2009). Deliberative Criteria for News Media Quality The deliberative turn in democratic theory has made deliberativeness an important ideal for democracy and news media. At the core, we find the conviction that all stakeholders should discuss as many different ideas as possible in a civil and reasoned debate, to come up with commonly agreed solutions for social problems (Burkhalter, Gastil, & Kelshaw, 2002; Ryfe, 2005; Wessler, 2008). Theorists expect, and some empirical studies confirm, that deliberative processes contribute to the quality of public opinion and policy-making (Delli Carpini, Cook, & Jacobs, 2004). For example, participants in deliberative polls—quasi-experimental events, lasting a few days, where citizens discuss a specific policy issue with experts and politicians—have been found to express more informed and reasoned opinions (Andersen & Hansen, 2007; Luskin, Fishkin, & Jowell, 2002). Research also shows that people that are exposed to disagreement in interpersonal political debates can better provide arguments in favor and against their own opinions (Price, Cappella, & Nir, 2002). In modern societies, however, genuine face-to-face deliberation between many stakeholders is not really feasible. Public deliberation instead takes place in mediated public spheres. Scholars have accordingly proposed that (these debates in) news media should meet the standards of deliberativeness, too (Wessler, 2008). The extent to which media meet these criteria varies across situations. The extent to which mediated deliberation subsequently contributes to the quality of opinions and decision-making in society is hardly investigated (see below). Criteria for Deliberativeness Public deliberation, whether face-to-face or in the media, should meet three important criteria (Burkhalter et al., 2002; Stromer-Galley, 2007; Wessler, 2008). The first is diversity: All those affected by an issue should consider all relevant ideas. Next, these different ideas should be justified and discussed in a rational way. Arguments should be based on accurate information, justifications for claims are to be given and demanded, and opposing arguments should be considered on substantial grounds only. Third, deliberation requires a shared, constructive debate climate. Participants should respect each other, listen to diverging views, make an effort to work together, look for shared solutions and the “common good,” and make difficult choices that transcend particular self-interests. A large argument repertoire, a firm grasp of the diverging arguments that are at play in an issue, would be the logical outcome of such deliberation (Cappella et al., 2002). Empirical research suggests that news media tend to adhere to the first criterion of deliberativeness. News media tend to provide citizens, at least to some extent and in comparison with interpersonal networks, with diverse issue information (Graber, 2003; Mutz & Martin, 2001). Research also shows that there are wide variations in the levels of diversity and justification in news media across countries, time periods, and news media (Van der Wurff, Verhoeven, & Gadellaa, 2013; Wessler & Rinke, 2014). Whether news media present policy debates as constructive, is not investigated (for an exception, see Cottle, 2006). The large body of literature on conflict framing, though, seems to point in a different direction (Schuck et al., 2013). Turning to the impact of these characteristics, research suggests that diversity of news media content contributes to public opinion quality. Diversity, or more precisely, competitive framing, motivates people to be accurate and to process information in-depth (Chong & Druckman, 2007a). It reduces the (manipulative) impact of single-issue frames on audience perceptions (Aday, 2006; Chong & Druckman, 2007b; Druckman, 2001) and leads people to take positions that better fit their “true” beliefs and values (Chong & Druckman, 2007a). The impact of justification and a constructive debate climate in news media on public opinion quality has not been investigated. Our study attempts to start filling this gap. Our starting point is the idea that news media can act as a model (Wessler, 2008). Perhaps, news media can stimulate individual citizens to deliberate “from within” (Goodin, 2000)—to conduct imaginative debates, to try to understand the viewpoints and arguments from opponents, and to express one's own reasons in return—even though citizens tend to be observers rather than participants in mediated debates. Such internal deliberation will increase awareness of pro- and counter-attitudinal arguments and hence translate into a larger argument repertoire. To better understand whether and how justification and a constructive debate climate, as deliberative criteria for news media quality, can influence argument repertoire, we turn to cognitive models of learning from news media. Learning from News Media News media are important sources of political information for citizens. How much citizens learn from these news media depends on the interplay between (a) the quality and quantity of news media content and (b) the willingness and ability of citizens to digest that information (Luskin, 1990). Along these lines, we follow Wessler (2008, p. 1) in arguing that deliberative news media can stimulate public opinion quality in two ways: by offering “diverging justifications for political positions” (i.e., relevant content) and by “model[ing] deliberative behavior in audiences” (i.e., triggering audiences to deliberate internally). News Media Content Comparative research shows that differences in news media content—across media types and/or media systems—matter for political knowledge. Some news outlets, like broadsheets and public TV news, pay more attention to hard news than others (tabloids, commercial TV), and thereby enable citizens to acquire more knowledge (Curran, Iyengar, Brink Lund, & Salovaara-Moring, 2009; Fraile & Iyengar, 2014). Moreover, in media systems where political or hard news is more abundant, or citizens have fewer options to choose less-informative news outlets, people learn about politics despite differences in attention. Gaps in political knowledge are consequently smaller (Elenbaas, de Vreese, Schuck, & Boomgaarden, 2014; Iyengar et al., 2009). The cited studies focus on factual knowledge. We assume, as tentatively suggested by David (2009), that the same reasoning applies to knowledge of arguments and hence argument repertoire. News media use has indeed been found to increase knowledge of arguments (Kim, Scheufele, Shanahan, & Choi, 2011). Accordingly, we suggest that when news media meet the criterion of justification (i.e., when news media provide more in-depth reasons for issue positions), users are fostered to learn these arguments and develop larger argument repertoires. We propose to refer to this effect as the informative contribution of news media to public opinion quality. In brief: H1a. Justification of arguments in news media increases argument repertoire. Elaboration of News Media Content How much people learn from available news media depends on cognitive behavior, in particular the level of elaboration of news media content (Beaudoin & Thorson, 2004; Eveland, 2001; Jensen, 2011). Eveland (2001, p. 573) defines elaboration as “the process of connecting new information to other information stored in memory, including prior knowledge, personal experiences, or the connection of two new bits of information together in new ways.” In the elaboration likelihood model (ELM), this is referred to as the “central route”; in cognitive psychology it is known as controlled processing. These concepts reference intentional types of information processing—spending cognitive effort and activating prior cognitions to carefully examine information and arguments. The opposite is known as automatic processing, or (in ELM) as the peripheral route. This involves the superficial processing of information, guided by prior opinions, cues, and heuristics (Petty, Briñol, & Priester, 2008; Shiffrin & Schneider, 1977). Elaboration and controlled processing are predicted to foster learning and long-term attitude change. Instead, automatic processing causes at best incidental learning and superficial changes in attitudes (Petty et al., 2008; Shiffrin & Schneider, 1977). Research confirms that elaboration stimulates learning (Fleming & Thorson, 2008). Accordingly, we predict that users who elaborate news media content, learn more arguments: H1b. Elaboration of news media content by news users increases argument repertoire. Media-Related Antecedents of Elaboration Generally speaking, humans tend to follow the peripheral route when processing information from news media (Druckman & Bolsen, 2011; Hart & Nisbet, 2012). It is therefore important to know whether and how citizens can be motivated to elaborate. Cognitive scholars tend to refer in this respect to personal characteristics (see below). We, however, are first of all interested in whether news media can stimulate elaboration. A few authors point in this direction. As already mentioned, Wessler (2008) suggests that deliberative news media may model deliberation among audiences. Manosevitch (2009) shows more specifically that news media can stimulate elaborate processing by providing “reflective” cues—gentle reminders to the audience about the importance of “thinking about issues.” ELM likewise suggests that news media can stimulate controlled processing by underlining the personal relevance of an issue, or by referencing multiple sources (Petty et al., 2008). Providing diverse or conflicting views in a news item, too, motivates people to be accurate and to process information more in-depth (Chong & Druckman, 2007a; Ryfe, 2005). Reviewing these arguments, we propose that justification in news media triggers citizens to elaborate. If participants in a mediated debate provide reasons and evidence for their views, users might follow their lead and consider an issue more in-depth, too—because news media provide a model, cue elaboration, and/or provide more complex information that triggers in-depth processing. Hence, we propose that justification has an activating effect: H1c. Justification in news media stimulates elaboration by news users. Personal Antecedents of Elaboration Personal characteristics are important determinants of elaboration, too. ELM proposes that people are more motivated to elaborate—and consequently learn more—when messages are perceived to have personal relevance (Petty et al., 2008). In political communication research, personal relevance tends to be discussed in terms of political or issue interest. Citizens who are interested in politics (or a specific issue) consider politics (or that issue) as relevant to them. Political interest is even equated with the internal motivation to learn about politics (Elenbaas et al., 2014). Accordingly, there is strong evidence that interest supports learning (Beaudoin, 2008; Elenbaas et al., 2014; Luskin, 1990). A second personal antecedent of elaboration is need for cognition (Petty et al., 2008; Ho, Peh, & Soh, 2013). Need for cognition is a “dispositional determinant” of central processing (Cacioppo & Petty, 1982, p. 130). It makes people structurally more likely to elaborate. People with a high need for cognition enjoy thinking. They engage spontaneously in effortful cognitive activity. Reviewing these arguments, we predict that issue interest and need for cognition moderate the impact of justification on elaboration. Justification triggers elaboration more strongly, when users are more interested in an issue, have a higher need for cognition, or both: H1d. Justification stimulates elaboration more strongly for news users with a higher issue interest. H1e. Justification stimulates elaboration more strongly for news users with a higher need for cognition. Justification and Argument Repertoire These hypotheses together result in our first model (Figure 1). Justification, the extent to which news media offer substantive reasons for issue positions, is predicted to have two effects on argument repertoire. The first is a direct, informative effect (H1a). Justification increases argument repertoire by providing more in-depth information. The second is indirect. Justification activates elaboration of news media content (H1c), and elaboration causes news users to develop a larger argument repertoire (H1b). The activating impact of justification will be stronger for people who for personal reasons are already more likely to elaborate, that is, for news users with a higher interest in the issue (H1d) or a higher need for cognition (H1e). Figure 1 View largeDownload slide Justification and argument repertoire Figure 1 View largeDownload slide Justification and argument repertoire Considering Opposing Arguments Deliberation involves the consideration of opposing arguments on substantive grounds, with the aim of reaching—if possible—common understanding (Burkhalter et al., 2002). Citizens should, therefore, be aware of various arguments. Their argument repertoire should encompass both pro- and counter-attitudinal arguments. Cognitive models and the literature on selective exposure suggest that the consideration of both pro- and counter-attitudinal arguments is associated with the central processing of information (Holbert, Garrett, & Gleason, 2010; Winter & Krämer, 2012). Elaboration is necessary to deal with the (perceived) complexity of counter-attitudinal information, and controlled effort is required to handle the negative emotions, feelings of dissonance, and threats to one's self-image that arise when one carefully considers arguments that go against one's standpoint (Fischer, Greitemeyer, & Frey, 2008). Automatic processing, in contrast, is associated with selective exposure and the (unconscious) exclusion of counter-attitudinal arguments (Fischer et al., 2008; Hart & Nisbet, 2012; Taber & Lodge, 2006). Elaboration Driven by Accuracy or Partisan Goals Elaboration, however, is not sufficient to foster the even-handed consideration of both pro- and counter-attitudinal arguments. What is equally important, is whether elaboration is driven by “accuracy” or “partisan” goals (Kunda, 1990; Taber & Lodge, 2006). Accuracy goals dominate when people need to make important decisions, or when they expect to be asked to justify their decisions later (Kunda, 1990; Ryfe, 2005). Accuracy goals motivate people “to seek out and carefully consider relevant evidence,” to process both pro- and counter-attitudinal information in a relatively unbiased way, “so as to reach a correct or otherwise best conclusion” (Taber & Lodge, 2006, p. 756). Partisan goals, on the other hand, cause people to “apply their reasoning powers in defense of a prior, specific conclusion” (Taber & Lodge, 2006, p. 756). This is also referred to as motivated reasoning: “the tendency to seek out and/or view new evidence as consistent with one’s prior views, even if this is not objectively accurate” (Druckman & Bolsen, 2011, p. 663). Motivated reasoning requires cognitive effort, to remember and construct beliefs and theories that counter-argue counter-attitudinal information and support a priori conclusions. But it does not contribute to the well-considered balancing of different opinions that is necessary to arrive at well-informed opinions. Hence, it is important to understand when and why citizens elaborate with the aim of forming an accurate opinion (Druckman, 2012, 2014). Personal Antecedents of Motivated Reasoning A personal characteristic that contributes to motivated reasoning is a high need to evaluate. People with a high need to evaluate are “chronically engaged in evaluation” (Bizer et al., 2004, p. 1001). They have more and stronger prior opinions that guide information processing more strongly, than people with a lower need to evaluate (Hart & Nisbet, 2012). Hence, people with a high need to evaluate are more likely to be motivated by directional (rather than accuracy) goals when elaborating media content. Empirically, this translates into a smaller repertoire of counter-attitudinal arguments (Nir, 2011). Thus, need to evaluate moderates the impact of elaboration on argument repertoire bias, too: H2a: Elaboration of news media content by news users results in a stronger pro-attitudinal bias in argument repertoire, if users have a higher need to evaluate. Media-Related Antecedents of the Consideration of Counter-Attitudinal Arguments Our primary interest, though, is again the possible contribution of news media to opinion quality. In this case, we are interested in whether media can trigger a more even-handed consideration of pro- and counter-attitudinal arguments. Our point of departure is the claim that citizens need to be able to imagine themselves, however briefly, in the position of others, to consider counter-attitudinal arguments (Goodin, 2000). Imagining oneself in the position of others requires that one perceives some common ground with these “others.” One needs to share a similar underlying value system or worldview, to be able to see a situation from an opponent's perspective (Ferree, Gamson, Gerhards, & Rucht, 2002). If there is no common ground at all, we cannot reasonably expect people to engage in deliberation (Burkhalter et al., 2002). The consideration of counter-attitudinal arguments is fruitless if differences between stakeholders are rooted in irreconcilable clashes of ideology and fundamental interests. For these reasons, we tentatively propose that news media may model, cue, or facilitate the consideration of counter-attitudinal arguments, when they present, in their coverage, a constructive debate climate: when they report stakeholders as adopting a consensual orientation (as one criterion for deliberation). A constructive debate climate is presented, when news media, for example, report that stakeholders make an effort to listen to each other and search for mutual agreement. Instead, when news media present only conflicting views, or even foreground seemingly irreconcilable differences, an adversarial debate climate is presented. This may induce news users to engage more readily in motivated reasoning, focusing only on like-minded arguments. Thus, we propose that the debate climate (as a characteristic of news media reporting) moderates the impact of elaboration: H2b: Elaboration of news media content by news users results in a better-balanced argument repertoire, if news media present the debate as constructive. Constructive Debate Orientation and Argument Repertoire Bias These hypotheses shape our second model (see Figure 2). Elaboration is a necessary condition to process not only pro- but also counter-attitudinal arguments. But the eventual impact of elaboration is moderated by the goals of information processing. Accuracy goals contribute to an unbiased focus and objective balancing of pro- and counter-attitudinal arguments, which will result in a relatively even-handed argument repertoire. We predict that such objective elaboration is more likely to occur if media present the debate as constructive (H2b). Citizens with a higher need to evaluate, on the other hand, are more likely to engage in motivated reasoning, which will result in a stronger pro-attitudinal bias (H2a). Figure 2 View largeDownload slide Constructive debate climate, need to evaluate, and argument repertoire bias Figure 2 View largeDownload slide Constructive debate climate, need to evaluate, and argument repertoire bias Method To investigate the impact of news media quality on public opinion, we conducted an online experiment with a two-by-two factorial design in the Netherlands. Participants were asked to read a news item on a local debate on underground CO2 storage. We considered CO2 storage an appropriate case for the experiment, because it is a relatively unfamiliar and at the same time contested issue in the Netherlands (European Commission, 2011, pp. 74, 80), that was not salient in the media at the time of our investigation. This enabled us to investigate experimentally to what extent citizens may learn different arguments from news media. Two dimensions of reporting were manipulated: justification and constructive debate climate. The default condition (1) is a news item in which arguments are not substantiated (no justification) and the debate climate is not constructive (but adversarial); the other conditions combine (2) no justification with a constructive debate climate; (3) justification with an adversarial debate climate; and (4) justification with a constructive debate climate. Participants Participants were randomly selected from the online panel of a Dutch market research institute. The sample was stratified according to gender, age, and education, to make it more representative for the Dutch population. After excluding straight-liners, we retained 557 of the 578 people who participated in the experiment. In total, 50.8% are female. The age of participants ranged between 17 and 81 (M = 45.26, SD = 15.24). Educational levels varied between primary school and PhD. The largest groups completed intermediate vocational education (32.1%) or higher professional education (19.4%). Stimulus Material We constructed four different news items, all variations on the same primary text. This text was a news report covering a local meeting on underground CO2 storage, chaired by the mayor, in an imaginary community. The text carefully presented residents and experts making arguments in favor and against CO2 storage. To increase ecological validity, large parts of the text were borrowed from existing news items, and the lay out followed the design of a widely read online news Web site in the Netherlands. For the two justified news items, we added evidence and supporting arguments to the claims made by the actors in the debate. For example, whereas participants in the nonjustified items merely claim that energy efficiency and green energy are better solutions, they add in the justified items that “research from the British Institute for Energy and Climate shows that CO2 storage is an expensive solution to protect the coal industry.” To manipulate the debate climate, the headings and body of the article described, in the constructive conditions, the debate as “constructive,” the actors as “searching for mutual understanding,” and the mayor as confident that “the inhabitants of his town would find a common way out.” In the adversarial conditions, the debate instead was described as “uproarious,” the actors as “diametrically at odds with each other,” and the mayor as concluding that “the differences are irreconcilable.” Draft versions of all articles were pretested on a student population. The results indicated that the items did not differ in readability, nor in the emphasis on arguments pro- and contra-CO2 storage. During the experiment, after reading the news items, participants were asked four questions (on 7-point Likert scale) to judge the level of justification and the debate climate. Results (available on request) show that the manipulations were successful. Dependent Variables Argument repertoire We assess argument repertoire with two open questions (based on Cappella et al., 2002), asking (in random order) for all possible reasons in favor and against underground CO2 storage (see Appendix). A research assistant coded the answers to these questions into 28 possible, relevant arguments. Following Cappella et al. (2002), we did not distinguish between accurate and inaccurate arguments. Per participant, a maximum of three arguments in favor and three arguments against were counted. The number of coded arguments provided the dependent variable argument repertoire count. On average, the participants suggested 2.35 arguments (SD = 1.83). Some participants just provided key words (“dangerous” and “climate change”), others more elaborate descriptions. We, therefore, assessed the level of articulation of arguments, too. We awarded one point for arguments that were just keywords, two points for considerations of at least a half-sentence, and three points for extended considerations of at least two half sentences concerning at least two different aspects. Adding all points resulted in an argument repertoire articulation variable that theoretically ranged from 0 (no arguments provided) to 18 (six extended considerations provided), and in practice from 0 to 16 (M = 4.57; SD = 3.85). To test intercoder reliability, a random sample of 100 answers was recoded by a second coder. The Krippendorff alpha values were excellent to satisfactory: 0.96 for the count of arguments, and 0.71 and 0.78 for the level of articulation arguments pro- and contra-CO2 storage. Argument repertoire bias To differentiate between pro- and counter-attitudinal arguments, we asked participants (before the argument repertoire questions, but after reading the news item) to indicate on a 7-point scale (with 1 = completely opposed and 7 = completely in favor), “considering everything you know about CO2 storage, are you in favour or against underground CO2 storage in the Netherlands?” (M = 3.53; SD = 1.60). In total, 26.0% expressed a neutral opinion. For the others (N = 412), we classified their arguments as either pro- or counter-attitudinal and estimated the pro-attitudinal bias of argument repertoire count as the number of pro-attitudinal minus the number of counter-attitudinal arguments. The variable ranges between −2 and 3, with a higher positive number indicating a stronger positive pro-attitudinal bias (M = 0.31, SD = 0.90). Likewise, we estimated the pro-attitudinal bias of argument repertoire articulation (ranging from −5 to 6, M = 0.71, SD = 1.76). Mediators and Moderators Elaboration was measured as the average score on three items proposed by Borah (2011) to assess whether participants processed information more deeply and consciously. For these and all remaining items, we used 7-point Likert scales (see Appendix). The items form a sufficiently reliable scale (Cronbach's alpha = 0.70), with higher scores indicating more effortful processing (M = 4.50, SD = 1.16). Need to evaluate was assessed at the beginning of the survey with five items previously used by Nir (2011), using the authorized Dutch translations by Hermans (1997). (Cronbach's alpha = .73, M = 4.44, SD = 0.95). Need for cognition was assessed with four items. In selecting these items, we started again from the items proposed by Nir (2011). But after considering the official 15-item Dutch scale (Pieters, Verplanken, & Modde, 1987), we made a partly different choice (see Appendix) for items with higher item-total correlations. The resulting scale is reliable (Cronbach's alpha = .75, M = 4.73, SD = 1.00). Issue interest. To assess participants' pretest issue interest, we asked them (again before reading the news item), among six other issues, to indicate whether they were personally concerned about underground CO2 storage and climate change. These two items make a reliable scale (Cronbach's alpha = 0.73, M = 4.04, SD = 1.39). Analysis To get a look at our first model (H1a–e), we compare argument repertoire across justified and nonjustified conditions with one-way analysis of variance (ANOVA). Next, in line with previous studies in the field (e.g., Blanton, Strauts, & Perez, 2012; Cho, 2013; Stroud & Muddiman, 2013), we use PROCESS, developed by Hayes, to assess the mediating role of elaboration (with PROCESS model 4), the moderating role of need for cognition and issue interest (PROCESS model 2), and finally, the impact of both moderators and the mediator in a model of moderated mediation (PROCESS model 9). For our second model (H2a–b), we follow a similar procedure. We compare the pro-attitudinal bias in argument repertoire across debate climates with one-way ANOVA, and then use PROCESS model 2 to assess more specifically the moderating role of need to evaluate and debate climate. PROCESS generates coefficients using ordinary least squares regression that represent both “direct effects” and “indirect effects” via a mediator and under consideration of possible moderators. To obtain confidence limits for the indirect effects, we use bootstrapping techniques as recommended by Preacher, Rucker, and Hayes (2007) and others. Specifically, we generate 95% bias-corrected accelerated confidence intervals on the basis of 1,000 bootstrap samples for specific indirect effects. If intervals do not include zero, the indirect effect significantly differs from zero. To facilitate interpretation, all independent, moderator, and mediator variables are mean-centered, as recommended by Hayes (2013, p. 282 ff.) when investigating interactions between variables where zero has no natural meaning. Results Justification and Argument Repertoire We predicted that readers of news items in which arguments are justified develop a larger argument repertoire than readers of news media items in which arguments are not justified. Results show that readers of justified news items offer more arguments (M = 2.51, SD = 1.90) than readers of nonjustified items (M = 2.18, SD = 1.75), F(1,555) = 4.60, p < .05). Likewise, readers of justified news items provide more articulated reasons (M = 5.01, SD = 4.12) than readers in the nonjustified conditions (M = 4.165, SD = 3.53), F(1,555) = 6.89, p < .01. We conclude that justification causes a larger argument repertoire. H1a is supported. Next, we argued that the impact of justification on argument repertoire is mediated by elaboration. Justification is predicted to trigger elaboration (H1c), and elaboration is expected to cause people to learn more arguments from news media (H1b). Mediation analysis produces mixed results for these hypotheses. Justification does have a positive effect on elaboration (B = 0.16) but this effect is only significant at p < .1. Elaboration in turn is clearly positively related to argument repertoire count (B = 0.60, p < .001) and articulation (B = 1.34, p < .001). Combined, the indirect path from justification, via elaboration, to argument repertoire is not significant at p < .05. H1b is confirmed but H1c not. The main effect of justification on argument repertoire is direct rather than mediated by elaboration. Moderation analysis adds that the impact of justification on argument repertoire is moderated by issue interest and need for cognition (Table 1). Participants in justified conditions develop larger and more elaborate argument repertoires, if they are interested in the issue and have a moderate to high need for cognition. These findings support H1d and H1e. Table 1 Conditional Effects of Justification on Argument Repertoire at Specific Levels of Issue Interest and Need for Cognition Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 Table 1 Conditional Effects of Justification on Argument Repertoire at Specific Levels of Issue Interest and Need for Cognition Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Issue interest Need for cognition Argument repertoire Count Articulation Low Low 0.20 0.40 Low Mean 0.12 0.45 Low High 0.05 0.49 Mean Low 0.42 0.84 Mean Mean 0.35* 0.89** Mean High 0.27 0.93* High Low 0.65* 1.28* High Mean 0.57* 1.32** High High 0.49* 1.37** Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 For a final test of the hypotheses, we integrate the mediator and moderators in a model of moderated mediation. Qualifying the previous findings, we find that justification has the predicted indirect effect on argument repertoire, but only when issue interest is moderate to high, and when need for cognition is high (Table 2). These findings confirm H1b, H1c, H1d, and H1e. But the mediating role of elaboration is far from perfect. There still is, in the final model, a significant direct effect of justification on argument repertoire (confirming H1a). Table 2 Conditional Indirect Effects of Justification on Argument Repertoire, Via Elaboration, at Specific Values of Issue Interest and Need for Cognition and Direct Effects Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. Number of bootstraps samples: 1000. Level of confidence: 0.95. LLCI = loser limit confidence interval; ULCI = upper limit confidence interval. Table 2 Conditional Indirect Effects of Justification on Argument Repertoire, Via Elaboration, at Specific Values of Issue Interest and Need for Cognition and Direct Effects Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Issue interest Need for cognition Argument repertoire Count Articulation B Boot LLCI Boot ULCI B Boot LLCI Boot ULCI Low Low 0.036 −0.152 0.223 0.080 −0.333 0.482 Low Mean 0.095 −0.074 0.271 0.210 −0.186 0.582 Low High 0.153 −0.075 0.376 0.341 −0.173 0.837 Mean Low 0.044 −0.107 0.194 0.097 −0.243 0.423 Mean Mean 0.102 −0.004 0.212 0.227 −0.012 0.462 Mean High 0.161 0.003 0.320 0.358 0.006 0.710 High Low 0.052 −0.154 0.259 0.114 −0.352 0.580 High Mean 0.110 −0.042 0.267 0.244 −0.099 0.593 High High 0.169 0.002 0.351 0.375 0.003 0.784 Direct effects 0.232 −0.005 0.513 0.631 0.046 1.122 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. Number of bootstraps samples: 1000. Level of confidence: 0.95. LLCI = loser limit confidence interval; ULCI = upper limit confidence interval. Debate Climate and Argument Repertoire Bias We ran similar analyses to assess whether news items with a constructive debate climate reduce the pro-attitudinal bias in citizens' argument repertoire. A comparison of means shows the debate climate has no overall effect. Mean values for pro-attitudinal bias are not significantly different across debate climates, regardless of whether we look at argument repertoire count (F(1,410) = 0.06, p = .810) or articulation (F(1,410) = 0.06, p = .811). Table 3 presents the findings from the moderation analyses. These show that elaboration increases the pro-attitudinal bias of argument repertoire when two conditions are met: news items adopt an adversarial debate climate and participants have a moderate to low need to evaluate. The first condition is in line with H2b (although the emphasis is more on the bias-enhancing effect of an adversarial climate than a balancing effect of a constructive climate). Table 3 Conditional Effects of Elaboration on Argument Repertoire Bias at Specific Levels of Need to Evaluate and Debate Climate Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 Table 3 Conditional Effects of Elaboration on Argument Repertoire Bias at Specific Levels of Need to Evaluate and Debate Climate Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Need to evaluate Debate climate Pro-Attitudinal Bias of Argument Repertoire Count Articulation Low Adversarial 0.177** 0.321* Low Constructive 0.117 0.201 Mean Adversarial 0.146** 0.263* Mean Constructive 0.086 0.143 High Adversarial 0.115 0.205 High Constructive 0.058 0.085 Note: Cell entries are unstandardized regression coefficients; low = mean – 1 SD; high = mean + 1 SD. * p < .05. ** p < .01 The second condition contradicts H2a. In our sample, a high need to evaluate is associated with a better-balanced argument repertoire, not a stronger bias, as we expected on the basis of Nir (2011). To explore this interesting finding post hoc, we repeated some of Nir's analyses (results not reported in detail). Results show that in our sample especially citizens that both have a higher need for cognition and a higher need to evaluate tend to list fewer pro-attitudinal arguments. One tentative explanation is that these citizens, both for intellectual and evaluative reasons, are especially interested in counter-attitudinal arguments and may take like-minded arguments in news media coverage for granted. The implications of this for opinion quality need to be further explored. Discussion Our findings show that news media do contribute to public opinion quality, if they meet the normative requirements of deliberative theory. When news media provide evidence and substantive arguments to support the positions of actors in debates, citizens develop larger argument repertoires. They are able to provide more articulated reasons why one might be in favor or against a particular course of action. Especially the direct, informative effect of news media on argument repertoire is relatively strong. In addition, justification triggers more elaborate processing, but only for citizens who are already inclined to deliberate—because they have a high need for cognition and an interest in the issue. Third, elaboration contributes to a biased argument repertoire, especially for citizens with a moderate to lower need to evaluate, when the debate climate is adversarial. A constructive debate climate prevents this negative effect on opinion quality. Limitations Before discussing these findings, we review the limitations of our study. The experimental design enabled us to investigate in relative detail the impact of two characteristics of news media content quality on argument repertoire. But we do not know how many citizens would read the news item in normal (i.e., nonexperimental) circumstances. This reduces the external validity of our findings. Related, we acknowledge that our experiment informs us only about short-term effects (Gaines, Kuklinski, & Quirk, 2007) and that our findings might be message-, medium-, issue-, or country-specific (Slater, 1991). There is, therefore, a need for future studies to replicate our findings, to assess how long the established effects last after the experimental treatment, and to extend the analysis to other types of media, issues, and political systems. Second, there are limitations in the extent to which we can draw conclusions in terms of causality. Our experimental design allows us to conclude that justification causes some respondents (namely those with a high need to evaluate and issue interest) to elaborate more, and that justification causes the average respondent to offer more arguments. But the evidence for the relationship between the mediator and the outcome variable is only correlational, not causal. A similar argument applies to our second model. Again, we can conclude that under specific conditions (low to moderate need to evaluate, adversarial debate climate in news item), elaboration and bias are positively associated. More theoretical and empirical work is necessary before any causality can be inferred. Implications While keeping these limitations in mind, our findings have several interesting implications for the political communications literature. First, our study confirms that the impact of news media on opinion quality follows from the interaction between news media content, cognitive behavior, and issue interest (e.g., Elenbaas et al., 2014; Iyengar et al., 2009; Luskin, 1990). Second, our findings support other studies that argue that the emphasis of news media on conflicts may have negative consequences for society. Specifically, our study suggests that an emphasis on irreconcilable differences may hinder internal deliberation. Third, and most importantly, our study supports the normative claim that news media should meet the standard of deliberativeness. Justification and a constructive debate climate, two deliberative qualities of news media elaborated in this article, do matter. To support this claim, we have shown that justification contributes to an increase in argument repertoire, which in turn enables, though not necessarily implies, that opinions are based on better knowledge of conflicting positions and underlying arguments. We also noted that justification and a constructive debate climate have an activating and balancing effect, respectively. This suggests that news media can stimulate deliberation “from within,” if and when these media meet deliberative quality criteria. Combined, these findings provide tentative support for the idea that not only face-to-face but also mediated deliberation can improve the quality of opinions in society. We hope our study contributes to a growing body of literature in which this claim is better justified. Future Research Reflecting on our analysis, we recommend three areas for future research. First, as already indicated above, our findings need to be replicated in different contexts with different news media. One interesting factor to be investigated in this respect, we propose, is news media trust. News media trust varies considerably across media systems. It may be considered both as an outcome of the deliberative quality of news media and a moderator, affecting the impact of news media on opinion quality. Another interesting comparative question is to what extent media and story types, other than the informative online news item investigated here, may stimulate interest or trigger deliberation. What is the potential impact of personalization, of storytelling in journalism, of informative games, of televised documentaries? We see this as an important avenue for further research. Second, our study suggests to study the deliberative quality of existing news media content as well as to explore its determinants (such as competition, journalistic professionalism, public broadcasting, and news cultures). Because it matters whether media provide justifications and a constructive debate climate, we need to explore ways to stimulate news organizations to provide such content. A third promising area of research concerns the noncognitive aspects of opinion quality. Deliberative theory emphasizes the disinterested and rational weighing of arguments, in search for the best possible solution. We accordingly focused our study on media coverage and a cognitive measure of opinion quality. But in reality, engagement, (self-)interest, values, emotions, and political identities play a role in opinion formation, too. Following our theoretical argument, we propose in particular to further explore the notion of a constructive debate climate. The willingness of citizens to adopt a consensual orientation is a crucial component of democracy (Goodin, 2000). We see it as a challenging question for research to what extent a constructive debate climate in media, in interaction with personal characteristics (such as openness), emotions, and engagement, can stimulate citizens to deliberation “from within” and seriously consider the different arguments that quality media offer. Conclusions Research has consistently shown that public opinion does not meet normative democratic criteria, owing to both shortcomings in news media and personal failings. In this article, we propose that these arguments also imply that there are several ways to improve public opinion. Along these lines, our study suggests that when journalists (1) ensure that actors provide justifications for their positions, rather than mere claims, and (2) emphasize the search for common ground rather than fundamental differences, citizens might develop a larger and more balanced argument repertoire. This is an important building block for public opinion quality. These conclusions call, in our view, for more research on how we can strengthen the quality of news media in today's profit-oriented and highly competitive news media industry. Appendix: Overview of Measures Argument Repertoire “Suppose some-one you know is invited to participate in a debate on underground CO2 storage. This person wants to prepare well, but knows little about the issue. He asks you about the arguments. What would you answer? What are the reasons you or other people may have to be in favour of [against] underground CO2 storage. Please list all the reasons in favour of [against] underground CO2 storage that come to mind.” Elaboration “To what extent do you agree or disagree with the following statements?” (1 = disagree completely to 7 = agree completely). “I paid attention to the article; My mind wandered while I was reading the news article (reversed); The article made me think.” Need to Evaluate Same question and scale as above. "I form opinions about everything; It is very important to me to hold strong opinions; It bothers me to remain neutral; I would rather have a strong opinion than no opinion at all; I pay a lot of attention to whether things are good or bad." Need for Cognition Same question and scale as above. "I like to have the responsibility of handling a situation that requires a lot of thinking; I really enjoy a task that involves coming up with new solutions to problems; I would prefer a task that is intellectual, difficult, and important to one that is somewhat important but does not require much thought; I find satisfaction in deliberating hard and for long hours." Issue Interest "Different people consider different issues important. What about you? Which of the following issues [Underground CO2 storage; Climate change] is very important to you personally and which not at all?" (1 = is not at all important to me personally to 7 = is very important to me personally). Acknowledgments We thank two anonymous reviewers and the editors for their valuable comments. References Aday S. ( 2006 ). The framesetting effects of news: An experimental test of advocacy versus objectivist frames . Journalism and Mass Communication Quarterly , 83 , 767 – 784 . doi: 10.1177/107769900608300403 Google Scholar CrossRef Search ADS Andersen V. N. , Hansen K. M. ( 2007 ). How deliberation makes better citizens: The Danish Deliberative Poll on the euro . European Journal of Political Research , 46 , 531 – 556 . doi: 10.1111/j.1475-6765.2007.00699.x Google Scholar CrossRef Search ADS Beaudoin C. E. ( 2008 ). The internet’s impact on international knowledge . New Media and Society , 10 , 455 – 474 . doi: 10.1177/1461444807085327 Google Scholar CrossRef Search ADS Beaudoin C. E. , Thorson E. ( 2004 ). Testing the cognitive mediation model: The roles of news reliance and three gratifications sought . Communication Research , 31 , 446 – 471 . doi: 10.1177/0093650204266098 Google Scholar CrossRef Search ADS Bizer G. Y. , Krosnick J. A. , Holbrook A. L. , Wheeler S. C. , Rucker D. D. , Petty R. E. ( 2004 ). The impact of personality on cognitive, behavioral, and affective political processes: The effects of need to evaluate . Journal of Personality , 72 , 995 – 1027 . doi: 10.1111/j.0022-3506.2004.00288.x Google Scholar CrossRef Search ADS Blanton H. , Strauts E. , Perez M. ( 2012 ). Partisan identification as a predictor of cortisol response to election news . Political Communication , 29 , 447 – 460 . doi: 10.1080/10584609.2012.736239 Google Scholar CrossRef Search ADS Borah P. ( 2011 ). Seeking more information and conversations: Influence of competitive frames and motivated processing . Communication Research , 38 , 303 – 325 . doi: 10.1177/0093650210376190 Google Scholar CrossRef Search ADS Burkhalter S. , Gastil J. , Kelshaw T. ( 2002 ). A conceptual definition and theoretical model of public deliberation in small face-to-face groups . Communication Theory , 12 , 398 – 422 . doi: 10.1111/j.1468-2885.2002.tb00276.x Cacioppo J. , Petty R. ( 1982 ). The need for cognition . Journal of Personality , 42 , 116 – 131 . doi: 10.1037/0022-3514.42.1.116 Cappella J. N. , Price V. , Nir L. ( 2002 ). Argument repertoire as a reliable and valid measure of opinion quality: Electronic dialogue during campaign 2000 . Political Communication , 19 , 73 – 93 . doi: 10.1080/105846002317246498 Google Scholar CrossRef Search ADS Cho J. ( 2013 ). Campaign tone, political affect, and communicative engagement . Journal of Communication , 63 , 1130 – 1152 . doi: 10.1111/jcom.12064 Google Scholar CrossRef Search ADS Chong D. , Druckman J. N. ( 2007a ). Framing public opinion in competitive democracies . American Political Science Review , 101 , 637 – 655 . doi: 10.1017/S0003055407070554 Google Scholar CrossRef Search ADS Chong D. , Druckman J. N. ( 2007b ). Framing theory . Annual Review of Political Science , 10 , 103 – 126 . doi: 10.1146/annurev.polisci.10.072805.103054 Google Scholar CrossRef Search ADS Cottle S. ( 2006 ). Between display and deliberation: Analyzing TV news as communicative architecture . Media, Culture and Society , 28 , 163 – 189 . doi: 10.1177/0163443706061680 Google Scholar CrossRef Search ADS Curran J. , Iyengar S. , Brink Lund A. , Salovaara-Moring I. ( 2009 ). Media system, public knowledge and democracy: A comparative study . European Journal of Communication , 24 , 5 – 26 . doi: 10.1177/0267323108098943 Google Scholar CrossRef Search ADS David C. C. ( 2009 ). Learning political information from the news: A closer look at the role of motivation . Journal of Communication , 59 , 243 – 261 . doi: 10.1111/j.1460-2466.2009.01414.x Google Scholar CrossRef Search ADS Delli Carpini M. X. , Cook F. L. , Jacobs L. R. ( 2004 ). Public deliberation, discursive participation, and citizen engagement: A review of the empirical literature . Annual Review of Political Science , 7 , 315 – 344 . doi: 10.1146/annurev.polisci.7.121003.091630 Google Scholar CrossRef Search ADS Druckman J. N. ( 2001 ). The implications of framing effects for citizen competence . Political Behavior , 23 , 225 – 256 . doi: 10.1023/A:1015006907312 Google Scholar CrossRef Search ADS Druckman J. N. ( 2012 ). The politics of motivation . Critical Review , 24 , 199 – 216 . doi: 10.1080/08913811.2012.711022 Google Scholar CrossRef Search ADS Druckman J. N. ( 2014 ). Pathologies of studying public opinion, political communication, and democratic responsiveness . Political Communication , 31 , 467 – 492 . doi: 10.1080/10584609.2013.852643 Google Scholar CrossRef Search ADS Druckman J. N. , Bolsen T. ( 2011 ). Framing, motivated reasoning, and opinions about emergent technologies . Journal of Communication , 61 , 659 – 688 . doi: 10.1111/j.1460-2466.2011.01562.x Google Scholar CrossRef Search ADS Elenbaas M. , de Vreese C. , Schuck A. , Boomgaarden H. ( 2014 ). Reconciling passive and motivated learning: The saturation-conditional impact of media coverage and motivation on political information . Communication Research , 41 , 481 – 504 . doi: 10.1177/0093650212467032 Google Scholar CrossRef Search ADS European Commission . ( 2011 ). Special Eurobarometer 364. Public awareness and acceptance of CO2 capture and storage (Wave 75.1) . Brussels : European Commission . Retrieved from http://ec.europa.eu/public_opinion/archives/ebs/ebs_364_en.pdf Eveland W. P. ( 2001 ). The cognitive mediation model of learning from the news: Evidence from nonelection, off-year election, and presidential election contexts . Communication Research , 28 , 571 – 601 . doi: 10.1177/009365001028005001 Google Scholar CrossRef Search ADS Ferree M. M. , Gamson W. A. , Gerhards J. , Rucht D. ( 2002 ). Four models of the public sphere in modern democracies . Theory and Society , 31 , 289 – 324 . doi: 10.1023/A:1016284431021 Google Scholar CrossRef Search ADS Fischer P. , Greitemeyer T. , Frey D. ( 2008 ). Self-regulation and selective exposure: The impact of depleted self-regulation resources on confirmatory information processing . Journal of Personality and Social Psychology , 94 , 382 – 395 . doi: 10.1037/0022-3514.94.3.382 Google Scholar CrossRef Search ADS Fleming K. , Thorson E. ( 2008 ). Assessing the role of information-processing strategies in learning from local news media about sources of social capital . Mass Communication and Society , 11 , 398 – 419 . doi: 10.1080/15205430801950643 Google Scholar CrossRef Search ADS Fraile M. , Iyengar S. ( 2014 ). Not all news sources are equally informative: A cross-national analysis of political knowledge in Europe . The International Journal of Press/Politics , 19 , 275 – 294 . doi: 10.1177/1940161214528993 Google Scholar CrossRef Search ADS Gaines B. J. , Kuklinski J. H. , Quirk P. J. ( 2007 ). The logic of the survey experiment reexamined . Political Analysis , 15 , 1 – 20 . doi: 10.1093/pan/mpl008 Google Scholar CrossRef Search ADS Goodin R. E. ( 2000 ). Democratic deliberation within . Philosophy and Public Affairs , 29 , 81 – 109 . doi: 10.1111/j.1088-4963.2000.00081.x Google Scholar CrossRef Search ADS Graber D. ( 2003 ). The media and democracy: Beyond myths and stereotypes . Annual Review of Political Science , 6 , 139 – 160 . doi: 10.1146/annurev.polisci.6.121901.085707 Google Scholar CrossRef Search ADS Hart P. S. , Nisbet E. C. ( 2012 ). Boomerang effects in science communication: How motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies . Communication Research , 39 , 701 – 723 . doi: 10.1177/0093650211416646 Google Scholar CrossRef Search ADS Hayes A. F. ( 2013 ). Introduction to mediation: A regression-based approach. New York, NY : Guilford Publications . Hermans D. ( 1997 ). N.E.S. – Dutch version. Geautoriseerde Nederlandse vertaling van de Need to Evaluate Scale. Retrieved from http://www.ekgp.ugent.be/pages/nl/vragenlijsten/NES.pdf Holbert R. L. , Garrett R. K. , Gleason L. S. ( 2010 ). A new era of minimal effects? A response to Bennett and Iyengar . Journal of Communication , 60 , 15 – 34 . doi: 10.1111/j.1460-2466.2009.01470.x Google Scholar CrossRef Search ADS Ho S. S. , Peh X. , Soh V. W. L. ( 2013 ). The cognitive mediation model: Factors influencing public knowledge of the H1N1 pandemic and intention to take precautionary behaviors . Journal of Health Communication , 18 , 773 – 794 . doi: 10.1080/10810730.2012.743624 Google Scholar CrossRef Search ADS Iyengar S. , Hahn K. S. , Bonfadelli H. , Marr M. ( 2009 ). “Dark areas of ignorance” revisited: Comparing international affairs knowledge in Switzerland and the United States . Communication Research , 36 , 341 – 358 . doi: 10.1177/0093650209333024 Google Scholar CrossRef Search ADS Jensen J. D. ( 2011 ). Knowledge acquisition following exposure to cancer news articles: A test of the cognitive mediation model . Journal of Communication , 61 , 514 – 534 . doi: 10.1111/j.1460-2466.2011.01549.x Google Scholar CrossRef Search ADS Kim S.-H. , Scheufele D. A. , Shanahan J. , Choi D.-H. ( 2011 ). Deliberation in spite of controversy? News media and the public’s evaluation of a controversial issue in South Korea . Journalism and Mass Communication Quarterly , 88 , 320 – 336 . doi: 10.1177/107769901108800206 Google Scholar CrossRef Search ADS Kunda Z. ( 1990 ). The case for motivated reasoning . Psychological Bulletin , 108 , 480 – 98 . doi:10.1037/0033-2909.108.3.480 Google Scholar CrossRef Search ADS Luskin R. C. ( 1990 ). Explaining political sophistication . Political Behavior , 12 , 331 – 361 . doi: 10.1007/BF00992793 Google Scholar CrossRef Search ADS Luskin R. C. , Fishkin J. S. , Jowell R. ( 2002 ). Considered opinions: Deliberative polling in Britain . British Journal of Political Science , 32 , 455 – 487 . doi: 10.1017/S0007123402000194 Google Scholar CrossRef Search ADS Manosevitch E. ( 2009 ). The reflective cue: Prompting citizens for greater consideration of reasons . International Journal of Public Opinion Research , 21 , 187 – 203 . doi:10.1093/ijpor/edp013 Google Scholar CrossRef Search ADS Mutz D. C. , Martin P. S. ( 2001 ). Facilitating communication across lines of political difference: The role of mass media . American Political Science Review , 95 , 97 – 114 . Nir L. ( 2011 ). Motivated reasoning and public opinion perception . Public Opinion Quarterly , 75 , 504 – 532 . doi: 10.1093/poq/nfq076 Google Scholar CrossRef Search ADS Petty R. E. , Briñol P. , Priester J. R. ( 2008 ). Mass media attitude change: Implications of the elaboration likelihood model of persuasion. In Bryant J. , Oliver M. B. (Eds.), Media effects: Advances in theory and research (pp. 125 – 164 ). New York, NY : Taylor & Francis . Pieters R. G. M. , Verplanken B. , Modde J. M. ( 1987 ). “Neiging tot nadenken”: Samenhang met beredeneerd gedrag . Nederlands Tijdschrift Voor de Psychologie , 42 , 62 – 70 . Preacher K. J. , Rucker D. D. , Hayes A. F. ( 2007 ). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions . Multivariate Behavioral Research , 42 , 185 – 227 . doi: 10.1080/00273170701341316 Google Scholar CrossRef Search ADS Price V. , Cappella J. , Nir L. ( 2002 ). Does disagreement contribute to more deliberative opinion? Political Communication , 19 , 95 – 112 . doi: 10.1080/105846002317246506 Google Scholar CrossRef Search ADS Price V. , Neijens P. ( 1997 ). Opinion quality in public opinion research . International Journal of Public Opinion Research , 9 , 336 – 360 . doi: 10.1093/ijpor/9.4.336 Google Scholar CrossRef Search ADS Ryfe D. M. ( 2005 ). Does deliberative democracy work? Annual Review of Political Science , 8 , 49 – 71 . doi: 10.1146/annurev.polisci.8.032904.154633 Google Scholar CrossRef Search ADS Schuck A. , Vliegenthart R. , Boomgaarden H. G. , Elenbaas M. , Azrout R. , van Spanje J. , Vreese C. H. D. ( 2013 ). Explaining campaign news coverage: How medium, time, and context explain variation in the media framing of the 2009 European parliamentary elections . Journal of Political Marketing , 12 , 8 – 28 . doi: 10.1080/15377857.2013.752192 Google Scholar CrossRef Search ADS Shiffrin R. M. , Schneider W. ( 1977 ). Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory . Psychological Review , 84 , 127 – 190 . doi: 10.1037/0033-295X.84.2.127 Google Scholar CrossRef Search ADS Slater M. D. ( 1991 ). Use of message stimuli in mass communication experiments: A methodological assessment and discussion . Journalism Quarterly , 68 , 412 – 421 . doi: 10.1177/107769909106800312 Google Scholar CrossRef Search ADS Stromer-Galley J. ( 2007 ). Measuring deliberation’s content: A coding scheme . Journal of Public Deliberation , 3 ( 1 ), 1 – 35 . http://services.bepress.com/jpd/vol3/iss1/art12 Stroud N. J. , Muddiman A. ( 2013 ). Selective exposure, tolerance, and satirical news . International Journal of Public Opinion Research , 25 , 271 – 290 . doi: 10.1093/ijpor/edt013 Google Scholar CrossRef Search ADS Taber C. , Lodge M. ( 2006 ). Motivated skepticism in the evaluation of political beliefs . American Journal of Political Science , 50 , 755 – 769 . doi: 10.1111/j.1540-5907.2006.00214.x Google Scholar CrossRef Search ADS Van der Wurff R. , Verhoeven P. , Gadellaa M. ( 2013 ). Scientists and deliberativeness of European public television news . JCOM , 12 , A02 . Wessler H. ( 2008 ). Investigating deliberativeness comparatively . Political Communication , 25 , 1 – 22 . doi: 10.1080/10584600701807752 Google Scholar CrossRef Search ADS Wessler H. , Rinke E. M. ( 2014 ). Deliberative performance of television news in three types of democracy: Insights from the United States, Germany, and Russia . Journal of Communication , 64 , 827 – 851 . doi: 10.1111/jcom.12115 Google Scholar CrossRef Search ADS Winter S. , Krämer N. C. ( 2012 ). Selecting science information in web 2.0: How source cues, message sidedness, and need for cognition influence users’ exposure to blog posts . Journal of Computer-Mediated Communication , 18 , 80 – 96 . doi: 10.1111/j.1083-6101.2012.01596.x Google Scholar CrossRef Search ADS © The Author 2016. Published by Oxford University Press on behalf of The World Association for Public Opinion Research. All rights reserved. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

International Journal of Public Opinion ResearchOxford University Press

Published: Aug 30, 2016

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off