Kaisa Herne
Josefina Sipinen1- 1Politics, Faculty of Management and Business, Tampere University, Tampere, Finland
- 2Department of Political Science, Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Turku, Finland
We study the influence of argument sources on argument quality evaluations. Argument source refers to the person who present the argument. We ask whether partial, impartial and reluctant sources generate different evaluations of argument quality. We explore the source effect via a survey experiment where participants are asked to evaluate the quality of political arguments. Previous research on source partiality mainly concerns persuasion. The results from these studies suggest that source characteristics such as expertise and trustworthiness affect the persuasiveness of communication. Both impartial and reluctant sources have been observed to promote persuasion, whereas partial sources tend to hinder it. However, the evidence on the difference between impartial and reluctant sources are inconclusive, and research on argument quality evaluations is scarce. In our study, respondents are randomly allocated into four conditions according to who presents a political argument: Control (no argument source); Partial Source, Reluctant Source and Impartial Source. Our results show that overall impartial sources give rise to higher evaluations of argument quality in comparison to reluctant sources, but not in comparison to partial sources. Furthermore, reluctant sources are also perceived low in credibility.
Introduction
It happened that a Fox caught its tail in a trap, and in struggling to release himself lost all of it but the stump. At first he was ashamed to show himself among his fellow foxes. But at last he determined to put a bolder face upon his misfortune, and summoned all the foxes to a general meeting to consider a proposal which he had to place before them. When they had assembled together the Fox proposed that they should all do away with their tails. He pointed out how inconvenient a tail was when they were pursued by their enemies, the dogs; how much it was in the way when they desired to sit down and hold a friendly conversation with one another. He failed to see any advantage in carrying about such a useless encumbrance. “That is all very well,” said one of the older foxes; “but I do not think you would have recommended us to dispense with our chief ornament if you had not happened to lose it yourself.”
Moral of Aesop’s Fable: “Distrust Interested Advice"
Assessment of arguments characterizes political communication–own arguments are typically praised as strong, whereas opponents’ arguments are claimed to be weak. Existing evidence suggests that lay people can distinguish strong arguments from weak ones (Hoeken et al., 2012, 2014), and also that strong arguments tend to be more persuasive than weak arguments (Johnson et al., 2004; Park et al., 2007). Argument sources may provide essential information that helps to evaluate arguments. Especially expertise and trustworthiness of the source can give important hints about the likelihood that the presented argument provides correct and relevant information. But how do citizens evaluate the quality of arguments that different types of sources present? Do these evaluations depend on the characteristics of the source? The ability to evaluate sources is likely to get more and more relevant because of a still increasing use of the internet and social media as sources for political information (Newman et al., 2020).
We study one aspect of the argument source, namely his or her partiality. We ask how impartial, partial and reluctant sources influence the evaluation of argument quality? To study this question, we conducted a survey experiment where respondents evaluated political arguments presented by different sources. Respondents were randomly assigned into experimental treatment conditions that varied the partiality of the source. We use two types of data, the first is collected form a representative pool of adult citizens, whereas the second is collected from university students. Our results suggest that the null hypothesis of no impact of the argument source can be rejected, even though the source effect varied depending on the specific argument. In our data, an impartial source generated higher evaluations of argument quality compared to reluctant sources, but not compared to partial sources. Source credibility was also linked to the source’s perceived partiality, and reluctant sources were deemed specifically uncredible.
Framework for the Message Source Effect
Theoretical Background
We understand an argument as a statement that includes a claim and a justification for the claim. An argument is formulated to give support to the claim which is used either to establish a matter or to persuade others (Chittleborough and Newman 1993). Argument quality varies depending on the connection between the claim and its justification as well as on the content of the justification. We study how lay people evaluate argument quality. Moreover, we define argument source as a person who presents an argument. The source can be characterized by several criteria, for example, expertise, trustworthiness, attractiveness, power, status or similarity. Our interest lies in the partiality of the source, but we will also measure source credibility, i.e. the combination of expertise and trustworthiness.
An impartial, or unbiased, source advocates a policy that does not promote his or her vested interests, and neither is against them, i.e., he or she is not a stakeholder in the issue. Stakeholders’ interests, on the other hand, are influenced by proposed policies, which can be either in accordance with or in conflict with stakeholders’ interests. We call stakeholders who present arguments that serve their own interests partial, and stakeholders who present arguments that are in conflict with their interests reluctant. It is noteworthy that while we can conceptually categorize sources according to their partiality, empirically differences in partiality are better characterized as ends in a continuum. It should also be noted that we use impartial as a synonym of unbiased. A partial source is the same as biased, but a reluctant source is harder to characterize with these terms because he or she testifies against his or hers own biased interests.
Partiality is closely connected to source credibility which has two dimensions, expertise and trustworthiness (O’Keefe, 2002; Pornpitakpan, 2004; Lee, 2005). Expertise refers to the source’s likelihood of knowing what is true, whereas trustworthiness refers to the source’s motivation for telling the truth. Expertise thereby means an ability to provide correct information, whereas trustworthiness means an intention to do so. We may hold a doctor a competent source of messages that concern health, and thereby an expert, but if the doctor has been caught for lying in the past, we may question his or her trustworthiness. Source credibility is likely to be associated to source partiality. Compared to a partial source, an impartial source is likely to be seen more credible because of a lack of self-interested reasons to deliver a message. A reluctant source does not have self-interested reasons to deliver a message but he or she may give rise to suspicion about the true motives for delivering the message. We will also study the association between partiality and credibility.
The widely used elaboration likelihood model (ELM) (Petty and Cacioppo, 1984, 1986), provides an interpretation of why sources might influence argument evaluations. While an influential model, ELM, as well as other dual process models (Chaiken and Maheswaran, 1994; Lee, 2005), have also been questioned (Kruglanski and Thompson, 1999; Kruglanski et al., 2006). However, we are not mainly interested in finding evidence for or against ELM but rather present it as a framework for understanding why message sources may influence message evaluations. ELM states that variation in the success of persuasion depends on the likelihood that a receiver of a message will go through a thoughtful processing of information. ELM distinguishes two routes individuals can take when they evaluate messages. The central route is characterized by extensive and effortful information processing with the aim of uncovering the central merits of the message. The peripheral route, in turn, is governed by non-issue-relevant concerns, also called persuasion cues or cognitive short cuts. The two routes should not be understood as mutually exclusive but rather two ends of a continuum, which may also influence message evaluation simultaneously (Benoit and Kennedy, 1999).
Which one of the routes has more influence on message evaluation depends on issue involvement. When a decision-maker conceives an issue as important, he or she is likely to reflect more carefully on the issue and invest more resources to the evaluation of the content of the message. With low issue involvement less resources are invested, and heuristic cues are more likely to influence evaluation. In addition to issue involvement, a decision-maker’s ability to evaluate the message, e.g., cognitive sophistication, distraction, and time constraints, may influence which type of processing is used (Petty and Cacioppo, 1984; Lee, 2005).
According to ELM, message source characteristics, such as likeability, attractiveness, similarity, credibility, and partiality, can serve as cues about the quality of the message. In other words, when people are unable or unmotivated to thoughtfully consider the strengths and weaknesses of a message, they tend to rely on heuristic cues such as source characteristics. When people are motivated for a thorough evaluation, on the other hand, messages that contain arguments that are high in quality tend to be more persuasive than weak ones.
Existing Evidence on the Message Source Influence
A majority of studies on the influence of the message source focus on persuasion, i.e., generation of attitudes or choices that are congruent with the message content. Most studies on persuasion examine the effects of source credibility, or one of its dimensions, expertise or trustworthiness. A rather robust observation is that source credibility induces persuasion (Chebat et al., 1990; Clark et al., 2012; Hawkins et al., 2019; Hovland and Weiss, 1951; Kim and Benbasat, 2009; López and Sicilia, 2014; Petty et al., 1981; Pornpitakpan, 2004; Smith et al., 2005; Tormala et al., 2006). Beyond persuasion, source expertise has been observed to increase perceived information quality (Yi et al., 2013); and source trustworthiness to generate higher evaluations of argument quality (Hahn et al., 2009); as well as belief in conditional premises (Wolf et al., 2012). There is also evidence that certain factors tend to moderate the influence of source credibility. These factors include source’s similarity with the recipient, argument strength, and congruence with the source’s self-interest or with the subject’s attitudes (Clark et al., 2012; Kim and Benbasat, 2009; Pornpitakpan, 2004; Tormala et al., 2006). These studies suggest that, in accordance with ELM, a motivation to carefully scrutinize a message may reduce the influence of the source (Petty et al., 1981; Petty and Cacioppo, 1984). However, support for ELM is not robust in this respect because in some studies involvement has not moderated the effect of the source (Chebat et al., 1990).
With regards to source partiality, the rudimentary assumption is that partial sources are less persuasive than impartial and reluctant sources because people realize that self-interest may motivate partial sources to deliver the message. This assumption has got wide empirical support: partial sources are inferior in their power to persuade compared to both impartial and reluctant sources (Benoit and Kennedy, 1999; Kim and Benbasat, 2009; McPeek and Edwards, 1975; Noteberg et al., 2003; Walster et al., 1966).1 Following the same logic, reluctant sources should be even more persuasive than impartial sources because they deliver a message that is against their self-interest, and should not have private motivations to deliver the message. However, existing evidence fails to detect a significant difference between reluctant and impartial sources regarding their effect on persuasion (Benoit and Kennedy, 1999). A similar pattern is observed regarding the credibility of partial, impartial and reluctant sources. Evidence on the inferiority of partial sources is rather robust (Arnold and McCroskey, 1967; Anderson, 1970; Benoit and Kennedy, 1999; Callison, 2004), but the difference between impartial and reluctant sources is ambiguous. Sometimes impartial testimony produces higher credibility ratings than reluctant testimony (Arnold and McCroskey, 1967; McCroskey, 1969; Stubb and Colliander, 2019), whereas one study observes no difference between impartial and reluctant sources concerning trustworthiness (Benoit and Kennedy, 1999). An overall conclusion on the influence of source partiality seems to be that both impartial and reluctant sources outperform partial sources in terms of persuasion and credibility, whereas there is no robust evidence concerning the difference between reluctant and impartial sources regarding their persuasiveness nor credibility.
Regarding lay peoples’ evaluations of argument quality, evidence shows that lay people are able to distinguish high and low quality arguments (Hoeken et al., 2012, 2014), and that high quality arguments are more persuasive than low quality arguments (a review of results in Hoeken et al., 2012; Park et al., 2007). Hoeken et al., 2012, Hoeken et al., 2014 also demonstrate that arguments presented by sources with vested interests in the issue, i.e., partial sources, are less convincing than sources with no vested interests, i.e., impartial sources. Moreover, Birnbaum and Mellers (1983) show that impartial and expert sources have more influence on belief revision compared to non-expert and partial sources.
We build in particular on Benoit and Kennedy (1999) who compare the persuasiveness of partial, impartial and reluctant sources in political communication, and on Hoeken et al. (2012), Hoeken et al. (2014) who study argument quality evaluations and compare partial and impartial sources. We study how argument quality evaluations depend on the source who presents the argument, whether he or she is partial, impartial or reluctant. We test arguments that make policy proposals typical of political communication taking place in different fora.
We focus on argument quality evaluations because lay people’s ability to evaluate argument quality is becoming more and more important in today’s political communication where information is often coming from rather unreliable sources. For many people, social media is an important channel to get information about politics, and despite some fact checking, a major part of information in the social media goes unchecked. It is therefore important that people can critically evaluate message contents, message sources and the connection between the message and the source, which may deliver important information about the quality of the message. In political communication, the ultimate interest often lies in the ability of arguments to persuade people. It is possible that sources influence persuasion via argument quality evaluations. However, there is also evidence that consistency between the source and argument quality may influence how carefully people scrutinize arguments, which may in turn influence persuasion (Ziegler et al., 2002). A dishonest expert source is likely to generate a more thorough evaluation of argument content than an honest expert source because of the inconsistency between source characteristics in the first case. We leave to future research to examine the influence of the source and argument quality on political persuasion and focus here only on the connection between a source and evaluated argument quality.
To summarize, with respect to existing literature our contribution lies in comparing the influence of sources representing each of the three partiality types. Moreover, we study a range of arguments that make a policy proposal, and a range of sources, which both increase the generalizability of our results. In addition to student data we use population-based data to increase the generalizability of our results.
Hypotheses
Based on ELM and existing empirical evidence we expect that the partiality of the source will influence the evaluations of argument quality. Existing evidence concerning partial sources is rather robust: both impartial and reluctant sources are more persuasive than partial sources, and argument quality is rated higher when an impartial rather than a partial source presents the argument. Existing evidence regarding the difference between impartial and reluctant sources is more ambiguous. In theory, if partial sources lack influence because they testify according to their self-interest, reluctant sources should be most influential because they testify against their self-interest. Testifying against own interests could be conceived as an indication that the presented message is correct because of a lack of other motivations to deliver the message, in fact, the source has a motivation not to deliver the message. However, existing empirical evidence suggest that reluctant sources are not particularly effective in persuasion. Reluctant sources can give rise to doubt or uncertainty because they are perceived untrustworthy or surprising (Briñol and Petty 2009). This may influence argument quality evaluations directly, but it is also possible that inconsistency between the argument and the source generates a more thorough scrutiny of the argument (Ziegler et al., 2002). In the latter case, argument quality evaluations would be more likely to be influenced by the content of the argument, indicating a smaller effect of the argument source.
We formulate our first hypothesis on the basis of the robust evidence on partial sources in terms of persuasion. Regarding the difference between impartial and reluctant sources, evidence is mixed. However, since there is evidence that impartial sources are perceived more credible than reluctant sources, and since reluctant sources may give rise to doubt and uncertainty, we assume that impartial sources also generate higher evaluations of argument quality compared to reluctant sources.
H1 Impartial sources give rise to higher argument quality evaluations compared to partial and reluctant sources.
While evidence on the predictive power of ELM is not unambiguous and alternative models have been presented, we formulate a hypothesis on the moderating effect of issue involvement. In other words, when an issue is important for a respondent, he or she tends to scrutinize argument content carefully. In that case the relevant aspects of the argument, that is, argument content should mainly influence argument quality evaluations, instead of more irrelevant cues such as argument source.
H2 The argument source effect interacts with issue involvement so that when issue involvement is high, sources do not influence argument quality evaluations.
Following Benoit and Kennedy (1999), we assume that the influence of the source partiality goes via credibility. On one hand, existing evidence shows rather unequivocally that presenting an argument that serves one’s own interests is likely to undermine the source’s perceived credibility, whereas impartial sources are seen credible. On the other hand, evidence on the perceived credibility of reluctant sources seems inconclusive. According to Benoit and Kennedy (1999), argumentation theory conceives reluctant sources most credible because they cannot be motivated by self-interest, whereas social science literature conceives impartial sources most credible. We see it likely that if the characteristics of the source and the message he or she delivers are highly incompatible, e.g., an animal rights activist promoting fur production or a company manager promoting higher taxes on business profits, people may experience doubt, surprise and related cognitive dissonance (Festinger, 1957). Dissonance occurs because the source is presenting an argument he or she is not expected to present. Based on previous empirical results on source credibility and potential cognitive dissonance related to reluctant sources, we formulate hypothesis H3. It is noteworthy that concerning reluctant sources our expectation is again more tentative.
H3 Impartial sources are evaluated as more credible compared to partial or reluctant sources.
We do not have specific assumptions about the control condition with no argument source, but we pose a research question on the difference between evaluations of argument quality when an argument source is mentioned compared to when it is not.
Materials and Method
We conducted a survey experiment to test our hypotheses. In the experiment, participants filled in surveys where they first evaluated the quality of two arguments and thereafter responded to a set of questions related to the argument source. Participants were randomly allocated into conditions that varied the description of the person who presented the argument.
Participants
We use two separate data sets.2 The first is based on a representative sample of Finns (n = 1,600) who filled in the survey online in June 2018.3 The sample was representative in terms of age, gender and residential area, and it was based on a panel of respondents. In this sample, a half of respondents (n = 800) evaluated arguments A1 and A2, and another half (n = 800) evaluated arguments A3 and A4.4 To get a larger variance of argument source types, we collected another data based on a sample of students (n = 225) who participated in an unrelated choice experiment in a decision-making laboratory at the (University of Turku, Finland) in September 2018.5 After completing incentivized choice tasks, participants filled in the argument evaluation survey. Student participants were recruited through an existing panel of volunteers maintained for the recruitment of choice experiment participants. Student participants evaluated arguments 5–6. This yields three different surveys: the first contained arguments 1–2; the second arguments 3–4; and the third arguments 5–6.
Design and Experimental Procedures
We administered the three treatment conditions and a control in the representative sample. To get more observations per condition, the control condition was not administered in the student data. In the Control condition, arguments were presented without an argument source. In the impartial source condition, the argument source was impartial with respect to the presented argument. In the partial source condition, the source presented an argument that was in accordance with the interests of the source, and in the reluctant source condition the source presented an argument that was against his or her interests.
We formulated five different arguments that include a policy proposal and its justification.6 The arguments covered the following topics: protection against employee dismissals, wind power, income distribution, policies to restrict the number of wolves, and refugee policies. All arguments concern topics that have been widely debated in Finland. For example, wolf policies, which directly concern only a limited number of people in certain rural areas, have been regularly covered in the media. All arguments are roughly equal in length, they are not too difficult to understand, e.g., they do not contain difficult terms, and each provides only one justification for the policy proposal. Providing only one justification indicates that the arguments are not particularly strong. Argument sources varied in terms of their connection to the presented argument so that the promoted policy either served the interests of the source (partial sources), were against them (reluctant sources), or the interests of the source were not affected (impartial sources). To create these types, we varied things such as sources’ occupation, party affiliation and income level. The gender of the source was not revealed. The arguments and their partial, impartial and reluctant sources are presented in Table 1.
TABLE 1. The arguments and partial, impartial and reluctant sources.
We had several goals in formulating the argument-source pairs. We tried to formulate arguments that are typically presented in political communication and are topical in Finnish political discussion and we connected arguments to sources that could realistically present the argument. For example, an environmentalist is not likely to argue against the use of wind power, and if she does, in addition to being a reluctant source, she would also be a highly unrealistic person to make such a claim. To avoid highly unrealistic sources, we varied the expertise of the source along with partiality in three cases. In arguments A1, A2 and A4, the professional expertise of the source was not held constant, whereas in arguments A3, A5 and A6 it was. In the case of A3, sources were described as persons, whereas in the case of A5 and A6 sources were described as members of the parliament. Partiality was varied by describing the income level of the source (A3), or his or her party affiliation (A5 and A6). Members of parliament represent four parties: the Finns Party is a right-wing anti-immigration populist party, the National Coalition Party is a center-right urban party, the Green League is an environmentalist party, and the Centre Party is a center-right rural party. The support of these parties varied between 11.5 and 17.7 percent in the 2019 general election in Finland, and they are among the six biggest parties in the parliament.
Materials
The survey started with the following instruction: “Imagine that the (argument source) represents the following claim”. The argument was then presented, and respondents were instructed to “Present your evaluation on how qualified people in general would perceive the argument. Give your answer on a scale from zero to ten where zero means not at all qualified and ten means extremely qualified”.7 Instead of asking respondents’ personal evaluation of argument quality, we used the third party judgement to decrease the influence of participants’ personal opinions, that would most likely influence their judgements (Dillard et al., 2007; Hahn et al., 2009). Hahn et al. (2009) point out that third party judgements are appropriate for studying normative evaluations such as the quality of arguments. We were motivated to decrease the influence of respondents’ personal views on the matter, which they are highly likely to hold over widely discussed political topics. Third party judgement was not used regarding source characteristics because respondents cannot have pre-existing opinions on the sources. However, it is noteworthy that third party judgement of arguments may produce different evaluations of what people personally find convincing (Hahn et al., 2009). Since we are not investigating which types of arguments people consider strong but rather investigating how argument sources may influence their evaluations, third party judgement seems justified. It is more likely to expose argument source effects when strong pre-exiting attitudes do not determine argument quality evaluations. The control condition asked respondents to “Imagine that the following statement is presented to you”.
After argument evaluations, a manipulation check question: “Who represented the (argument)?” was presented to measure whether respondents remembered the argument source in a multiple-choice task. This question was followed by an item on the perceived self-interest congruence between the argument and the source: “What is your evaluation of the connection between the argument on (argument topic) and its source, is the argument against or in accordance with the interests of the source?“; source trustworthiness: “How trustworthy do you perceive the source of the argument on (argument topic)”; and source expertise “How high in expertise do you perceive the source of the argument on (argument topic)”. Issue involvement was measured by two items “How interested are you in (argument topic)”, and “How well do you know (argument topic)”. These questions were asked separately on each of the two arguments the respondent rated. The survey continued with respondent’s own opinion on the topic of the argument, two personality measures (only in the representative sample), party affiliation, questions related to social trust, political trust, political efficacy and political interest, as well as background variables. Each item was measured on a scale from zero to ten. Going backward in the survey was not possible, and both surveys included 29 items in total.8
Results
Table 2 presents the shares of correct answers to the manipulation check question in each treatment condition and for each argument A1–A6. Table 2 shows that well over half of the subjects remembered the sources in 16 out of 18 cases when the respondents were exposed to a partial, reluctant or an impartial source. The two exceptions were found for argument A1 where only 36 percent of the respondents remembered the reluctant and impartial sources. For arguments A1–A4 no source was not remembered very well.
TABLE 2. Share of respondents who remembered the argument source (%).
We also tested the strength of our manipulation by asking respondents how they perceived the partiality of the argument source. The scale was from zero to ten, ten being most partial. Table 3 presents means of the perceived partiality of the sources in the three treatment conditions. The treatment worked rather well in this respect. In all cases, partial sources get highest means, impartial sources are in the middle, and reluctant sources get lowest mean scores. However, it is noteworthy that in the case of A5 and A6, the difference between the mean scores of partial and impartial sources are not statistically significant. This can be due to the perceived ideological similarity of the two partial sources (A5: the Finns Party and A6: the Centre Party) to the impartial source (A5 and A6: National Coalition Party). Respondents lean towards partiality in their answers, i.e., impartial sources are somewhat above the theoretical midpoint at five.
TABLE 3. Perceived partiality of the source.
To study the influence of the treatments on perceived argument quality, we conducted multilevel regression analyses. The reference category for the treatment effect was the impartial source. Independent variables included the treatment conditions, source credibility, issue involvement and party affiliation. Since we did not hold the professional expertise of the source constant in A1, A2 and A4, and since both expertise and trustworthiness are likely to influence argument quality evaluations, we controlled their combination in the regressions. For the regression model, we therefore created a sum variable credibility which combines source expertise and trustworthiness. These two items correlate strongly: Cronbach’s alpha is across A1–A6 over 0.84. Issue involvement is a sum variable of being interested in and having knowledge on the topic (Cronbach’s alpha = 0.72). Further, since respondents’ party preferences are likely to influence their argument quality evaluations, and since argument sources were MPs in the case of A5 and A6, we also controlled for respondents’ party preferences.
The results of our survey experiments are reported as average marginal effects. We have different models for the representative sample data (Figure 1) and student data (Figure 2). The detailed numerical estimates are presented in the Appendix (Supplementary Appendix Tables SA2, SA3). First, we used a pooled estimate based on the multilevel model as each participant (level 2) was exposed to two arguments (level 1). We also accounted for the possibility that the treatment effects might differ according to type of issue by interacting type of source with type of issue.
FIGURE 1. Marginal effects based on the estimated multilevel model using representative sample data.
FIGURE 2. Marginal effects based on the estimated multilevel model using student data.
The regression results provide partial support to H1 (impartial sources give rise to higher argument quality evaluations compared to partial and reluctant sources) as an impartial source generates higher argument quality in a portion of the treatment conditions. The pooled estimates for reluctant source (versus impartial source) are both significantly different from zero. An impartial source increases perceived argument quality by 0.45 and 0.52 points compared to a reluctant source. In addition to these pooled estimates, the individual estimates based on interaction models show that for three of the six issues receiving information from a reluctant source leads to lower perceived argument quality. On the other hand, the two pooled estimates for partial source (versus impartial source) are not significantly different from zero. Getting information from an impartial source rather than a partial source produces significantly higher perceived argument quality for only one of six issues (i.e., the issue of refugees in the student data).9
To test H2 (high issue involvement alleviates the argument source effect) we interacted issue involvement with the treatment conditions. Models 3 in Supplementary Appendix Tables SA2, SA3 in Appendix show that these interaction terms are not statistically significant in any of the cases A1–A6. In other words, issue involvement did not moderate the source partiality effect. Issue involvement did have a direct effect on argument quality evaluations (see models 1 and 2).
To test H3 (impartial sources are evaluated as more credible compared to partial or reluctant sources) we analyzed the connection between perceived source partiality and perceived credibility. Perceived source partiality refers to the respondents’ subjective views about how partial the source was. It was measured on a 0–10 scale where 0 indicates that the source was perceived as highly reluctant and 10 that the source was perceived as highly partial. Respondents who perceived the source to be impartial should be more likely to provide responses in the middle of the scale. This implies a curvilinear association of source partiality on source credibility. In two of the six cases (A1 and A5) the highest level of perceived credibility was found in the middle of the continuum of the perceived partiality scale (see Figures 3, 4). However, Figures 3, 4 demonstrate that reluctant sources were deemed to be of very low credibility which is in line with our observation that also the arguments they present are evaluated as low in quality. H3 therefore received partial support.
FIGURE 3. Quadratic regression curves predicting the relationship between perceived source partiality and perceived source credibility.
FIGURE 4. Quadratic regression curves predicting the relationship between perceived source partiality and perceived source credibility.
Discussion
We conducted a survey experiment to study the influence of source partiality on the evaluation of argument quality. We tested different types of arguments containing a policy proposal and a partial, an impartial or a reluctant source who presented the argument. Our main observation was that argument sources had an influence on the argument quality evaluations. When a statistically significant effect was detected, it was impartial sources that produced higher quality scores compared to reluctant sources. Impartial sources outperformed partial sources in only one case, otherwise the difference between impartial and partial sources were not statistically significant.
Our results replicate earlier studies’ observations that a certain argument can be evaluated differently depending on who presents the argument. We show that this result holds for different types of politically relevant arguments and among a representative pool of respondents. Moreover, our results give further support for the inferiority of reluctant sources compared to impartial sources, but we fail to detect a statistically significant difference between impartial and partial sources in most cases. Compared to evidence on persuasion, this result is somewhat puzzling. It may depend on the particular sources we used, but certain limitations in our design can also be pointed out. It is possible that we provided insufficient information on the sources which influenced participants’ reactions. In the case of partial sources, a seemingly self-interested argumentation may have confused respondents, whereas in the case of reluctant sources, a seemingly irrational argumentation may have confused them. In both cases, respondents may have suspected that there were other reasons, not exposed to them, for the sources to endorse those policies they did. This may have created cognitive dissonance which influenced argument quality evaluations. Impartial sources do not give rise to similar suspicion, which may partly explain why they generate higher argument quality evaluations. Furthermore, we also conceive it possible that in the case of political arguments, partial sources are seen to possess issue ownership (Budge and Farlie 1983), which leads respondents to believe that they also present good arguments. Future research should examine the impact of giving more information about the sources as well as the impact of issue ownership. Moreover, it is likely that participants’ evaluation of arguments that contain policy proposals are affected by their personal opinions on these policy issues. While we asked them to use a third person perspective, it is possible that participants’ own opinions have influenced their evaluations. Future research could test proposals not related to policy issues to attenuate the impact of participants’ own opinions. Alternatively, respondents’ opinions could be measured before argument evaluations to control for the effect of participants’ own opinions.10
Another observation worth future research is that when no sources were linked to arguments, evaluations of argument quality were similar to those we observed with impartial sources. The reason may be that no source is parallel to an impartial source in the sense that neither of them suggests potentially biased reasons to represent the argument.
We do not find clear evidence that the arguments impartial sources present would be valued because the impartial sources are perceived most credible. In our data, the impartiality of the source gave rise to highest credibility ratings in only two out of six cases. However, our analysis also demonstrates that reluctant sources are not perceived high in credibility. Finally, we did not detect evidence on the influence of issue involvement, in other words, the impact of the argument source is not bigger for those with lower issue involvement.
What are the implications of our results to political communication? In terms of political rhetoric, our observations do not give support to the view that one should emphasize that certain policies are promoted even though they are against the interests of the speaker. Regarding citizens, it seems reassuring that it is the impartial sources that people tend to value. With a growing demand for the ability to critically evaluate internet and social media content, it may also a be a good sign that reluctant sources do not generate high argument quality evaluations, in particular if it indicates that people critically ask themselves, why is the source presenting an argument that is clearly against his or her interests.
It is noteworthy that the arguments we tested are all relatively simple and not particularly strong. It is indeed possible that sources influence quality evaluations only when arguments are rather weak. Future research should test the interaction of the source with the quality of the argument. Is it so that sources influence argument ratings only when arguments are rather weak? Furthermore, the possibility that reluctant sources give rise to a more thorough consideration of the quality of the argument should also be further scrutinized.
In political communication, arguments are seldom presented in isolation of preceding communication and a wider context. It would therefore be interesting to test whether arguments embedded in vignettes describing the wider context where arguments are presented would influence quality ratings.
Data Availability Statement
The representative sample data is deposited in the Finnish Social Sciences Data Archive (FSD), https://www.fsd.tuni.fi/en/, code is FSD3587. Both data sets are also available from the authors.
Ethics Statement
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
Author Contributions
KH, EK-K, LM, and JS have contributed equally to the design of the experiment. JS and LM were mainly responsible for the implementation of the study in practice. KH holds the main responsibility of the original idea of the experiment and she is also mainly responsible of writing the introduction, theory, existing literature and discussion in the manuscript. JS and PS bear main responsibility of statistical analysis and reporting of the results. All authors have contributed to the revision of the manuscript during the writing process.
Funding
This work was supported by the Strategic Research Council of the Academy of Finland (grant numbers 312671/312675/326662) and the Academy of Finland (grant number 286237).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2022.778771/full#supplementary-material
Footnotes
1We use terms partial, reluctant and impartial, whereas the referred studies do not necessarily use the same terms. For example, biased is commonly used instead of partial, and impartial can be unbiased, objective or neutral.
2Both data sets are available from the authors upon request and the representative sample data is also available from the Finnish Social Science Data Archive (Data FSD3587, https://www.fsd.tuni.fi/en/).
3A private company with expertise in survey research conducted the survey.
4Supplementary Appendix Table SA1 in Appendix presents the respondents of the representative sample in each treatment condition according to gender and age. Differences between treatment conditions were not statistically significant regarding gender (A1 and A2: χ2 = 2.083, p = 0.555; A3 and A4: χ2 = 1.7350, p = 0.629) and age (A1 and A2: F = 0.46, p = 0.7092; A3 and A4: F = 0.06, p = 0.9798).
5The student data set includes graduate and postgraduate students, 74 percent of the respondents were female (corresponds roughly to the share of female students at Finnish universities), the average age was 28.7 years, and respondents represented a variety of disciplines. Differences between treatment conditions were not statistically significant regarding gender (A5 and A6: χ2 = 3.6691, p = 0.160) and age (A5 and A6: F = 0.08, p = 0.9251).
6We pretested eight arguments with two sets of student respondents (n = 162 and n = 40). The pretest evaluated variation in argument quality evaluations, perceived source partiality and remembering the argument source. On the basis of the pretest, we rejected five arguments because they did not work well according to one of these criteria. We selected three arguments that worked well in the pretest and formulated two novel arguments based on the pretest results to be used in the actual experiment. The two new arguments were not pretested.
7The Finnish word “pätevä” is translated here as “qualified”. Other possible translations are valid, adept, or adequate.
8An English translation of the survey is available from the authors.
9We also ran regressions predicting argument quality evaluations with the no source treatment condition as one of the predictors in the representative sample data. The estimates are available from the authors on request. In these regressions, the control variable perceived source credibility had to be excluded because the respondents in the no source condition were not asked to evaluate how credible the source was. The results showed that no source produced similar argument quality evaluations as the impartial source condition. In all but one case was there a statistically significant difference: in the case of A4 no source generated higher evaluations of argument quality compared to impartial source.
10We are grateful for an anonymous referee for pointing out several possibilities for future research.
References
Anderson, L. (1970). An Experimental Study of Reluctant and Biased Authority-Based Assertions. Argumentation and Advocacy 7 (2), 79–84. doi:10.1080/00028533.1970.11951417
Arnold, W. E., and McCroskey, J. C. (1967). The Credibility of Reluctant Testimony. Cent. States Speech J. 18 (2), 97–103. doi:10.1080/10510976709362870
Benoit, W. L., and Kennedy, K. A. (1999). On Reluctant Testimony. Commun. Q. 47 (4), 376–387. doi:10.1080/01463379909385568
Birnbaum, M. H., and Mellers, B. A. (1983). Bayesian Inference: Combining Base Rates with Opinions of Sources Who Vary in Credibility. J. Personal. Soc. Psychol. 45 (4), 792–804. doi:10.1037/0022-3514.45.4.792
Briñol, P., and Petty, R. E. (2009). Source Factors in Persuasion: A Self-Validation Approach. Eur. Rev. Soc. Psychol. 20 (1), 49–96. doi:10.1080/10463280802643640
Budge, I., and Farlie, D. J. (1983). Explaining and Predicting Elections. Issue Effects and Party Strategies in Twenty-Three Democracies. London: George Allen and Unwin.
Callison, C. (2004). The Good, the Bad, and the Ugly: Perceptions of Public Relations Practitioners. J. Public Relations Res. 16 (4), 371–389. doi:10.1207/s1532754xjprr1604_3
Chaiken, S., and Maheswaran, D. (1994). Heuristic Processing Can Bias Systematic Processing: Effects of Source Credibility, Argument Ambiguity, and Task Importance on Attitude Judgment. J. Personal. Soc. Psychol. 66 (3), 460–473. doi:10.1037/0022-3514.66.3.460
Chebat, J.-C., Filiatrault, P., and Perrien, J. (1990). Limits of Credibility: The Case of Political Persuasion. J. Soc. Psychol. 130 (2), 157–167. doi:10.1080/00224545.1990.9924566
Chittleborough, P., and Newman, M. E. (1993). Defining the Term "Argument". Informal Logic 15 (3), 189–207. doi:10.22329/il.v15i3.2486
Clark, J. K., Wegener, D. T., Habashi, M. M., and Evans, A. T. (2012). Source Expertise and Persuasion. Pers Soc. Psychol. Bull. 38 (1), 90–100. doi:10.1177/0146167211420733
Dillard, J. P., Shen, L., and Vail, R. G. (2007). Does Perceived Message Effectiveness Cause Persuasion or Vice Versa? 17 Consistent Answers. Hum. Comm Res 33 (4), 467–488. doi:10.1111/j.1468-2958.2007.00308.x
Hahn, U., Harris, A. J. L., and Corner, A. (2009). Argument Content and Argument Source: An Exploration. Il 29 (4), 337–367. doi:10.22329/il.v29i4.2903
Hawkins, D., Brook, L. C., Hansen, I. M., Hoopes, N. A., and Tidwell, T. R. (2019). Do Citizens See Through Transparency? Evidence from Survey Experiments in Peru. Br. J. Polit. Sci. 49 (1), 205–228. doi:10.1017/S0007123416000466
Hoeken, H., Šorm, E., and Schellens, P. J. (2014). Arguing about the Likelihood of Consequences: Laypeople's Criteria to Distinguish strong Arguments from Weak Ones. Thinking & Reasoning 20 (1), 77–98. doi:10.1080/13546783.2013.807303
Hoeken, H., Timmers, R., and Schellens, P. J. (2012). Arguing about Desirable Consequences: What Constitutes a Convincing Argument? Thinking & Reasoning 18 (3), 394–416. doi:10.1080/13546783.2012.669986
Hovland, C. I., and Weiss, W. (1951). The Influence of Source Credibility on Communication Effectiveness. Public Opin. Q. 15 (4), 635–650. doi:10.1086/266350
Johnson, B. T., Smith-McLallen, A., Killeya, L. A., and Levin, K. D. (2004). Truth or Consequences: Overcoming Resistance to Persuasion with Positive Thinking. (CHIP Documents No. 14). Center for health, intervention, and prevention. Available at; https://opencommons.uconn.edu/chip_docs/14
Kim, D., and Benbasat, I. (2009). Trust-Assuring Arguments in B2C E-Commerce: Impact of Content, Source, and Price on Trust. J. Manage. Inf. Syst. 26 (3), 175–206. doi:10.2753/MIS0742-1222260306
Kruglanski, A. W., Chen, X., Pierro, A., Mannetti, L., Erb, H.-P., and Spiegel, S. (2006). Persuasion According to the Unimodel: Implications for Cancer Communication. J. Commun. 56 (1), 105–122. doi:10.1111/j.1460-2466.2006.00285.x
Kruglanski, A. W., and Thompson, E. P. (1999). Persuasion by a Single Route: A View From the Unimodel. Psychol. Inq. 10 (2), 83–109. doi:10.1207/S15327965PL100201
Lee, G. B. (2005). Persuasion, Transparency, and Government Speech. Hastings L. J. 56 (5), 983–1057.
López, M., and Sicilia, M. (2014). eWOM as Source of Influence: The Impact of Participation in eWOM and Perceived Source Trustworthiness on Decision Making. J. Interactive Advertising 14 (2), 86–97. doi:10.1080/15252019.2014.944288
McCroskey, J. C. (1969). A Summary of Experimental Research on the Effects of Evidence in Persuasive Communication. Q. J. Speech 55 (2), 169–176. doi:10.1080/00335636909382942
McPeek, R. W., and Edwards, J. D. (1975). Expectancy Disconfirmation and Attitude Change. J. Soc. Psychol. 96 (2), 193–208. doi:10.1080/00224545.1975.9923285
Newman, N., Fletcher, R., Schulz, A., Andı, S., and Nielsen, R. K. (2020). Reuters Institute Digital News Report 2020. Reuters Institute for the Study of Journalism. Available at; https://www.digitalnewsreport.org/.
Noteberg, A., Christiaanse, E., and Wallage, P. (2003). Consumer Trust in Electronic Channels: The Impact of Electronic Commerce Assurance on Consumers' Purchasing Likelihood and Risk Perceptions. E-Service J. 2 (2), 46–67. doi:10.1353/esj.2003.0014
Park, H. S., Levine, T. R., Westerman, C. Y. K., Orfgen, T., and Foregger, S. (2007). The Effects of Argument Quality and Involvement Type on Attitude Formation and Attitude Change: A Test of Dual-Process and Social Judgment Predictions. Hum. Comm Res 33 (1), 81–102. doi:10.1111/J.1468-2958.2007.00290.X
Petty, R. E., Cacioppo, J. T., and Goldman, R. (1981). Personal Involvement as a Determinant of Argument-Based Persuasion. J. Personal. Soc. Psychol. 41 (5), 847–855. doi:10.1037/0022-3514.41.5.847
Petty, R. E., and Cacioppo, J. T. (1984). Source Factors and the Elaboration Likelihood Model of Persuasion. Adv. Consumer Res. 11 (1), 668–672.
Petty, R. E., and Cacioppo, J. T. (1986). The Elaboration Likelihood Model of Persuasion. Adv. Exp. Soc. Psychol. 19, 123–205. doi:10.1016/S0065-2601(08)60214-2
Pornpitakpan, C. (2004). The Persuasiveness of Source Credibility: A Critical Review of Five Decades' Evidence. J. Appl. Soc. Pyschol 34 (2), 243–281. doi:10.1111/j.1559-1816.2004.tb02547.x
Smith, D., Menon, S., and Sivakumar, K. (2005). Online Peer and Editorial Recommendations, Trust, and Choice in Virtual Markets. J. Interactive Marketing 19 (3), 15–37. doi:10.1002/dir.20041
Stubb, C., and Colliander, J. (2019). "This Is Not Sponsored Content" - The Effects of Impartiality Disclosure and E-Commerce landing Pages on Consumer Responses to Social media Influencer Posts. Comput. Hum. Behav. 98, 210–222. doi:10.1016/j.chb.2019.04.024
Tormala, Z. L., Briñol, P., and Petty, R. E. (2006). When Credibility Attacks: The Reverse Impact of Source Credibility on Persuasion. J. Exp. Soc. Psychol. 42 (5), 684–691. doi:10.1016/j.jesp.2005.10.005
Walster, E., Aronson, E., and Abrahams, D. (1966). On Increasing the Persuasiveness of a Low Prestige Communicator. J. Exp. Soc. Psychol. 2 (4), 325–342. doi:10.1016/0022-1031(66)90026-6
Wolf, A. G., Rieger, S., and Knauff, M. (2012). The Effects of Source Trustworthiness and Inference Type on Human Belief Revision. Thinking & Reasoning 18 (4), 417–440. doi:10.1080/13546783.2012.677757
Yi, M. Y., Yoon, J. J., Davis, J. M., and Lee, T. (2013). Untangling the Antecedents of Initial Trust in Web-Based Health Information: The Roles of Argument Quality, Source Expertise, and User Perceptions of Information Quality and Risk. Decis. Support Syst. 55 (1), 284–295. doi:10.1016/j.dss.2013.01.029
Keywords: argument evaluation, argument source effect, partial source, impartial source, reluctant source, experiment
Citation: Herne K, Sipinen J, Kestilä-Kekkonen E, Mattinen L and Söderlund P (2022) The Force of the Argument Source: The Partiality of the Source Influences the Evaluation of Political Arguments. Front. Commun. 7:778771. doi: 10.3389/fcomm.2022.778771
Received: 17 September 2021; Accepted: 05 January 2022;
Published: 02 February 2022.
Edited by:
Pradeep Nair, Central University of Himachal Pradesh, IndiaReviewed by:
Steve Oswald, Université de Fribourg, SwitzerlandBarbara Ann Bush, University of California, San Diego, United States
Copyright © 2022 Herne, Sipinen, Kestilä-Kekkonen, Mattinen and Söderlund. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Kaisa Herne, kaisa.herne@tuni.fi