The Dunning-Kruger Effect Based on Educational Attainment and Political Affiliation Concerning Trust and Perceived Accuracy of the Media

Abstract

Belief in the accuracy of media sources and trust in media outlets has become a topic of great interest given the current political climate in the United States following the 2016 presidential election, and the emergence of so-called “fake news.” An understanding of how educational attainment may affect the level at which a person believes the media to be accurate and how much people trust the media is examined here, with the results connected to what is termed the Dunning-Kruger Effect (DKE) (Kruger & Dunning, 1999). A survey distributed in the months prior to the fall 2018 midterm elections collected data on political affiliation, level of education attained, trust in the media, and the perceived effectiveness and re-electability of United States President Donald Trump. Data were examined by educational attainment only, then by educational attainment of those indicating a political affiliation of Democrat, Republican, or Independent/Other. While some significant relationships emerged, overall, there was no indication of the DKE present in responses focused on the media. A further examination of question coupling was undertaken, which revealed that respondents with lower levels of education created illogical response couplings more frequently than respondents with higher education levels. Respondents that indicated they did not follow politics, were uninformed concerning politics, did not think that the media was accurate, and did not trust the media, were shown to construct definitive answers concerning questions that require knowledge of politics in the present. While no political affiliation was attached to the last response group, educational attainment was, which showed that those in lower attainment categories comprised at least two-thirds of the group respondents who constructed illogical or contradictory response couplings. Inattention while completing the survey may explain some of the effects, but the DKE does appear to be present in the results.

Introduction

People’s perception of their skill level can have a distorted effect on their actual ability to perform a task or understand a concept[1], and was first expressed in a systematic study by [2], resulting in what is now commonly referred to as the[3] . This phenomenon explains that individuals with low skill levels or education routinely self-assess at a level higher than their actual ability. In contrast, those with higher skill and education levels tend to underscore their ability to perform well on a skill or assessment exam[3]. While not without its detractors[4], the DKE has been utilized in multiple areas[3] to help explain why people of lesser knowledge consistently overestimate their ability, including graph interpretation[5] sports coaching[6] , healthcare[7,8], information literacy[9], workplace computing[10], a general and rising level of distrust in science and scientific expertise[11], climate change literacy[12], and even within the peer-review process for journal article publication[13]. Further,[14] found that people likely overestimate their actual knowledge on political topics

Educational attainment was not featured as a parameter in[2] original work but has since come to be seen as a suitable proxy for reported ability in self-assessments[15]. However,[16] disagree that educational attainment and intelligence can be related in a strictly linear fashion. Education is known to play a role in understanding politics, and particularly, voter turnout[17], which directly affects the level to which people should claim knowledge about the political process and the candidates within a given race. [18]provides a framework in which we can better understand how the DKE may cause those with lesser knowledge (and as a proxy measure, perhaps lesser education) to affect political processes through “political overconfidence” (p. 2). The Anson framework is applied here in that respondents to a survey concerning the 2018 midterm elections provided answers that allowed for the binary classification of data focused on trust in the media, perceived accuracy of the media, the effectiveness of President Donald Trump, and the belief or disbelief as to whether or not President Trump will be re-elected in 2020.

While the DKE was likely not present in answers concerning the media, it was observable in answer couplings that a reasonable person would likely understand to be illogical. Such couplings include the belief that a person can be uninformed on politics and not follow politics yet be able to somehow possess enough knowledge about the president to make a determination of effective or ineffective (as a leader). Similarly, a coupling was observed in which the same self-reported uninformed respondents were able to make a definitive determination as to whether or not President Trump would be re-elected in the fall 2020 presidential election. The DKE emerged when the educational attainment of these respondents was examined.

Methods

Data were collected from an online survey administered in the fall of 2018 to students and faculty at 15 colleges in six states, which were chosen based on their electoral votes in the most recent presidential election (2016); three that awarded their votes to the Democratic candidate and three that awarded their votes to the Republican candidate. Email addresses were collected (22,763) and used to distribute the survey, to which 781 responses were received (3.43%). Respondents were asked the questions listed in Table 1 using a five-point Likert Scale. Of 781, respondents under 18 (legal voting age) were eliminated from further consideration, leaving 693 usable surveys.

Table 1. Questionnaire items..

Educational attainment data was categorized into four categories (identifier): less than high school/high school diploma or equivalent (H), associate degree (A), bachelor degree (B), and master’s degree or higher (M). Political affiliation of the 693 was recategorized into Democrat (187, 27.0%), Republican (208, 30.0%), and Independent/Other (152, 21.9%), with 146 (21.1%) reporting no political affiliation (the latter category was eliminated from further examination). Values of reported trust in the media, perceived level of accuracy, perceived trust in the media level to which the respondent follows politics, and level to which they are informed on politics were calculated as an average, and examined in a spreadsheet pivot table, aggregating the data fields by educational attainment and political affiliation. Average values of ranking data by political affiliation and educational attainment were tested using multiple comparison models (Tukey’s b). Data on sources of information for political information were also evaluated in multiple comparisons models (also Tukey’s b), to determine whether or not significant relationships existed between average age, trust in the media, and perceived accuracy of the media, and types of media sources used. In examining types of media sources used, each source was re-coded into a category aligning with a respondent who answered that they get their information from one and only one media source for categories one through six, and “multiple sources” (category 7), average age, average level of trust in the media (in general), and average level of perceived media accuracy (in general) for that information group was added. Tukey’s b was applied to determine what, if any, relationships exist between age, trust, and accuracy based on the singular (1-6) source for new or if multiple news sources are used.

Results and Discussion

Educational attainment versus reported perception of media accuracy (1 = extremely inaccurate, 5 = extremely accurate) shows the lowest ranking among the A group but just 0.07 below the H group. Average trust climbs to 3.05 for the B group, and further to 3.40 for the M group (Figure 1, Table 2). The Tukey’s b test showed that a significant difference exists between the M group and both the H and A group indicating no presence of the DKE here, as those with lower levels of education do not claim a higher degree of faith in media accuracy.

Table 2. Multiple comparison model and Tukey's b results, educational attainment.

Figure 1. Educational attainment vs. reported perception of media accuracy.

To determine if the DKE was apparent within groups defined by political affiliation, the same values were plotted with the addition of the three affiliation categories (Figure 2). Democrats and Independent/Other showed the same higher value at the H level, with a dip at the A level. Democrats showed a subsequent rise at the B and M levels, and Independent/Other showed a rise at the B level and a slight drop (0.04 points) at the M level. Republicans showed a steady rise from the H to B levels, but a drop at the M level.

Figure 2. Educational attainment vs. reported perception of media accuracy by political affiliation.

Separate Tukey’s b tests were performed for the three political affiliation groups as the dependent variable. For Democrats (Table 3), a significant difference exists between the M and H groups as well as the M and A groups. Using both Republicans and Independent/Other as the dependent variable showed no significant relationships between average levels of media trust between the four classes of educational attainments, which may indicate the DKE present among non-democrats when examining the data by political affiliation; education should be playing a role in determining as to how accurate media sources tend to be, but in nondemocrats, it is not.

Table 3. Multiple comparison model and Tukey's b results, educational attainment, Democrats only.

Trust in media was also examined in this manner, with similar results. Figure 3 shows the H and A groups reported the lowest levels of trust in the media, with A reporting a mean value of 2.87, just 0.09 lower than the H group. From those two sub-3 values, both the B and M groups reported higher levels of trust in the media. However, the Tukey's b test revealed that no significant difference exists between these groups. When broken out by political affiliation, the results are similar, in that the general trend shows higher levels of trust being associated with higher levels of education. However, no significant relationships emerged between the groups within their political categories (Figure 4).

Figure 3. Educational attainment vs. reported trust in media.

Figure 4. Educational attainment vs. reported trust in media by political affiliation.

In examining the responses to whether or not the respondent trusted the media and thinks the media is accurate by political affiliation, some interesting patterns emerge; Table 4 highlights these response patterns broken out by educational attainment and political affiliation. As shown in the table, some confusion between the relationship connecting trust and accuracy seems to exist. For example, Republicans consistently across educational attainment categories answered that they trust the media, yet think the media is not accurate at higher rates than both Democrats and Independent/other. Conversely, but related, when combining the responses of trust and accuracy, Democrats and Independents/other consistently answered with a higher percentage that they both trust the media and think it is accurate. Further, the change in percentage for each category of education attainment went down as educational attainment went up, further indicating that educational attainment influences how people perceive, whether correctly or incorrectly, the status the media holds in society in terms of how we trust them or find them to be accurate.

Table 4. Answers that appear to conflict, by political affiliation and educational attainment.

The type of media source, or whether or not a person gets their information on politics may influence the level to which they trust the media, how accurately they think the media portrays the news and considering age as a factor, that may also be related to the selection type and singularity of news source selections. Table 5 shows that the average age of respondents is highest (52) for those who get their news from a print source such as newspapers or magazines only. The youngest average age group, perhaps not surprisingly, was 26.8, representing those who only get their news from social media. The multiple comparison models showed no significant relationship between the selection of news sources and whether the media was perceived as trusted and/or accurate. However, significant relationships do exist when comparing the source of news to average age. A significant difference exists between the average age of those in category two and those in categories four (0.001), six (0.010), and seven (0.040); this expresses the idea that the older respondents were significantly more likely to get their news solely from print sources, while a younger respondent was more likely to obtain their news from more "modern" sources, i.e., social media. How this might relate to distortion of truth, trust, and perceived accuracy is not examined here. However, a growing body of literature suggests that the concept of the "echo chamber" is working to divide political opinion strongly and that social media platforms are rife with groups that tend to reinforce the beliefs of the others in the group (Brugnoli et al., 2017; Hart et al., 2009; Kunda, 1990; Del Vicario et al., 2016; Lewandowsky et al., 2017; Zollo et al., 2017).

Table 5: Media sources and relationship to age, trust in media, and perception of the media's accuracy.

In a similar examination, a scenario emerges where we must question how the respondent interacts with information in the world, and from what internal and external methods do they derive their knowledge and opinions about politics. In a reduced cohort of 48 respondents consisting of those who provided the following responses; they:

1) do not follow politics

2) claimed to be uninformed about politics

3) stated that they did not think the media was accurate

4) stated that they do not trust the media

33 out of the 48 (68.8%) stated definitively they believe President Trump was either an effective or ineffective president (termed here sub-question 1). Further, 32 of the 48 (66.7%) were able to state definitively that President Trump would or would not be re-elected in 2020 (termed here sub-question 2). Note that providing an opinion in support or not in support of President Trump is irrelevant here, as is political affiliation (for these specific sub-questions), but a definitive answer is the focus. Given the combination of four answers provided by this group (list above) and examining the educational attainment level of the 33 and 32 respondents and their corresponding answers to sub-questions 1 and 2, two-thirds or more fall into the less than high school/ HS/GED category (H), which highlights an appearance of the DKE (33 of 48, 68.8% for sub-question 1 and 32 out of 48, 66.7% for sub-question 2). The lower the level of educational attainment categories here house a majority of respondents who were able to make a definitive statement concerning the performance of the president as well as the probability of the president being reelected in 2020; both of which it would seem to require the person to either follow politics, be informed about politics, have faith in media accuracy, and/or trust the media or some combination of those four. For those in sub-cohort 1 not in the H category (15), the level of educational attainment makes up a progressively smaller percentage of the total, indicating that the more educated are less represented in this group, which provided the definitive answer President Trump is effective or ineffective, yet, the respondent provided the answers in the list above. For the remaining 15 in sub-cohort 1, 11 (22.9%) were from the A category, 3 (6.3%) from the B category, and just 1 (2.1%) from the M category. Similarly, 16 respondents in sub-cohort 2 had attained at least an Associate degree. For those that claimed not to follow politics, not be informed about politics, not think the media was accurate, and not trust the media, yet felt definitively that President Trump would be either re-elected or not re-elected, 11 (22.9%) were in the A category, 4 (8.3%) were in the B category, and as with sub-cohort 1, just 1 (2.1%) were in the M category.

Conclusions

Data collected from 693 respondents, narrowed to 547 that reported a political affiliation, were used to determine if the DKE was observable concerning the midterm elections in the fall of 2018. Data on educational attainment and political affiliation, as well as the perceived level of trust in the media, whether or not the media is accurate, whether or not the respondent claimed to follow political issues, and whether or not they claimed to be informed on political issues were examined by level of educational attainment. When no political affiliation was considered, the DKE appears not to be present; no statistical significance exists in the level of perceived accuracy. When political affiliation is considered, the results showed that there was a difference among Democrats, with the M group showing statistically significant higher levels of belief that the media is accurate than the H or M groups. No such relationship was found among Republicans or the Independent/Other groups. These results indicate that the DKE is not present in these data, as the respondents in the lower levels of education (Associate degree or less) did not claim an outsized belief in the accuracy of the media when compared to respondents holding a Master’s degree or higher. Trust in the media was also examined, and the results aligned with the results of the beliefs in media accuracy. Higher levels of education reported higher levels of trust in the media (where trust is assumed to be a proxy indicator in that those with higher education levels should hold a firmer understanding of the role of the media in American culture).

Contradictory answers provided by respondents were also examined and categorized by level of educational attainment. Results showed that when a contradictory answer is provided (such as, "I trust the media," coupled with, "I do not think the media is accurate"), the level of educational attainment steadily goes up as the percentage of these types of answers goes down (as a percent of row totals of political affiliation and educational attainment). The converse is also true that when answers that align logically were provided (such as "I trust the media," coupled with "I think the media is accurate"), an increase in education attainment aligns with the delivery of such logical constructions. This may suggest the presence of the DKE in lower levels of educational attainment in that a clear mindset is not established in the respondent concerning the connection between trusting a news source and believing that the source is also accurate. In contrast, higher degree-holders assembled these illogical response couplings less frequently.

Finally, and perhaps most interestingly, is the apparent ability of a sub-cohort of respondents to arrive at definitive conclusions concerning two questions about President Trump. A group of 48 respondents that provided four specific answers was isolated. This group indicated that they do not follow politics, are uninformed about politics, do not feel the media was accurate and did not trust the media. Nevertheless, this group was also able to state definitively that President Trump is an effective or ineffective president. Concerning the second question of a similar nature, the 48 respondents were also able to state definitively that President Trump would be re-elected or not re-elected (answers are exclusive). Where the DKE is likely observable here is in the addition of educational attainment to this cohort of 48 respondents. Of the smaller group of 48 respondents, 33 and 32 respondents were able to make a definitive statement concerning President Trump, a statement that would require one to be informed on politics and, most likely, follow the media and trust what they ingest from news sources. However, those groups of 33 and 32 were from the H category. Just one respondent of the 48 was from the M category, indicating that the higher the degree of educational attainment, the less likely such an illogical answer was provided. It is noted, however, that research has indicated respondents do not always attentively engage with survey material, and as such, some of the illogical constructions uncovered here are due to that inattention as opposed to the lack of mental ability to formulate a consistent response when presented with two different questions whose answers should align logically (Liu & Wronski, 2018).

This research's primary goal was to determine if the DKE could be observed in respondents concerning the 2018 midterm elections. While the DKE does not appear to be present concerning perceived accuracy and trust in the media, it does appear in results dealing with contradictory and illogical answer constructs provided via the survey instrument, with higher educational attainment levels showing a lower incidence of providing such answer couplings.

Declarations

Study limitations: The effects driving the contradictory response couplings may be, in part, driven by inattention on the part of the respondent; this is noted in the text. The survey response was lower than the researcher would have liked.

Acknowledgements: None Funding source: None Competing interests: None Human and Animal-related Study: Ethical approval: IRB approved in fall 2018, Cloud County Community College, Concordia, Kansas. Protocol No. 1008. Informed consent text: Dear Respondent,

We are conducting a questionnaire survey to understand the voting tendencies or lack thereof in the community, and to learn how and why voters make decisions when voting. This curiosity arose due to the upcoming midterm elections and media coverage of the same. We would specifically like to know how you educate yourself on political concerns, your voting tendencies, and your opinion of media coverage of political events.

We would very much appreciate your collaboration in the completion of this questionnaire. All information from this survey will be kept confidential, stored on a secure (https) password protected site, with access granted only to the faculty research supervisor. Information received will be used solely for academic research purposes. No respondent information that can identify the person (i.e., name) is attached to any survey, nor included in any analysis.

Your participation in this survey is completely voluntary and you are free to not answer questions that you do not wish to. There will be no penalty for not participating, and during the completion of the questionnaire, you can quit at any time if you are not comfortable with answering the topic.

References

1. Gross M, et.al. What’s skill got to do with it? Information literacy skills and self-views of ability among first-year college students. Journal of the Association for Information Science and Technology, 2012;63:574-583
2. Kruger K, et.al. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 1999;77:1121.
3. Dunning, D. The Dunning-Kruger effect: On being ignorant of one’s own ignorance. In J. M. Olson & M. P. Zanna (Eds.), Advances in experimental social psychology, San Diego, CA: Academic Press.
4. 2011;44:247–296.
5. Krajc O, et.al. Are the unskilled really that unaware? An alternative explanation. Journal of Economic Psychology, 2008;29:724-738.
6. Nuhfer E, et.al. How random noise and a graphical convention subverted behavioral scientists explanations of self-assessment data: numeracy underlies better alternatives. Numeracy, 2017;10: 4
7. Sullivan P.J, et.al. An investigation into the Dunning-Kruger Effect in sport coaching. International Journal of Sport Exercise Psychology, 2018;17:591-599.
8. Abdullah J. M. The eight states of trust and “Amanah” in medicine and the Dunning Kruger Effect. The Malaysian Journal of Medical Sciences, PMCID: PMC4418106, 2014; 21:1-3
9. McShea L, et.al. Paid support workers for adults with intellectual disabilities: their current knowledge of hearing loss and future training needs. Journal of Applied Research in Intellectual Disabilities, 2016; 29:422-432
10. Mahmood K. Do people overestimate their information literacy skills? A systematic review of empirical evidence on the Dunning-Kruger Effect. Communications in Information Literacy, 2016;10:198-213.
11. Gibbs S, et.al. The Dunning-Kruger Effect in a workplace computing setting. Computers in Human Behavior, 2017;72:589-595.
12. Benegal B. Overconfidence and the discounting of expertise: A commentary. Social Science & Medicine, 2018; 213:95-97.
13. Guy S, et.al. Investigating the effects of knowledge and ideology on climate change beliefs. European Journal of Social Psychology, 2014; 44:421–429.
14. Huang S. When peers are not peers and don’t know it: The Dunning-Kruger Effect and self-fulfilling prophecy in peer-review. Bioessays, 2013; 35:414-416.
15. Michael P , et.al. Is belief superiority justified by superior knowledge? 2018 Pages 2018; 76:290-306.
16. Berry C. M, et.al. Education attainment as a proxy for cognitive ability in selection: effects on levels of cognitive ability and adverse impact. Journal of Applied Psychology, 2006; 91:696-705.
17. Deary I. J, et.al. Intelligence and education: causal perceptions drive analytical processes and therefore conclusions. International Journal of Epidemiology, 2010;39:1362-1369.
18. Burden B. C. The dynamic effects of education on voter turnout. Electoral Studies, 2009;28:540-549.
19. Anson I. Partisanship, political knowledge, and the Dunning-Kruger Effect. Political Psychology, 2018; 39:1173-1192.