Sterling Williams-Ceci,Michael W. Macy,Mor Naaman
Abstract:People rely on search engines for information in critical contexts, such as public health emergencies–but what makes people trust some search results more than others? Can search engines influence people's levels of trust by controlling how information is presented? And, how does the presence of misinformation influence people's trust? Research has identified both rank and the presence of misinformation as factors impacting people's search behavior. Here, we extend these findings by measuring the effects of these factors, as well as misinformation warning banners, on the perceived trustworthiness of individual search results. We conducted three online experiments (N = 3196) using Covid-19-related queries, and found that although higher-ranked results are clicked more often, they are not more trusted . We also showed that misinformation does not damage trust in accurate results displayed below it. In contrast, while a warning about unreliable sources might decrease trust in misinformation, it significantly decreases trust in accurate information. This research alleviates some concerns about how people evaluate the credibility of information they find online, while revealing a potential backfire effect of one misinformation-prevention approach; namely, that banner warnings about source unreliability could lead to unexpected and nonoptimal outcomes in which people trust accurate information less.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to explore and answer the following key questions:
1. **How does the ranking of search results affect users' trust?**
- Although previous research has shown that a higher ranking of search results increases the likelihood of users clicking, does this ranking also increase users' trust in these results? The research hypothesizes that a higher ranking will lead to a higher click - through rate (H0), and a higher level of trust (H2).
2. **How does the presence of misinformation affect users' trust in accurate information?**
- Does misinformation affect users' trust in the accurate information that follows? Specifically, when misinformation appears in a high - ranking position, does it reduce users' trust in the low - ranking accurate information? The research hypothesizes that misinformation will be clicked less and considered less trustworthy (H3a, H3b), and that high - ranking misinformation will not damage the trust in the accurate information below it (H4a, H4b).
3. **What is the effect of warning banners?**
- Are the warning banners used by search engines when displaying inaccurate information effective? Do these warning banners have a negative impact on users' trust in accurate information? The research hypothesizes that warning banners will reduce users' trust in all results (H5).
### Research background
- **The influence of search result ranking**: Previous research has shown that users are more likely to click on search results with a higher ranking, but this does not mean that they trust these results more. The Cognitive Miser Theory explains this phenomenon, that is, people rely on simple heuristics to avoid using too many cognitive resources.
- **The influence of misinformation**: The frequent appearance of misinformation in search engine results has raised concerns about public health, especially during the COVID - 19 pandemic. Exposure to misinformation may lead to poor medical decisions and may weaken users' trust in subsequent accurate information.
- **The role of warning banners**: Search engines such as Google have introduced warning banners to remind users to be aware of unreliable information sources. However, the effectiveness of these warning banners has not been fully verified, and some research even shows that they may be counterproductive and reduce users' trust in accurate information.
### Experimental design and methods
To verify the above hypotheses, the researchers conducted three online experiments involving a total of 3,196 participants. The experiments evaluated users' trust in search results by manipulating the ranking of search results, the position of misinformation, and the presence or absence of warning banners. The main findings include:
- **The relationship between ranking and click - through behavior**: Although results with a higher ranking are clicked more often, they are not considered more trustworthy by users.
- **The influence of misinformation**: Misinformation itself is highly untrusted, but its presence does not damage the trust in the accurate information that follows.
- **The counter - effect of warning banners**: Warning banners may reduce users' trust in accurate information, showing a potential counter - effect.
In conclusion, this study reveals the complex influence of search result ranking, misinformation, and warning banners on users' trust, and provides important implications for improving the design of search engines.