Abstract:Background: People are increasingly accessing health-related social media sites, such as health discussion forums, to post and read user-generated health information. It is important to know what criteria people use when deciding the relevance of information found on health social media websites, in different situations. Objective: The study attempted to identify the relevance criteria that people use when browsing a health discussion forum, in 3 types of use contexts: when seeking information for their own health issue, when seeking for other people's health issue, and when browsing without a particular health issue in mind. Methods: A total of 58 study participants were self-assigned to 1 of the 3 use contexts or information needs and were asked to browse a health discussion forum, HealthBoards.com. In the analysis, browsing a discussion forum was divided into 2 stages: scanning a set of post surrogates (mainly post titles) in the summary result screen and reading a detailed post content (including comments by other users). An eye tracker system was used to capture participants' eye movement behavior and the text they skim over and focus (ie, fixate) on during browsing. By analyzing the text that people's eyes fixated on, the types of health information used in the relevance judgment were determined. Post-experiment interviews elicited participants' comments on the relevance of the information and criteria used. Results: It was found that participants seeking health information for their own health issue focused significantly more on the poster's symptoms, personal history of the disease, and description of the disease (P=.01, .001, and .02). Participants seeking for other people's health issue focused significantly more on cause of disease, disease terminology, and description of treatments and procedures (P=.01, .01, and .02). In contrast, participants browsing with no particular issue in mind focused significantly more on general health topics, hot topics, and rare health issues (P=.01, .01, and .01). Conclusion: Users browsing for their own health issues used mainly case-based relevance criteria to relate the poster's health situation to their own. Participants seeking for others' issues used mostly general knowledge-based criteria, whereas users with no particular issue in mind used general interest- and curiosity-based criteria.

Relevance Judgment in Epistemic and Hedonic Information Searches

Relevance Judgment: What Do Information Users Consider Beyond Topicality?

User-Oriented Relevance Judgment: A Conceptual Model

Understanding Relevance Judgments in Legal Case Retrieval

Novelty and Topicality in Interactive Information Retrieval

When does Relevance Mean Usefulness and User Satisfaction in Web Search?

Relevance judgments of mobile commercial information

A Linguistic Study on Relevance Modeling in Information Retrieval

Investigating Passage-level Relevance and Its Role in Document-level Relevance Judgment

A Comparison of Retrieval Result Relevance Judgments Between American and Chinese Users

Review on Relevance Criteria(2000-2010)

Information search, behavioural economics, and relevance decisions in the online media industry: how strongly do the algorithms of intermediaries influence the relevance evaluation of information?

On Relevance in Information Retrieval Based on Relevance Theory

Relevance Estimation with Multiple Information Sources on Search Engine Result Pages.

Does Document Relevance Affect the Searcher's Perception of Time?

What Can You Perceive? Understanding User’s Information Quality Judgment on Academic Social Networking Sites

Investigating Reading Behavior in Fine-grained Relevance Judgment

Multidimensional Models of Information Need

Relevance Judgment Convergence Degree -- A Measure of Inconsistency among Assessors for Information Retrieval

Who Will You Ask? an Empirical Study of Interpersonal Task Information Seeking

The Criteria People Use in Relevance Decisions on Health Information: An Analysis of User Eye Movements When Browsing a Health Discussion Forum