Abstract:Due to the rapid development of the World Wide Web (WWW), Web-based search engines have become the most, important tools for accessing global information resources on the Internet. Although these search engines are easily accessible and highly popular, systematic evaluation of their performance based on real users and their information problems has not been conducted. This study describes a pilot study using a systematic methodology proposed by Su (1997) to gather such data. Measures used to assess four Web-based search engines are based on five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Eleven participants, who were faculty, staff, and graduate students at the University of Pittsburgh, searched for their individual information topics on four search engines using simple search form. They tested two randomly selected hyperlinks from each set of twenty items retrieved for each engine. They made relevance assessments of the first twenty items retrieved by their best strategy on each engine. They chose and ranked five most relevant-items from each set of the twenty items in relation to their individual information problems. Preliminary data analysis reveals that none of the four engines dominates in every aspect of the multidimensional evaluation. The study results shows that even though Lycos has the best relevance ranking performance and the highest precision ratio, Alta Vista has the best connectivity, the highest user's satisfaction ratios for precision and for three components of system features, and the highest value of the search result as a whole. One-way ANOVA test indicates that user's satisfaction ratings with online documentation (F=9.97 gr p=0.000) and value of search results as a whole (F=3.65 & p=0.023) are significantly different among the four search engines.

Are self-assessment of search ability and performance reliable?

"Satisfaction with Failure" or "Unsatisfied Success": Investigating the Relationship between Search Success and User Satisfaction.

How do Information Source Selection Strategies Influence Users' Learning Outcomes'.

Evaluation Of Web-Based Search Engines From The End-User'S Perspective: A Pilot Study

A Comparative Study of the Relationship between the Subjective Difficulty, Objective Difficulty of Search Tasks and Search Behaviors

From organization to elaboration: relationships between university students’ online information searching experience and judgements

Information searching strategies in web-based science learning: the role of internet self-efficacy

A Day Without a Search Engine: an Experimental Study of Online and Offline Searches

Identification of Information Search Strategies and Their Impact on Learning Outcome in Learning‐related Tasks

AN INVESTIGATION OF SCORER RELIABILITY OF SELF AND PEER ASSESSMENT OF EFL WRITING AMONG CHINESE COLLEGE STUDENTS

Satisfaction Prediction of Web Search Users

Meta-evaluation of Online and Offline Web Search Evaluation Metrics

The accuracy of student self-assessments of English-Chinese bidirectional interpretation: a longitudinal quantitative study

The interplay between internet searching styles and academic self-efficacy: a longitudinal study

Still Haven't Found What You're Looking For -- Detecting the Intent of Web Search Missions from User Interaction Features

How Well Do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?

Research on the Relationship Between Exploratory Search Task Attributes and Information Search Behavior

To Search is to Believe? A Comparative Study of Health Information Use by Internet Users.

Are Search Engine Users Equally Reliable?

Exploring and predicting search task difficulty.

An Exploratory Study on the Re-finding Behavior on the Web