Design Considerations in Instrumenting and Monitoring Web-Based Information Retrieval Systems

M. Cooper

DOI: https://doi.org/10.1002/(SICI)1097-4571(199808)49:10%3C903::AID-ASI6%3E3.0.CO;2-F

1998-08-01

Journal of the American Society for Information Science

Abstract:The Internet Web environment opens up extraordinary information are changing rapidly. Daily, hundreds of opportunities for user access to information. Techniques thousands of individuals use the Internet and Web for monitoring users and systems and for evaluating sys- browsers like Netscape to obtain a much broader range tem design and performance have not kept pace with of information than has previously been possible. This Web development. This article reviews concepts of Web information consists not only of bibliographic citations operations (including browsers, clients, information retrieval applications, servers, and data communications but of full text, data, images, sound, programs, and the systems) with specific attention given to how monitoring like. The manner of searching has changed as has the should take place and how privacy can be protected. It content of search results, but the design of transaction examines monitoring needs of users, systems designers, logging programs has not caught up with these changes. managers, and customer support staff and outlines mea- Despite the changing nature of the content of informasures of workload, capacity, and performance for hardware, software, and data communications systems. Fi- tion retrieval systems, the user interface, and the data nally, the article proposes a client-server design for communications interface, a basic research question remonitoring, which involves creation of a series of server mains: Is the system effective in helping users find inforand client systems to obtain and process transaction mation? It is growing more difficult to answer this quesand computer performance information. These systems tion because interface and communications systems are include: A Log Server, which captures all levels of transactions and packets on the network; a Monitor Server, more complex. However, there is much useful informawhich synthesizes the log and packet data; an Assis- tion ideally suited for logging ( such as user actions and tance Server, which processes requests for information system performance ) and these logs can be used to supand help from the Web Server in real time; and an Ac- plement qualitative evaluations or confirm user uncercounting Server, which authenticates user access to the tainty regarding actions taken. system. A special System Administrator Client is proposed to control the monitoring system, as is a System It is extremely expensive to conduct personal interInformation Client to receive real-time and on-demand views with users to find out if they are satisfied with reports of system activity. the information retrieved and / or the information system itself. So efficient evaluation must rely on computerized methods to capture this data as unobtrusively and effec