The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces

Kyle Lo,Joseph Chee Chang,Andrew Head,Jonathan Bragg,Amy X. Zhang,Cassidy Trier,Chloe Anastasiades,Tal August,Russell Authur,Danielle Bragg,Erin Bransom,Isabel Cachola,Stefan Candra,Yoganand Chandrasekhar,Yen-Sung Chen,Evie Yu-Yen Cheng,Yvonne Chou,Doug Downey,Rob Evans,Raymond Fok,Fangzhou Hu,Regan Huff,Dongyeop Kang,Tae Soo Kim,Rodney Kinney,Aniket Kittur,Hyeonsu Kang,Egor Klevak,Bailey Kuehl,Michael Langan,Matt Latzke,Jaron Lochner,Kelsey MacMillan,Eric Marsh,Tyler Murray,Aakanksha Naik,Ngoc-Uyen Nguyen,Srishti Palani,Soya Park,Caroline Paulic,Napol Rachatasumrit,Smita Rao,Paul Sayre,Zejiang Shen,Pao Siangliulue,Luca Soldaini,Huy Tran,Madeleine van Zuylen,Lucy Lu Wang,Christopher Wilhelm,Caroline Wu,Jiangjiang Yang,Angele Zamarron,Marti A. Hearst,Daniel S. Weld

2023-04-23

Abstract:Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the need for new technology to support the reading process grows. In contrast to the process of finding papers, which has been transformed by Internet technology, the experience of reading research papers has changed little in decades. The PDF format for sharing research papers is widely used due to its portability, but it has significant downsides including: static content, poor accessibility for low-vision readers, and difficulty reading on mobile devices. This paper explores the question "Can recent advances in AI and HCI power intelligent, interactive, and accessible reading interfaces -- even for legacy PDFs?" We describe the Semantic Reader Project, a collaborative effort across multiple institutions to explore automatic creation of dynamic reading interfaces for research papers. Through this project, we've developed ten research prototype interfaces and conducted usability studies with more than 300 participants and real-world users showing improved reading experiences for scholars. We've also released a production reading interface for research papers that will incorporate the best features as they mature. We structure this paper around challenges scholars and the public face when reading research papers -- Discovery, Efficiency, Comprehension, Synthesis, and Accessibility -- and present an overview of our progress and remaining open challenges.

Human-Computer Interaction,Artificial Intelligence,Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: With the explosive growth of scientific research literature and the increasing complexity of interdisciplinary research, scholars face challenges in reading and understanding research papers. Specifically, the paper explores how to use the latest artificial intelligence (AI) and human - computer interaction (HCI) technologies to create intelligent, interactive, and accessible reading interfaces to improve the reading experience of academic papers.

### Specific manifestations of the problem

1. **Discovery**:
- When conducting literature reviews, scholars need to handle a large number of cited references, which makes it very difficult to track and prioritize relevant literature.

2. **Efficiency**:
- With the exponential growth of the number of papers, scholars have difficulty keeping up with the latest research results and need to efficiently browse and read a large number of papers.

3. **Comprehension**:
- Research papers are usually content - dense, containing many terms and technical details, which pose a challenge to readers, especially those outside the professional field.

4. **Synthesis**:
- Integrating knowledge scattered in multiple papers to form a comprehensive understanding is a labor - intensive but important task, which helps to identify future research opportunities.

5. **Accessibility**:
- The static PDF format is not suitable for many reading interfaces. In particular, it is very difficult for visually impaired people and scholars using mobile devices to read PDF - format papers.

### Solutions in the paper

To solve the above problems, the author proposes the "Semantic Reader Project", which is a multi - institutional collaborative effort aimed at improving the reading experience of academic papers in the following ways:

- **Develop intelligent and interactive reading interfaces**: Use AI technology to parse the content of PDF documents and provide dynamic reading assistance functions.
- **Improve accessibility**: Ensure that the new reading interface can better support assistive tools such as screen readers and adapt to reading needs on different devices.
- **Enhance user interaction**: Through functions such as visualization enhancement and personalized recommendation, help readers explore and understand literature more efficiently.
- **Integrate existing resources**: Integrate existing scientific research achievements and tools into the new reading interface, such as automatic summarization and annotation systems.

### Specific implementation

The paper details ten research prototype systems, such as CiteSee, CiteRead, Scim, Ocean, etc. These systems respectively provide innovative solutions for different reading challenges. For example:

- **CiteSee**: By highlighting cited references related to the reader's existing interests, it helps the reader prioritize important literature.
- **CiteRead**: By showing relevant comments of subsequent research papers in the annotation sidebar, it helps the reader understand subsequent work.
- **Scim**: By automated area - based highlighting, it guides the reader to focus on the key information of the paper.
- **Ocean**: Provides navigation support for low - vision users and allows readers to create and share hyperlinks.

In general, this paper aims to significantly improve the reading experience of academic papers through technological innovation, enabling scholars to obtain and understand scientific knowledge more efficiently.