Abstract:In the contemporary information era, significantly accelerated by the advent of Large-scale Language Models, the proliferation of scientific literature is reaching unprecedented levels. Researchers urgently require efficient tools for reading and summarizing academic papers, uncovering significant scientific literature, and employing diverse interpretative methodologies. To address this burgeoning demand, the role of automated scientific literature interpretation systems has become paramount. However, prevailing models, both commercial and open-source, confront notable challenges: they often overlook multimodal data, grapple with summarizing over-length texts, and lack diverse user interfaces. In response, we introduce an open-source multi-modal automated academic paper interpretation system (MMAPIS) with three-step process stages, incorporating LLMs to augment its functionality. Our system first employs the hybrid modality preprocessing and alignment module to extract plain text, and tables or figures from documents separately. It then aligns this information based on the section names they belong to, ensuring that data with identical section names are categorized under the same section. Following this, we introduce a hierarchical discourse-aware summarization method. It utilizes the extracted section names to divide the article into shorter text segments, facilitating specific summarizations both within and between sections via LLMs with specific prompts. Finally, we have designed four types of diversified user interfaces, including paper recommendation, multimodal Q\&A, audio broadcasting, and interpretation blog, which can be widely applied across various scenarios. Our qualitative and quantitative evaluations underscore the system's superiority, especially in scientific summarization, where it outperforms solutions relying solely on GPT-4.

Jack the Reader - A Machine Reading Framework

Multi-Matching Network for Multiple Choice Reading Comprehension

Feeding What You Need by Understanding What You Learned

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension.

Hierarchical Answer Selection Framework For Multi-Passage Machine Reading Comprehension

A Survey on Neural Machine Reading Comprehension

Sogou Machine Reading Comprehension Toolkit

Human Behavior Inspired Machine Reading Comprehension

StackReader: An RNN-Free Reading Comprehension Model.

Bi-directional Cognitive Thinking Network for Machine Reading Comprehension

Plug-and-Play Module for Commonsense Reasoning in Machine Reading Comprehension

Reinforced Mnemonic Reader for Machine Reading Comprehension

UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model

Bridging Research and Readers: A Multi-Modal Automated Academic Papers Interpretation System

Retrospective Reader for Machine Reading Comprehension

Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent

Question Answering for Machine Reading with Lexical Chain.

ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules

Interpretation of Natural Language Rules in Conversational Machine Reading

Multi-Paragraph Machine Reading Comprehension with Hybrid Reader over Tables and Text

Teach Machine How to Read