Abstract:In the contemporary information era, significantly accelerated by the advent of Large-scale Language Models, the proliferation of scientific literature is reaching unprecedented levels. Researchers urgently require efficient tools for reading and summarizing academic papers, uncovering significant scientific literature, and employing diverse interpretative methodologies. To address this burgeoning demand, the role of automated scientific literature interpretation systems has become paramount. However, prevailing models, both commercial and open-source, confront notable challenges: they often overlook multimodal data, grapple with summarizing over-length texts, and lack diverse user interfaces. In response, we introduce an open-source multi-modal automated academic paper interpretation system (MMAPIS) with three-step process stages, incorporating LLMs to augment its functionality. Our system first employs the hybrid modality preprocessing and alignment module to extract plain text, and tables or figures from documents separately. It then aligns this information based on the section names they belong to, ensuring that data with identical section names are categorized under the same section. Following this, we introduce a hierarchical discourse-aware summarization method. It utilizes the extracted section names to divide the article into shorter text segments, facilitating specific summarizations both within and between sections via LLMs with specific prompts. Finally, we have designed four types of diversified user interfaces, including paper recommendation, multimodal Q\&A, audio broadcasting, and interpretation blog, which can be widely applied across various scenarios. Our qualitative and quantitative evaluations underscore the system's superiority, especially in scientific summarization, where it outperforms solutions relying solely on GPT-4.

Open educational resource based information understanding via pdf document interaction.

Scientific Information Understanding Via Open Educational Resources (OER).

Scaffolding of Scientific Publications with Open Educational Resources (OER)

Open Educational Resources (OER) Usage and Barriers: a Study from Zhejiang University, China

College Students' Open Educational Resources (OER) Experience and Their Perceived Attributes: A Study from Zhejiang University.

Chinese Faculty Members' Open Educational Resources (OER) Usage Status and the Barriers to OER Development and Usage

Bibliometric Analysis on International Studies on Open Educational Resources and Massive Open Online Courses (MOOCs) (2000-2015)

Qlarify: Recursively Expandable Abstracts for Directed Information Retrieval over Scientific Papers

PKUSpace: A Collaborative Platform for Scientific Researching

Community-based Cyberreading for Information Understanding

Reconstructing Undergraduate Courses Based on OER in Public Universities: An Exploratory Study in Zhejiang University

The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces

A Technical Mode For Sharing And Utilizing Open Educational Resources In Chinese Universities

User-based collaborative filtering approach for content recommendation in OpenCourseWare platforms

Bridging Research and Readers: A Multi-Modal Automated Academic Papers Interpretation System

OCR++: A Robust Framework For Information Extraction from Scholarly Articles

Development of an open education resources (OER) system: a comparative analysis and implementation approach

OpenResearcher: Unleashing AI for Accelerated Scientific Research

Structure extraction from PDF-based book documents.

From Big Scholarly Data to Solution-Oriented Knowledge Repository