Abstract:Microblogging platforms have seen exponential growth, leading to an abundance of usergenerated content. The challenge now is to efficiently extract crucial information from this vast and dispersed text data. It also serves as the goal of our research on Automatic Keyphrase Extraction (AKE) for microblog. Eye-tracking signals, that reflect users' tendency to prioritize certain words while reading, have been employed to enhance AKE performance from microblogs. However, relying solely on eye-tracking has its limitations owing to constraints in physiological mechanism support, acquisition techniques, and feature decoding. Consequently, we propose the integration of electroencephalogram (EEG) signals with eye-tracking signals to improve microblogs-based AKE, thereby overcoming the aforementioned limitations. Our first step is identifying specific features present in cognitive signals generated during human reading. We selected EEG signals (8 features) and eye-tracking signals (17 features) from the cognitive language processing corpus ZUCO, to examine the efficacy when they are combined with the microblogs-based AKE. To avoid cognitive signal distortion by certain model structures, we introduced these signals at the inputs of the soft attention layer and at the query vectors of the self-attention layer. For evaluation, we performed several AKE tests on microblogs with various combinations of cognitive signals. The results demonstrate a consistent enhancement in the performance of AKE due to cognitive signals generated during human reading, regardless of different feature combinations and models. Specifically, EEG signals exhibited the most significant improvement. However, combining EEG signals with eye-tracking signals yielded results that fell between the performance levels of the two signal types, indicating that their integration might have some synergistic effects. Further investigation is needed to understand the underlying mechanisms responsible for this outcome. The code and dataset for this paper can be accessed at https://github.com/yan-xinyi/AKE.

Utilizing Cognitive Signals Generated During Human Reading to Enhance Keyphrase Extraction from Microblogs

Enhancing keyphrase extraction from microblogs using human reading time

Using Human Attention to Extract Keyphrase from Microblog Post.

Towards a Better Understanding of Human Reading Comprehension with Brain Signals

Towards a Better Understanding Human Reading Comprehension with Brain Signals

Integrating LLM, EEG, and Eye-Tracking Biomarker Analysis for Word-Level Neural State Classification in Semantic Inference Reading Comprehension

An Iterative Approach for EEG based Rapid Face Search: A Refined Retrieval by Brain Computer Interfaces

From Word Embedding to Reading Embedding Using Large Language Model, EEG and Eye-tracking

Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencoder

Integrating Large Language Model, EEG, and Eye-Tracking for Word-Level Neural State Classification in Reading Comprehension

Investigating Human Reading Behavior During Sentiment Judgment

Investigating brain mechanisms underlying natural reading by co-registering eye tracking with combined EEG and MEG

Human brain activity for machine attention

Joint Modeling of Characters, Words, and Conversation Contexts for Microblog Keyphrase Extraction

BrainECHO: Semantic Brain Signal Decoding through Vector-Quantized Spectrogram Reconstruction for Whisper-Enhanced Text Generation

EEG Signal and Feature Interaction Modeling-Based Eye Behavior Prediction Research

Combining Facial Behavioral Cues, Eye Movements and EEG-Based Attention to Improve Prediction of Reading Failure

Improve Affective Learning with EEG Approach

SEE: Semantically Aligned EEG-to-Text Translation

Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network

Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models