FastLexRank: Efficient Lexical Ranking for Structuring Social Media Posts

Mao Li,Frederick Conrad,Johann Gagnon-Bartsch
2024-10-02
Abstract:We present FastLexRank\footnote{<a class="link-external link-https" href="https://github.com/LiMaoUM/FastLexRank" rel="external noopener nofollow">this https URL</a>}, an efficient and scalable implementation of the LexRank algorithm for text ranking. Designed to address the computational and memory complexities of the original LexRank method, FastLexRank significantly reduces time and memory requirements from $\mathcal{O}(n^2)$ to $\mathcal{O}(n)$ without compromising the quality or accuracy of the results. By employing an optimized approach to calculating the stationary distribution of sentence graphs, FastLexRank maintains an identical results with the original LexRank scores while enhancing computational efficiency. This paper details the algorithmic improvements that enable the processing of large datasets, such as social media corpora, in real-time. Empirical results demonstrate its effectiveness, and we propose its use in identifying central tweets, which can be further analyzed using advanced NLP techniques. FastLexRank offers a scalable solution for text centrality calculation, addressing the growing need for efficient processing of digital content.
Computation and Language,Computation
What problem does this paper attempt to address?