Abstract:Measuring the coherence of text is a vital aspect of evaluating the quality of written content. Recent advancements in neural coherence modeling have demonstrated their efficacy in capturing entity coreference and discourse relations, thereby enhancing coherence evaluation. However, many existing methods heavily depend on static embeddings or focus narrowly on nearby context, constraining their capacity to measure the overarching coherence of long texts. In this paper, we posit that coherent texts inherently manifest a sequential and cohesive interplay among sentences, effectively conveying the central theme, purpose, or standpoint. To explore this abstract relationship, we introduce the "BBScore," a novel reference-free metric grounded in Brownian bridge theory for assessing text coherence. Our findings showcase that when synergized with a simple additional classification component, this metric attains a performance level comparable to state-of-the-art techniques on standard artificial discrimination tasks. We also establish in downstream tasks that this metric effectively differentiates between human-written documents and text generated by large language models under a specific domain. Furthermore, we illustrate the efficacy of this approach in detecting written styles attributed to diverse large language models, underscoring its potential for generalizability. In summary, we present a novel Brownian bridge coherence metric capable of measuring both local and global text coherence, while circumventing the need for end-to-end model training. This flexibility allows for its application in various downstream tasks.

A Graph-based Readability Assessment Method Using Word Coupling.

GRAW+: A Two‐view Graph Propagation Method with Word Coupling for Readability Assessment

An Extended Graph-Based Label Propagation Method For Readability Assessment

Coh-Metrix Model-Based Automatic Assessment of Interpreting Quality

BBScore: A Brownian Bridge Based Metric for Assessing Text Coherence

A Refined Method for Computing Bibliographic Coupling Strengths

Joint Learning of Frequency and Word Embeddings for Multilingual Readability Assessment.

Enriching Word Embeddings with Domain Knowledge for Readability Assessment.

Exploring the Impact of Linguistic Features for Chinese Readability Assessment.

A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss

Linguistic Feature Representation with Statistical Relational Learning for Readability Assessment.

Generalized word shift graphs: a method for visualizing and explaining pairwise comparisons between texts

Assessment of text coherence based on the cohesion estimation

Cross-corpus Readability Compatibility Assessment for English Texts

Research of Software Module Coupling Measure Method Based on Weighted Coupling Graph

Analyzing program readability based on WordNet

EFL Text Readability Assessment Based on Feature Engineering

ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis

Sentence-Level Readability Assessment for L2 Chinese Learning

Adaptive Approach to Concordance Readability

Readability Assessment of Textbooks in Low Resource Languages