Abstract:The introduction of Transformer architectures - with the self-attention mechanism - in automatic Natural Language Generation (NLG) is a breakthrough in solving general task-oriented problems, such as the simple production of long text excerpts that resemble ones written by humans. While the performance of GPT-X architectures is there for all to see, many efforts are underway to penetrate the secrets of these black-boxes in terms of intelligent information processing whose output statistical distributions resemble that of natural language. In this work, through the complexity science framework, a comparative study of the stochastic processes underlying the texts produced by the English version of GPT-2 with respect to texts produced by human beings, notably novels in English and programming codes, is offered. The investigation, of a methodological nature, consists first of all of an analysis phase in which the Multifractal Detrended Fluctuation Analysis and the Recurrence Quantification Analysis - together with Zipf's law and approximate entropy - are adopted to characterize long-term correlations, regularities and recurrences in human and machine-produced texts. Results show several peculiarities and trends in terms of long-range correlations and recurrences in the last case. The synthesis phase, on the other hand, uses the complexity measures to build synthetic text descriptors - hence a suitable text embedding - which serve to constitute the features for feeding a machine learning system designed to operate feature selection through an evolutionary technique. Using multivariate analysis, it is then shown the grouping tendency of the three analyzed text types, allowing to place GTP-2 texts in between natural language texts and computer codes. Similarly, the classification task demonstrates that, given the high accuracy obtained in the automatic discrimination of text classes, the proposed set of complexity measures is highly informative. These interesting results allow us to add another piece to the theoretical understanding of the surprising results obtained by NLG systems based on deep learning and let us to improve the design of new informetrics or text mining systems for text classification, fake news detection, or even plagiarism detection.

Automatic Classification of Text Complexity

Text Complexity Classification Based on Linguistic Information: Application to Intelligent Tutoring of ESL

Morpho-Phraseological Based Classification of CEFR Italian L2 Learner Writing Proficiency

Automatic Classification of the Complexity of Nonfiction Texts in Portuguese for Early School Years

Text classification by CEFR levels using machine learning methods and BERT language model

A survey on text classification: Practical perspectives on the Italian language.

A Readable Read: Automatic Assessment of Language Learning Materials based on Linguistic Complexity

Efficient Measuring of Readability to Improve Documents Accessibility for Arabic Language Learners

Controlling Text Complexity in Neural Machine Translation

Towards Automated Website Classification by Deep Learning

Linguistic Features Distinguishing Students' Writing Ability Aligned with CEFR Levels

Research on Automatic Text Classification Based on a Hybrid Language Model

Experiments with Universal CEFR Classification

Predicting CEFRL levels in learner English on the basis of metrics and full texts

A Comparative Analysis of Word Embeddings Techniques for Italian News Categorization

Human versus Machine Intelligence: Assessing Natural Language Generation Models through Complex Systems Theory

Machine learning in automated text categorization

Exploring the Use of Text Classification in the Legal Domain

Classifying text using machine learning models and determining conversation drift

Text characterization based on recurrence networks

Automatic Classification and Comparison of Words by Difficulty.