Abstract:Large Language Models are transformer neural networks which are trained to produce a probability distribution on the possible next words to given texts in a corpus, in such a way that the most likely word predicted is the actual word in the training text. In this paper we find what is the mathematical structure defined by such conditional probability distributions of text extensions. Changing the view point from probabilities to -log probabilities we observe that the subtext order is completely encoded in a metric structure defined on the space of texts $\mathcal{L}$, by -log probabilities. We then construct a metric polyhedron $P(\mathcal{L})$ and an isometric embedding (called Yoneda embedding) of $\mathcal{L}$ into $P(\mathcal{L})$ such that texts map to generators of certain special extremal rays. We explain that $P(\mathcal{L})$ is a $(\min,+)$ (tropical) linear span of these extremal ray generators. The generators also satisfy a system of $(\min+)$ linear equations. We then show that $P(\mathcal{L})$ is compatible with adding more text and from this we derive an approximation of a text vector as a Boltzmann weighted linear combination of the vectors for words in that text. We then prove a duality theorem showing that texts extensions and text restrictions give isometric polyhedra (even though they look a priory very different). Moreover we prove that $P(\mathcal{L})$ is the lattice closure of (a version of) the so called, Isbell completion of $\mathcal{L}$ which turns out to be the $(\max,+)$ span of the text extremal ray generators. All constructions have interpretations in category theory but we don't use category theory explicitly. The categorical interpretations are briefly explained in an appendix. In the final appendix we describe how the syntax to semantics problem could fit in a general well known mathematical duality.

Lattice and Metric Completions of the Classical Logic Metric Space and a Comparison.

Stone-Gelfand duality for metrically complete lattice-ordered groups

The Lattice of Sahlqvist Logics

The c-completion of Lorentzian metric spaces

Tense Logics over Lattices

Choice-free Dualities for Lattice Expansions: Application to Logics with a Negation Operator

Directed Metric Structures arising in Large Language Models

Compatibility and accessibility: lattice representations for semantics of non-classical and modal logics

Propositional Logics for the Lawvere Quantale

Metric completions of triangulated categories from finite dimensional algebras

Continuous logic in a classical setting

Constructing Fully Complete Models of Multiplicative Linear Logic

Homotopical Models for Metric Spaces and Completeness

Lattice-theoretic Models of Conjectures, Hypotheses and Consequences

Uniform Residuated Lattices and their Cauchy Completions

Metric completions of discrete cluster categories

Complete relations on fuzzy complete lattices

Monadic ortholattices: completions and duality

The Metric Completion of the Space of Vector-Valued One-Forms

A NOTE ON LATTICES OF QUOTIENTS OF COMPLETE BOOLEAN ALGEBRAS

Barr-coexactness for metric compact Hausdorff spaces