Abstract:Real-world problems may contain latent dependencies (ie, hidden sub-structures) that are difficult to capture with conventional structured classifiers, such as conditional random fields. In such cases, models that exploit latent variables are advantageous in learning, and conditional latent variable models have been applied successfully into real-world tasks in natural language processing and vision processing communities, including syntactic parsing and vision recognition. In the first part of this thesis, I perform experiments in a variety of tasks to confirm the advantages of conditional latent variable models, and investigate what kind of latent dependencies are learned by the model. While the experiments confirmed the advantages of conditional latent variable models, the same experiments revealed two problems in applying such models for practical usages. First, establishing an efficient inference method on latent conditional models remains an open question. Second, because of the incorporation of latent variables, training a latent conditional model brings a heavy computational cost. To deal with those critical problems, I propose efficient methods for inference and training on latent conditional models.In the inference stage, I propose the “latent-dynamic inference”, which is able to produce the exact optimal label sequence on latent conditional models by systematically combining efficient search strategy (the A* search) and dynamic programming (the Forward-Backward method). I also describe a straightforward solution on approximating the exact method, and show that the approximated version performs as well as the exact one, while the speed …

Exact Decoding on Latent Variable Conditional Models is NP-Hard

Sequential Labeling with Latent Variables: an Exact Inference Algorithm and Its Efficient Approximation

EFFICIENT INFERENCE AND TRAINING FOR CONDITIONAL LATENT VARIABLE MODELS¡¢£¤¥¦ § ©

A Unified Theory of Exact Inference and Learning in Exponential Family Latent Variable Models

A Thorough Examination of Decoding Methods in the Era of LLMs

Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding

Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Modeling Latent-Dynamic in Shallow Parsing

Latent Variable Double Gaussian Process Model for Decoding Complex Neural Data

Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability

Speculative Contrastive Decoding

Deep Recurrent Modelling of Granger Causality with Latent Confounding

Mitigating the Influence of Distractor Tasks in LMs with Prior-Aware Decoding

Conditional Random Fields with Decode-based Learning: Simpler and Faster

Graph-Structured Speculative Decoding

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time

On Speculative Decoding for Multimodal Large Language Models

Decoding Speculative Decoding

Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs