Abstract:Modern statistical machine learning (SML) methods share a major limitation with the early approaches to AI: there is no scalable way to adapt them to new domains. Human learning solves this in part by leveraging a rich, shared, updateable world model. Such scalability requires modularity: updating part of the world model should not impact unrelated parts. We have argued that such modularity will require both "correctability" (so that errors can be corrected without introducing new errors) and "interpretability" (so that we can understand what components need correcting). To achieve this, one could attempt to adapt state of the art SML systems to be interpretable and correctable; or one could see how far the simplest possible interpretable, correctable learning methods can take us, and try to control the limitations of SML methods by applying them only where needed. Here we focus on the latter approach and we investigate two main ideas: "Teacher Assisted Learning", which leverages crowd sourcing to learn language; and "Factored Dialog Learning", which factors the process of application development into roles where the language competencies needed are isolated, enabling non-experts to quickly create new applications. We test these ideas in an "Automated Personal Assistant" (APA) setting, with two scenarios: that of detecting user intent from a user-APA dialog; and that of creating a class of event reminder applications, where a non-expert "teacher" can then create specific apps. For the intent detection task, we use a dataset of a thousand labeled utterances from user dialogs with Cortana, and we show that our approach matches state of the art SML methods, but in addition provides full transparency: the whole (editable) model can be summarized on one human-readable page. For the reminder app task, we ran small user studies to verify the efficacy of the approach.

Automating Crowd-supervised Learning for Spoken Language Systems.

Automated Curriculum Learning for Turn-level Spoken Language Understanding with Weak Supervision

Towards Unsupervised Speech Recognition Without Pronunciation Models

SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale

Robust Speech Recognition via Large-Scale Weak Supervision

An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems

Improved training for online end-to-end speech recognition systems

Discriminative Spoken Language Understanding Using Word Confusion Networks

Robust Spoken Language Understanding With Unsupervised Asr-Error Adaptation

A Weakly Supervised Learning Approach for Spoken Language Understanding.

A Spoken English Teaching System Based on Speech Recognition and Machine Learning

Crowdsourced and Automatic Speech Prominence Estimation

Unsupervised Training and Directed Manual Transcription for LVCSR

Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognition

Exploring Retraining-Free Speech Recognition for Intra-sentential Code-Switching

Personalized language modeling by crowd sourcing with social network data for voice access of cloud applications

Self-supervised Speaker Recognition Training Using Human-Machine Dialogues

Semi-Supervision in ASR: Sequential MixMatch and Factorized TTS-Based Augmentation

Cost-efficient Crowdsourcing for Span-based Sequence Labeling: Worker Selection and Data Augmentation

A Base Camp for Scaling AI

Speech Foundation Models and Crowdsourcing for Efficient, High-Quality Data Collection