Abstract:Momentive offers solutions in market research, customer experience, and enterprise feedback. The technology is gleaned from the billions of real responses to questions asked on the platform. However, people may create biased questions. A double-barreled question (DBQ) is a common type of biased question that asks two aspects in one question. For example, "Do you agree with the statement: The food is yummy, and the service is great.". This DBQ confuses survey respondents because there are two parts in a question. DBQs impact both the survey respondents and the survey owners. Momentive aims to detect DBQs and recommend survey creators to make a change towards gathering high quality unbiased survey data. Previous research work has suggested detecting DBQs by checking the existence of grammatical conjunction. While this is a simple rule-based approach, this method is error-prone because conjunctions can also exist in properly constructed questions. We present an end-to-end machine learning approach for DBQ classification in this work. We handled this imbalanced data using active learning, and compared state-of-the-art embedding algorithms to transform text data into vectors. Furthermore, we proposed a model interpretation technique propagating the vector-level SHAP values to a SHAP value for each word in the questions. We concluded that the word2vec subword embedding with maximum pooling is the optimal word embedding representation in terms of precision and running time in the offline experiments using the survey data at Momentive. The A/B test and production metrics indicate that this model brings a positive change to the business. To the best of our knowledge, this is the first machine learning framework for DBQ detection, and it successfully differentiates Momentive from the competitors. We hope our work sheds light on machine learning approaches for bias question detection.

EBAMQ: Exposure Bias Alleviation via Moment Matching for Question Generation.

Mixture of Experts for Biomedical Question Answering

Unified Question Generation with Continual Lifelong Learning

Double-Barreled Question Detection at Momentive

Mitigating Bias for Question Answering Models by Tracking Bias Influence

Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation

Synthetic Multimodal Question Generation

Extended Answer and Uncertainty Aware Neural Question Generation

AMQAN: Adaptive Multi-Attention Question-Answer Networks for Answer Selection

EEE-QA: Exploring Effective and Efficient Question-Answer Representations

QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluation

CFMatch: Aligning Automated Answer Equivalence Evaluation with Expert Judgments For Open-Domain Question Answering

Explicit Diversity Conditions for Effective Question Answer Generation with Large Language Models

MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization

QGEval: Benchmarking Multi-dimensional Evaluation for Question Generation

A Contrastive Framework for Enhancing Knowledge Graph Question Answering: Alleviating Exposure Bias

Hypersphere Embedding and Additive Margin for Query-by-example Keyword Spotting

Essay-Anchor Attentive Multi-Modal Bilinear Pooling for Textbook Question Answering

EQG-RACE: Examination-Type Question Generation

Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering

QD-VMR: Query Debiasing with Contextual Understanding Enhancement for Video Moment Retrieval