Abstract:The questionnaire is a professional research methodology used for both qualitative and quantitative analysis of human opinions, preferences, attitudes, and behaviors. However, designing and evaluating questionnaires demands significant effort due to their intricate and complex structure. Questionnaires entail a series of questions that must conform to intricate constraints involving the questions, options, and overall structure. Specifically, the questions should be relevant and specific to the given research topic and intent. The options should be tailored to the questions, ensuring they are mutually exclusive, completed, and ordered sensibly. Moreover, the sequence of questions should follow a logical order, grouping similar topics together. As a result, automatically generating questionnaires presents a significant challenge and this area has received limited attention primarily due to the scarcity of high-quality datasets. To address these issues, we present Qsnail, the first dataset specifically constructed for the questionnaire generation task, which comprises 13,168 human-written questionnaires gathered from online platforms. We further conduct experiments on Qsnail, and the results reveal that retrieval models and traditional generative models do not fully align with the given research topic and intents. Large language models, while more closely related to the research topic and intents, exhibit significant limitations in terms of diversity and specificity. Despite enhancements through the chain-of-thought prompt and finetuning, questionnaires generated by language models still fall short of human-written questionnaires. Therefore, questionnaire generation is challenging and needs to be further explored. The dataset is available at: https://github.com/LeiyanGithub/qsnail.

For those who don't know (how) to ask: Building a dataset of technology questions for digital newcomers

Researchy Questions: A Dataset of Multi-Perspective, Decompositional Questions for LLM Web Agents

Information Seeking in the Spirit of Learning: a Dataset for Conversational Curiosity

Knowing When to Ask -- Bridging Large Language Models and Data

QACP: An Annotated Question Answering Dataset for Assisting Chinese Python Programming Learners

Understanding the Dataset Practitioners Behind Large Language Model Development

DELPHI: Data for Evaluating LLMs' Performance in Handling Controversial Issues

Learning to Ask: When LLMs Meet Unclear Instruction

AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMs

SyllabusQA: A Course Logistics Question Answering Dataset

A dataset of questions on decision-theoretic reasoning in Newcomb-like problems

NewsInterview: a Dataset and a Playground to Evaluate LLMs' Ground Gap via Informational Interviews

Do LLMs Find Human Answers To Fact-Driven Questions Perplexing? A Case Study on Reddit

A Dataset for Learning University STEM Courses at Scale and Generating Questions at a Human Level

Qsnail: A Questionnaire Dataset for Sequential Question Generation

I Could've Asked That: Reformulating Unanswerable Questions

Learning-to-Ask: Knowledge Acquisition Via 20 Questions

An Applied Statistics dataset for human vs AI-generated answer classification

Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method

Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

Dr.Academy: A Benchmark for Evaluating Questioning Capability in Education for Large Language Models