# AI Moderated Survey question setup UX (VOXDEV-3733) Market Research Report - Global

**Generated on:** 2026-10-19 11:47:42.612835  
**Industry:** AI Moderated Survey question setup UX (VOXDEV-3733)  
**Geography:** Global  
**Details:** # Deep Research Prompt: AI Moderated Survey question setup (VOXDEV-3733)

## Your role

You are a senior UX researcher and research methodologist with expertise in survey design, qualitative research, conversational AI and configuring AI agents. Produce an evidence-based report that a product designer can act on directly. Be impartial. Do not favour either design option below unless the evidence supports it. Where evidence is thin, say so plainly rather than filling gaps with assumptions.

## Context

Voxpopme is a video research platform used by enterprise market researchers and insight teams. We are about to release an **AI Moderated Survey** (renamed from "Guided Survey" in October 2026).

- A researcher writes a short survey.
- Respondents answer each question asynchronously by video, voice or text.
- An AI listens to each answer and asks written follow-up questions ("probes") before moving on.
- It can mix rating questions with open questions that the AI probes.
- It is used for concept tests and quali-quant work.
- It sits alongside our **AI Moderated Interview**, a live voice-to-voice conversation.

**The problem:** internal users like the product, but question setup is too complicated to release. Each question card currently asks for:

- Question type
- Stimulus
- Question
- Objective
- Advanced options: Listen for signals, Probe hints (kept minimal by design), Max probes, Probe depth (light or deep)

Most of these fields describe how the AI moderator works rather than what the researcher wants to learn. Our own designer could not explain what difference "how many probes" versus "light or deep" makes to the agent's behaviour.

**Known constraints and observations:**

- Question text is capped at 120 characters by the panel system.
- Probe count drives interview length, which drives respondent incentive cost.
- In internal testing, two probes per question made the survey feel long. One may be the right default.
- In testing, a "5 question" survey became roughly 12 questions once probes were added, and respondents felt this was dishonest.

**Two redesign candidates:**

- **Design A (three parts):**
  1. Question
  2. Objective: one free-text brief, hint text: "Brief the moderator like a colleague. Why you're asking this, what counts as a full answer, what to dig into, what to leave alone."
  3. Follow-up effort slider from Light to Thorough, showing "About 2 follow-ups"
- **Design B (three parts):**
  1. "What do you want to learn?"
  2. "You'll know it's answered when they mention..." (structured listen-for signals)
  3. "How much should the AI dig in?" A little / Some / A lot

**Other ideas under consideration:**

- AI-suggested objectives and signals generated from the question, which the researcher edits
- Survey-level follow-up defaults with per-question overrides
- A "preview the conversation" example on each question card

## Research sections

Answer every question. Cite a source for each factual claim and give a confidence level (High, Medium, Low) for each finding.

### 1. Free-text brief versus structured signals (AI output quality)

1. What evidence exists on whether LLM agents produce better follow-up questions from a free-text goal or brief, from structured criteria (such as completion signals or rubrics), or from a combination?
2. How do prompt engineering guides from model providers (Anthropic, OpenAI, Google) recommend briefing an agent with goals versus explicit success criteria?
3. Is there research on AI interviewers or chatbot interviewers where the instruction format was varied and probe quality measured?
4. What failure modes appear with each approach (for example, rigid checklist probing, leading questions, ignoring the brief, over-probing on irrelevant tangents)?
5. Which approach transfers best to non-expert researchers who may write vague objectives?

### 2. Researcher setup effort and cognitive load

1. Which approach (free-text brief, structured signals, hybrid) is faster and easier to complete for researchers, based on UX evidence from survey tools, form design or AI configuration tools?
2. How do researchers naturally articulate a research objective, and how close is that to each design's wording?
3. What does form design research say about blank free-text fields versus guided, scaffolded fields?
4. How much setup effort will researchers tolerate per question in a 5 to 10 question survey?

### 3. The follow-up control: slider versus named levels

1. Is a continuous slider with an estimated count ("About 2 follow-ups") or three named levels (A little / Some / A lot) clearer and more predictable for users?
2. Should the interface show a number of follow-ups when the AI decides dynamically whether to probe? What does research on expectation setting and trust in AI systems say about showing estimates that may not match outcomes?
3. Are "how many" (count) and "how deep" (depth) genuinely separate dimensions users can reason about, or should they be merged into one control?
4. What evidence exists from other AI products that expose "effort", "creativity" or "thoroughness" controls (for example temperature, reasoning effort, search depth settings), and how well do users understand them?
5. How should the control communicate cost and respondent time implications?

### 4. Defaults, survey-level settings and overrides

1. What default probe count and depth do AI-moderated research tools use, and what evidence supports a default of one versus two follow-ups?
2. When is a survey-level default with per-question overrides better than per-question settings only?
3. What does research on defaults (default effects, choice architecture) suggest about how many researchers will change them?
4. How should a rating question differ from an open question in terms of default probing?

### 5. Progressive disclosure and AI pre-fill

1. What patterns work best for hiding advanced AI configuration without losing expert control?
2. What evidence exists on AI pre-filled fields (suggested objectives or signals) in professional tools? Do users review and edit them, or accept them uncritically (automation bias)?
3. How should AI suggestions be presented so researchers stay accountable for the research design?
4. How useful is a "preview the conversation" or simulated respondent example for helping users understand agent behaviour? Are there examples from chatbot builders, AI agent builders or survey tools?

### 6. Naming and explaining probing

1. What terms do non-expert researchers understand: "probe", "follow-up", "dig deeper", "clarifying question" or others?
2. How do competitors and research tools name this concept in their interface?
3. What microcopy patterns help users understand what an AI moderator will do with their instructions?
4. Which terms work in both US and UK English without sounding unnatural in either?

### 7. Competitor setup flows

Investigate how these tools set up a question, its objective and its follow-up rules: **Outset, Listen Labs, Conveo, Qualtrics (AI follow-up features), Strella, Glaut**, plus any other AI-moderated research tools you find (for example Remesh, CloudResearch Engage, Knit, Wondering, UserCall, Voxpopme competitors in video research).

For each:

1. What fields does a researcher fill in per question?
2. Is there a research objective or brief, and at what level (study, section, question)?
3. What follow-up controls are exposed (count, depth, toggles, instructions), and what is hidden?
4. What are the defaults?
5. Do they offer AI-generated discussion guides, objectives or probes?
6. Do they offer a preview or test mode?
7. Do they support asynchronous video, voice and text, or only some?

### 8. Competitor reception

1. What do users say in reviews (G2, Capterra, Reddit, LinkedIn, Quirk's, Greenbook, ESOMAR content) about setup complexity and follow-up quality?
2. What complaints recur about AI moderators (repetitive probes, irrelevant probes, too many follow-ups, robotic tone)?
3. Separate vendor marketing claims from independent evidence and label which is which.

### 9. Respondent experience: probe count and answer quality

1. How do the number and depth of follow-ups affect answer quality, richness and relevance in asynchronous surveys?
2. At what point do follow-ups cause fatigue, shorter answers, satisficing or drop-off?
3. Does the effect differ by response mode (video, voice, text)?
4. What academic evidence exists on conversational surveys and chatbot surveys (for example studies on AI-driven probing in open-ended survey questions) and what did they find on engagement and data quality?
5. Do rating questions benefit from a follow-up as much as open questions?

### 10. Progress display when question count is unpredictable

1. How should progress be shown when probes make the total number of questions unpredictable?
2. What does research on progress indicators say (for example, misleading or slow-moving progress bars increasing drop-off)?
3. Should follow-ups be counted as questions, nested within a parent question, or not shown at all?
4. How do competitors handle this?
5. What upfront time estimate wording sets honest expectations?

### 11. Pacing, wait times and probe wording

1. How long can a respondent wait for an AI follow-up to generate before abandonment rises?
2. What loading or waiting patterns reduce perceived wait time?
3. How should AI follow-ups be worded and framed (acknowledgement, length, tone, signalling that it is a follow-up, offering a skip option)?
4. Should respondents be told an AI is asking the follow-up, and how does disclosure affect answers?

### 12. Constraints, edge cases and failure modes

1. How should the 120 character question limit interact with objectives and probes?
2. How do incentive costs scale with probe count, and how do other platforms communicate cost to researchers?
3. What happens when the objective is vague, contradictory or missing?
4. What happens when respondents give very short, off-topic, low-effort or inappropriate answers?
5. What quality safeguards (for example a maximum probe cap regardless of setting) are standard?
6. What accessibility considerations apply to follow-up questions in video and voice modes?

## Output instructions

1. **Executive summary** (one page maximum): your recommendation on Design A, Design B or a hybrid, with the three strongest reasons.
2. **Recommended question card**: exact field structure, field labels, hint text and placeholder text. Note which fields are visible by default and which are hidden.
3. **Follow-up control recommendation**: control type, labels, default value, whether to show a number, and survey-level versus per-question behaviour.
4. **What to hide or remove** from the current setup, with reasoning.
5. **Competitor comparison table** with columns: Tool, Question fields, Objective level, Follow-up controls exposed, Defaults, AI pre-fill, Preview/test mode, Modes supported, Source, Date checked.
6. **Respondent guidance**: recommended probe count, progress display pattern, time estimate wording and probe wording guidelines.
7. **Evidence table**: each key finding with source, source type (academic, independent industry, vendor, review site, opinion), date and confidence level.
8. **Evidence gaps**: list where evidence is thin or missing, and suggest what Voxpopme should test itself to fill each gap (for example a small A/B test or usability test).
9. **Surprises and counter-evidence**: anything that challenges the assumptions in this brief.

**Rules:**

- Prioritise sources from 2023 onwards. AI-moderated research is changing quickly, so state the date of each competitor finding.
- Do not invent competitor features. If you cannot verify a feature, say "not verified".
- Clearly separate evidence from your own reasoning.
- Write in UK English, except proposed interface copy, which must read naturally in both US and UK English.
- Do not use em dashes.

---

# Simplifying AI Moderated Survey Setup Without Sacrificing Quality

## Executive Summary

**Recommendation: develop a progressive hybrid, rather than release Design A or Design B unchanged.** Keep the participant-facing question, add a short researcher-facing goal, make completion criteria optional, and replace vague effort settings with explicit follow-up limits. **Confidence: Medium.** No directly comparable experiment located establishes that either proposed card produces better research.

- **Strongest Reason 1 - Intent And Criteria Are Complementary**: OpenAI and Google recommend clear instructions, context and constraints; Anthropic recommends measurable success criteria. None establishes that a mandatory signals checklist beats a well-written brief -> support both, without requiring both on every question. [11][13][10]
- **Strongest Reason 2 - Behaviour Must Be Predictable**: Listen Labs exposes follow-up counts, while Conveo documents bounded ranges. These are more concrete specifications than an unexplained Light/Thorough slider -> show **Off / Up to 1 / Up to 2**, with **Up to 1** as the initial open-question hypothesis. [8][18]
- **Strongest Reason 3 - More Probing Is Not Necessarily Better**: A **64-participant** experiment finds differences between probe strategies, including greater perceived repetition for explanatory probing -> optimise for relevant new information, not conversational length. [20]
- **Defaults Need Restraint**: A large default-effects meta-analysis supports defaults influencing choices, but cannot tell Voxpopme what proportion of researchers will change a setting -> choose a conservative default and make overrides visible. [29]
- **Preview Is A Practical Differentiator**: Listen Labs and Strella document respondent previews; CloudResearch Engage offers manual and simulated question tests -> put an interactive preview on each card, labelled as an example rather than a guarantee. [8][2][36]
- **Respondent Honesty Is A Release Requirement**: The supplied internal example turns five advertised questions into roughly twelve interactions -> describe **main questions plus possible follow-ups**, and use measured time ranges rather than an artificially small question count.
- **Evidence Has Boundaries**: There is no established universal optimum of one versus two probes, no validated setup-time tolerance per question, and no defensible universal generation-delay abandonment threshold. Treat these as product experiments, not industry facts.

**Scope:** global product-UX and research-methodology assessment for VOXDEV-3733. Competitor findings were checked on **19 October 2026**. The 120-character limit, product rename and internal testing observations are supplied Voxpopme context, not independently published results. **High** confidence means strong support for the stated, scoped finding; **Medium** means indirect or limited evidence; **Low** means an untested product hypothesis or substantial evidence gap. Recommendations below are explicitly distinguished from observed findings.

## 1. Briefs And Signals: Combine Intent With Optional Evidence Criteria

**Finding - Medium confidence:** a hybrid is the best-supported design direction, but its superiority over A or B is not experimentally established. Provider documentation supports clear goals alongside context, constraints and examples, rather than prescribing a particular form widget. Anthropic's success criteria concern evaluating an application; they are not an instruction to make interviewees mention predetermined words. [11][13][10]

| Approach | Mechanism | Potential benefit | Principal failure risk | Evidence for winning this specific comparison |
|---|---|---|---|---|
| Free-text brief, as in A | Communicates intent, context and boundaries together | Expressive; accommodates unfamiliar research topics | Vague or contradictory goals; buried priorities; ambiguous stopping | Low: no matched survey-setup trial located |
| Structured signals, as in B | Makes desired coverage explicit | Can clarify what information is missing | Checklist interviewing; keyword matching; leading participants towards expected themes | Low: no matched survey-setup trial located |
| Progressive hybrid | Uses a goal by default and adds criteria only when useful | Preserves context while enabling specific coverage and stopping rules | More fields if progressive disclosure fails | Medium as a design hypothesis; not demonstrated superiority |

The implication is not to maximise instructions. It is to give the moderator enough information to distinguish an incomplete answer from an unexpected but complete answer.

### What Model Providers Actually Recommend

| Provider | Verified guidance | Implication for Voxpopme |
|---|---|---|
| OpenAI | Be specific about context and outcome; separate instructions from context; show desired requirements; state what to do, not only prohibitions. [11] | A short goal plus explicit behavioural constraints is reasonable |
| Anthropic | Define specific, measurable, achievable and relevant success criteria; evaluate multiple dimensions where needed. [10] | Evaluate relevance, neutrality and stopping, not merely whether signals appeared |
| Google | Give clear instructions, context, constraints and examples; iterate for the task; different response formats can be appropriate. [13] | Do not confuse internal prompt structure with mandatory researcher fields |

These guides support the information architecture, not a causal claim that one layout improves interview quality. **Confidence: High for the documentation; Low for a direct A-versus-B inference.**

### Case Study: Probe Strategy Matters More Than A Generic Demand For Depth

The 2025 four-probe study compares descriptive, idiographic, clarifying and explanatory strategies with 64 participants. Idiographic probes, which elicit specific experiences, perform best overall; descriptive probes also perform well in exploration and requirements tasks. Explanatory probes perform less well on several relevance and specificity comparisons, and are perceived as more repetitive. [20]

This is an experiment on **probe strategies**, not free-text versus structured configuration. Its practical lesson is nevertheless useful: asking for a concrete occasion can add more information than asking another abstract "why". The study's technology-stress topic limits transfer to concept testing and other commercial research. [20]

**Recommendation:** retain a short goal and offer optional semantic criteria, phrased as **"What would make this answer useful?"** Include examples, boundaries and stopping conditions where necessary. Do not turn "You'll know it's answered when they mention..." into a mandatory list of expected themes. An answer such as "None of these apply" must be capable of satisfying the research goal.

## 2. Researcher Effort And The Recommended Question Card

**Finding - Low confidence on comparative speed; Medium on scaffolding:** there is no defensible published number for how long researchers will tolerate configuring each question in a five- to ten-question survey. Nor does the evidence establish that a blank brief, multiple signal fields or a hybrid is fastest for this audience.

### Case Study: Novices Can Edit Prompts Without Learning A Reliable Mental Model

In *Why Johnny Can't Prompt*, ten non-expert prompt designers make opportunistic local improvements but struggle to evaluate them systematically. They overgeneralise from individual successes and failures and rely on human-to-human instruction assumptions. This is relevant to A's "brief the moderator like a colleague" metaphor: it may feel approachable without producing reliable configuration. [28]

The implication is to scaffold the **research decision**, not teach prompt engineering through extra fields. A 2026 PromptFlow study offers encouraging evidence for structured scaffolding, but includes only six students, uses writing tasks and does not formally evaluate generated-output quality. It cannot establish faster or better survey setup. [14]

Researchers' natural wording should be investigated rather than presumed. "What do you want to learn?" is an accessible candidate, but no terminology study located shows that it matches how enterprise researchers naturally formulate per-question objectives. Conveo's documentation instead frames study setup around the decisions research should inform, illustrating a useful alternative emphasis. [18]

### Recommended Card: Exact Structure And Copy

**The following is proposed interface copy, not an observed winning design. Confidence: Medium overall; Low for comparative completion speed.**

| Element | Label | Hint or supporting text | Placeholder | Visibility and behaviour |
|---|---|---|---|---|
| Compact type selector | **Question type** | None unless the selected type needs explanation | None | Visible in the card header; do not make it a full configuration section |
| Participant wording | **Question for participants** | "This is the question participants will see" | None | Visible; required; live **0/120** counter; validate the supplied panel limit |
| Short researcher brief | **What do you want to learn?** | "Tell the AI what you need to understand beyond the question" | None | Visible, optional short textarea; if empty, explicitly use question-only clarification behaviour |
| Follow-up summary | **Follow-up questions** | "Ask only when more detail would help" | Not applicable | Visible summary: **"Use survey default: Up to 1"**; **Change** reveals the options |
| Stimulus attachment | **Add image, video or link** | "Shown with this question" | None | Visible action; expand attachment settings only after use |
| Optional completion criteria | **What would make this answer useful?** | "Add details the AI should check for, without suggesting an answer" | None | Under **More guidance**; add removable criteria, not mandatory keyword chips |
| Optional exclusions | **Anything to avoid?** | "Topics or assumptions the AI should leave alone" | None | Under **More guidance** |
| AI assistance | **Suggest guidance** | After generation: "AI suggestion - review before using" | Not applicable | Explicit action; do not silently overwrite existing wording |
| Behaviour preview | **Preview conversation** | "Example only. Actual follow-up questions may vary" | Not applicable | Visible action; opens a test using the current settings |

**Example text displayed below the goal field, not inside it:** "Understand what makes this concept feel useful or unnecessary. Ask for a specific example if their answer is unclear."

The deliberate placeholder choice is **none**. GOV.UK guidance requires labels, advises against placeholder text and recommends short hints; longer answers belong in a textarea. This is established form guidance, not proof of reduced setup time in Voxpopme. [41]

**Release implication:** keep the primary path to participant wording, a short optional goal and an inherited follow-up setting. Test setup time for complete five- and ten-question studies, including review and preview, rather than timing isolated fields.

## 3. Follow-Up Control: Use Discrete Limits, Not Unexplained Effort

**Recommendation - Medium confidence:** choose a radio group or equivalent accessible discrete selector. Do not ship a continuous slider whose positions cannot be explained in observable moderator behaviour.

**Exact control copy:**

**Follow-up questions**

- **Off** - "Ask only the main question"
- **Up to 1** - "Ask one follow-up if more detail would help"
- **Up to 2** - "Allow a second follow-up if useful detail is still missing"

Persistent supporting text: **"The AI may ask fewer. More follow-ups can increase completion time and participant cost."**

A review of smartphone questionnaire design recommends simpler selection formats over sliders and emphasises predictability across devices. That is useful adjacent evidence, but it concerns respondent questionnaires, not enterprise AI configuration. It does not prove the proposed radios outperform A's slider. [34]

### Counts And Depth Are Different, But Need Not Be Separate User Controls

**Analysis, not a tested finding:** count is an interaction budget; depth is the kind of information sought. One specific example can be more revealing than several repetitive requests for reasons. A single "effort" value only helps if the underlying policy is stable and explainable.

For the first release, expose **the budget** and keep a consistent moderator policy: identify the most valuable missing detail, ask one neutral question, then reassess. Put subject-specific priorities in the goal or optional guidance. Do not retain a second Light/Deep control unless Voxpopme can demonstrate a distinct, useful effect that researchers understand.

### Case Study: AI Effort Settings Do Not Transfer Directly To Interview Effort

OpenAI describes reasoning effort as controlling how much the model thinks, with lower effort favouring speed and lower token usage. Temperature is described separately as randomness, with high values potentially producing erratic outputs. These are technical controls over model generation, not validated measures of participant burden or interview depth. [40][33]

Consequently, familiar labels such as "effort" or "creativity" are not evidence that non-experts understand them. No relevant comprehension study located establishes that researchers can predict probe behaviour from these labels. **Confidence: High for the technical distinction; Low for user comprehension.**

**Number policy:** show an enforceable maximum, not **"About 2 follow-ups"** unless an estimate has been calibrated. A maximum can remain accurate when the AI decides not to probe. Separately show measured study-duration estimates and their uncertainty. This avoids presenting a dynamic decision as a promise that two follow-ups will occur.

## 4. Defaults And Overrides: One Conditional Follow-Up Is A Starting Hypothesis

**Recommendation - Medium confidence as a conservative product decision; Low as an empirically optimal count:** start open questions at **Up to 1**, not a compulsory one. Allow **Up to 2** as an explicit override. Apply a hard cap regardless of how expansive the written instructions become.

This recommendation combines the supplied internal finding that two probes felt long with documented market patterns. Listen Labs offers None, short-answer-only, one and two-to-three follow-ups. Conveo offers ranges, including zero-to-one and one-to-two. Neither source establishes the best shipped default, and neither is a trial demonstrating that two is better than one. [8][18]

### Case Study: Defaults Influence Behaviour, But Not By A Universal Percentage

The default-effects meta-analysis includes **58 datasets from 55 studies and 73,675 participants**. It finds an average effect of **d = 0.68**, while also reporting variation across contexts and mechanisms. Endorsement and endowment significantly moderate effects; ease does not in that analysis. [29]

This supports treating the default as consequential. It does **not** justify saying that a specific percentage of researchers will leave a one-probe setting unchanged. The practical implication is to choose a safe starting policy, measure real override behaviour and avoid treating untouched settings as informed preference.

### Survey-Level Behaviour

Use a survey-level default when most questions share the same interaction budget. It reduces repeated decisions in this proposed workflow; that benefit still needs usability testing. Per-question-only configuration is preferable when each item genuinely requires different treatment, not merely because the underlying agent exposes many parameters.

Recommended behaviour:

- Show **"Use survey default: Up to 1"** on inherited open-question cards.
- Show **"Custom: Up to 2"** on overrides, with **Reset to survey default**.
- A survey-level change updates inherited cards only; preserve explicit overrides and explain the impact before applying it.
- Include an always-visible survey summary of main questions, follow-up-enabled questions and the maximum additional interactions.

### Rating Questions Should Not Automatically Behave Like Open Questions

Start pure rating or classification items **Off**. Where the research purpose requires the reason, explicitly enable one follow-up such as **"What most influenced that rating?"** Do not assume all ratings need qualitative elaboration.

A chatbot-versus-form experiment finds no significant difference in trailer-purchase ratings, despite richer open-ended answers. It does not isolate the benefit of probing a rating, so it cannot settle this policy. **Confidence: Low for the optimal rating default.** [15]

**Decision:** treat one conditional probe as a testable default, not a research-method standard. Measure ratings with and without neutral follow-ups, and predefine which respondents receive them.

## 5. Progressive Disclosure, AI Suggestions And Conversation Preview

**Finding - High confidence that these patterns exist; Medium that they fit this product:** competitors combine AI assistance with editable guides and previews. Listen Labs generates editable study goals and guides; Strella generates guides from research goals; Engage generates and improves follow-up instructions. [8][5][32]

### Case Study: Helpful Suggestions Can Also Produce Overreliance

The cognitive-forcing experiment compares three intervention designs with explainable-AI approaches and a no-AI baseline, with **199 participants**. Cognitive forcing reduces overreliance, but the designs that reduce it most receive the least favourable subjective ratings. Explanations alone do not reliably solve the problem. [24]

This is not a study of pre-filled research objectives. Its relevant mechanism is that effortless acceptance can conflict with deliberate judgement, while heavy review requirements can damage usability. **Confidence: Medium for transfer; Low for the exact proposed interaction.**

**Recommended suggestion pattern:** generate only on request; mark suggestions; let researchers accept, edit or discard them; preserve the original when regenerating; and distinguish accepted from unreviewed guidance. Ask researchers to review the study's goals and follow-up behaviour before launch, rather than forcing an acknowledgement for every generated phrase.

### Case Study: Engage Makes Agent Behaviour Inspectable

Engage's documented question test offers **Manual**, where the researcher experiences the question and follow-ups, and **Simulated**, where an automated response is shown. Listen Labs also supports individual-question testing and running the interview from the start. [36][8]

These examples support preview as a practical configuration pattern, not proof that simulations predict respondent behaviour. Conveo explicitly warns that AI quality review does not establish validity and that test paths cannot guarantee all live recruitment and routing outcomes. [18][12]

**Voxpopme implementation:** use the actual current moderation policy in preview, show which instruction influenced the follow-up, and offer short, complete, contradictory and off-topic test answers. Label synthetic answers clearly. One fluent example is insufficient: the novice-prompting evidence shows the danger of overgeneralising from a single successful interaction. [28]

Hide advanced guidance, not the effective setting. A collapsed card must still show whether follow-ups are enabled, their cap and whether the value is inherited or custom.

## 6. Naming: Use "Follow-Up Questions" And Explain The Behaviour

**Recommendation - Medium confidence for plain-language suitability; Low for measured comprehension:** use **follow-up questions** as the primary term. Keep **probe** in expert documentation or methodology support, not as the only visible label.

### What Competitors' Labels Reveal

Listen Labs uses "Follow-up Control" and "Other Instructions". Strella uses "Dynamic follow-ups" and "Follow-up instructions for moderator". Qualtrics uses "Adaptive follow-up questions" and "Response clarity validation". Conveo uses probing levels, while Engage uses "Follow-Up Instructions". [8][5][7][18][32]

These labels establish market vocabulary, not which term users understand best. They also describe different behaviours: checking an unclear answer is narrower than exploring a new dimension. Voxpopme should explain the intended behaviour rather than rely on a familiar label to conceal ambiguity.

| Term | Recommended use | Risk or limitation |
|---|---|---|
| Follow-up question | Primary setup and respondent label | Still needs a sentence explaining when it appears |
| Probe | Optional expert terminology | Familiarity among non-experts is untested |
| Clarifying question | Use when resolving ambiguity | Too narrow for examples, trade-offs or missing context |
| Dig deeper | Informal explanatory copy only | Does not define count, stopping or informational intent |
| Signal | Avoid as the main completion label | May suggest keywords or expected answers rather than sufficient evidence |

This is a proposed vocabulary hierarchy, not the result of a terminology experiment. Use **follow-up question** consistently across setup, preview and respondent screens.

Recommended microcopy:

- **Purpose:** "The AI uses your guidance to decide whether to ask for more detail."
- **Stopping:** "It stops when the answer is complete or the follow-up limit is reached."
- **Privacy of guidance:** "Participants will not see this guidance."
- **Optional criteria:** "Describe useful information, not the answer you expect."

These phrases avoid region-specific idioms and are intended to read naturally in US and UK English. That is an editorial judgement, not a validated cross-market finding. Test interpretation with researchers in both markets, particularly whether **Up to 1** means optional rather than guaranteed.

**Decision:** test terminology through behaviour prediction, not word preference alone. Ask users to explain what the moderator will do with a complete answer, an unexpected answer and a very short answer.

## 7. Competitor Setup Flows: A Market Moving Towards Guided Configuration

The clearest observed trend is the combination of study-level intent, AI-assisted guide creation and question-level moderation controls. Listen Labs, Conveo and Strella expose meaningful editing rather than a completely opaque agent; Outset emphasises configurable methodology and deeper probing. These are documented capabilities, not independently demonstrated quality rankings. [8][18][5][9]

**Table conventions:** all findings below were checked **19 October 2026**. Most help pages do not state a publication date. **Not verified** means the public evidence examined does not establish the feature or initial setting; it does not mean the feature is absent. Confidence is **High for explicitly documented controls**, but lower for marketing descriptions and not established for comparative performance.

| Tool | Question fields | Objective level | Follow-up controls exposed | Defaults | AI pre-fill | Preview/test mode | Modes supported | Source | Date checked |
|---|---|---|---|---|---|---|---|---|---|
| **Outset** | Qualitative/quantitative questions, advanced probing guidelines and skipping logic; exact card fields not verified | Research goals feed guide creation; separate per-question objective not verified | Discussion guide, probing depth, moderator style, branching; Abyss mode advertises up to 10 follow-ups/question | Initial count/depth not verified; 10 is a capability ceiling, not a default | AI-generated guide from goals; dynamic probes | Not verified in sources examined | Vendor lists video, voice and text; exact asynchronous/live mode matrix not verified | Product, homepage and FAQ. [4][9][35] | 19 Oct 2026 |
| **Listen Labs** | Type, question blocks, follow-up count, **Other Instructions**; stimulus, URL and conditional logic options | Editable study goals; per-question probing instructions | None; on short answers; 1; 2-3 follow-ups | Selected default not verified | Study goals, guide, recruitment and screener drafting; chat editing | Respondent-view individual-question testing, Run from start, Open Preview | Company description explicitly lists text and voice; guide documents a video-interview preview; complete mode/asynchrony matrix not verified | Study Design 101; company description. [8][46] | 19 Oct 2026 |
| **Conveo** | Type, wording, choices, probing depth, follow-up context, stimuli and routing | Research briefing/business context and objectives; sections organise questions | Automatic, none, 0-1, 1-2, 2+; documented budgets extend to 5 | Initial selected option not verified; Automatic allows 0-5, not necessarily 5 | Assistant proposes guide changes for review; separate objective/probe pre-fill fields not fully verified | Full tests; start at a question; pin conditions and inspect paths | Spoken open-ended responses, typed open-text format, optional/required/off camera recording; complete live/asynchronous matrix not verified | Topic guide, settings, testing docs. [18][3][12] | 19 Oct 2026 |
| **Qualtrics Conversational Feedback** | Text-entry question; adaptive feature; maximum where available; company context/topics to avoid; clarity criteria and custom messages | Question-level response criteria/context; separate research-objective field not verified | Adaptive follow-ups; advanced English supports up to 2; response clarity validation; topic avoidance; page/survey restrictions | Initial UI selection not verified. Basic/non-English capacity of 1 must not be mistaken for a proven optimal default | AI-generated adaptive follow-ups; AI guide/objective pre-fill not verified for this feature | Preview response documented | Open-text feedback verified; voice/video support for this specific follow-up feature not verified | Support documentation. [7] | 19 Oct 2026 |
| **Strella** | Question text, **Dynamic follow-ups** toggle, depth/number, **Follow-up instructions for moderator**, guide/media elements | Study research objectives; per-question follow-up instructions | On/off, follow-up depth/number and moderator instructions | Some AI-generated questions have follow-ups automatically enabled; exact count/depth default not verified | AI guide from research goals; separate objective/signal pre-fill not verified | Full respondent preview before publishing | Voice-to-voice; optional video/screen recording described; text-only and exact asynchrony matrix not verified | Guide tutorial, preview tutorial, product explanation. [5][2][39] | 19 Oct 2026 |
| **Glaut** | Product lists matrix, NPS, numeric, blocks, stimulus and asset collection; exact per-question configuration fields not verified | Project context/goals create a brief; per-question objective field not verified | Dynamic personalised probing; exposed count/depth controls not verified | Not verified | Project Builder assists brief creation; separate guide/objective/probe pre-fill not fully verified | Not verified | Voice and typing verified; video and exact asynchronous/live mode matrix not verified | Product documentation. [6] | 19 Oct 2026 |
| **CloudResearch Engage** | Question-level **Follow-Up Instructions** toggle and instruction text; exact complete card inventory not verified | Question-level follow-up guidance; separate study-objective field not verified | Written instructions can specify cap, sufficient-answer stopping and one-question-at-a-time behaviour; enforced numeric UI cap not verified | Not verified; instructional example of a 3-question maximum is not a shipped default | Generate/Improve follow-up instructions | Manual and Simulated question test drives | Complete text/voice/video capability matrix not verified in the detailed sources examined | Follow-up and test-drive help articles; AI assistant article updated 28 May 2026, test-drive article 27 Nov 2024. [32][42][36] | 19 Oct 2026 |
| **Knit** | AI questionnaire includes quant, qualitative video and AI-moderated video questions; exact per-question fields not verified | Study Research Brief includes audience, objectives, hypotheses and deliverables | Specific exposed count/depth controls not verified | Not verified | Research brief/questionnaire drafting, with dedicated researcher collaboration | Not verified | Qualitative video and AI-moderated video documented; complete voice/text/asynchrony matrix not verified | Research workflow description, previously verified 28 Sep 2026. [43] | 19 Oct 2026, supplied verification retained |

**Takeaway:** the market supports both instructions and bounded controls, but there is no verified common default. A count capability, instruction example and preselected setting are three different things.

### Case Study: Conveo Makes Depth Concrete Without Promising Duration

Conveo documents follow-up ranges and separately states that these control follow-up depth, not guaranteed interview duration. Its guide also distinguishes researcher-facing business context from participant welcome content. [18]

This is close to Voxpopme's core problem: researchers need an understandable contract, while respondents need concise wording. The opportunity is to make the contract simpler than Conveo's broad choice set, not to copy every exposed parameter. For Voxpopme's asynchronous survey, a narrower budget can differentiate it from a longer live AI interview.

**Important Qualtrics qualification:** its documentation includes an advanced-follow-up limit at survey level and allows only one follow-up per page, with a page break for a second. Its maxima should not be read as unrestricted per-question budgets across an entire survey. [7]

## 8. Competitor Reception: Useful Warnings, Not A Reliable League Table

**Finding - Medium confidence that the issues occur; Low on prevalence:** user reviews and practitioner accounts provide plausible failure examples, but the evidence located is not sufficiently balanced or representative to rank the required vendors on setup simplicity or probe quality.

### Case Study: Conveo Reviews Show The Same Feature Can Help And Frustrate

The extracted Conveo G2 page contains **13 reviews**, including authored reviews dated August 2026. Positive comments describe relevant real-time follow-ups, centralised interview work and asynchronous research at scale. Negative comments include missed context, limited customisation, learning curves and unnatural probing. [31]

These reports support testing the failure modes; they do not establish their incidence, nor prove that one setup format causes them. They also reveal an important tension: reducing configuration work can increase dependence on the moderator correctly interpreting implicit intent. Separate authored review comments from G2's generated summary when interpreting this evidence. [31]

### What Recurs Across Different Evidence Types

| Issue | Evidence located | Interpretation and confidence |
|---|---|---|
| Repetitive follow-ups | Experimental qualitative feedback and higher perceived repetition for explanatory probes. [20] | Medium: demonstrated in one study context, not a vendor-wide complaint rate |
| Missing context or unnatural probing | Authored Conveo reviews. [31] | Medium for occurrence; Low for prevalence |
| Under-probing rather than over-probing | Small human/AI comparison finds AI failures to ask required follow-ups; Strella also acknowledges under-probing as a limitation. [21][39] | Medium: a restrictive default also needs quality checks |
| Too many follow-ups | Supplied Voxpopme internal testing; no robust cross-vendor frequency estimate located | Low externally; important internal release evidence |
| Robotic tone | Some contextual waiting research finds very fast replies implausibly robotic. [19] | Low transfer: not proof of a recurring complaint about these vendors |

The lesson is to assess **relevance, novelty, neutrality and appropriate stopping** together. Optimising only the average number of probes could trade over-probing for under-probing.

Vendor testimonials and vendor participant studies should remain labelled as such. Strella's own explanation references participant feedback while acknowledging wrong-thread probing and missed cultural cues; that is useful product candour, not independent comparative validation. [39]

Carl Pearson's 2026 critique adds a different caution: large-scale qualitative category generation should not be casually treated as quantitative prevalence measurement. This is methodological opinion, not a controlled finding about A or B. **Confidence: Medium as a relevant caution, not empirical proof.** [26]

## 9. Respondent Experience: Seek Useful New Information, Not Longer Answers

**Finding - Medium confidence:** conversational surveys can improve open-ended responses, but richer data often comes with longer interaction. There is no established probe count at which asynchronous respondents universally fatigue, shorten answers or abandon.

### Case Study: Juji Versus A Conventional Survey

The 2020 field study receives **582 completed surveys**, comparing a conversational chatbot with a conventional Qualtrics survey among gamers evaluating trailers. Chatbot responses are more relevant, specific and clear; participants also spend more time and write longer answers. [15]

This is valuable counter-evidence to the assumption that every extra interaction harms the experience. But the study cannot isolate probing from the other conversational features, and its gaming sample and recruitment conditions limit generalisation. It therefore supports **conditional probing**, not a rule to add two probes to every question. [15]

### Case Study: Voice Looks Richer, But Selection Matters

Glaut's August 2026 synthesis of a Human Highway study reports voice-only answers averaging **58.65 words**, versus **24.94** for AI text-only answers. Participants choose their mode, however, so motivation, fluency and context may account for part of the difference. This is a vendor synthesis of an external study, not a randomised demonstration that voice causes better answers. [37]

The same synthesis cautions that shorter text is not necessarily less considered and reports no significant difference in the quoted text-versus-voice experience comparison. There is no equivalent controlled video/voice/text probe-budget comparison established here. **Confidence: Low for causal mode differences.** [37]

### Recommended Respondent Policy

- Start at **zero or one conditional follow-up** for open questions; use two only where the learning value justifies the extra interaction.
- Ask about the highest-value missing detail, not every possible signal.
- Stop after a complete answer, a clear refusal, or the enforced limit.
- Keep the same quality and stopping policy across modes initially, then calibrate budgets separately from observed time and burden.
- Assess relevance, specificity and unique information alongside duration, skips, drop-off and respondent-rated burden. Word count alone is insufficient.

A 2025 German web-probing experiment with **2,184 participants** finds that probe placement and response format affect aspects of effort and answer quality. It uses one probe per question and explicitly notes that multiple probes may impose different burdens. This is cognitive question-pretesting evidence, not a direct LLM-moderation count trial. [16]

**Ratings:** there is insufficient isolated evidence to say a rating benefits as much as an open question. Test neutral reason-follow-ups explicitly rather than borrowing findings about richer open-ended responses.

## 10. Progress Displays: Count Main Questions Honestly, Nest Follow-Ups Visibly

**Recommendation - Medium confidence:** use stable main-question progress, with follow-ups visibly nested beneath their parent. Do not silently exclude follow-up work from the introduction or imply that "five questions" means five respondent actions.

### Case Study: A Progress Bar Can Increase Drop-Off

The progress-indicator meta-analysis combines **32 randomised experiments**. Constant indicators do not significantly reduce dropout overall; fast-to-slow indicators reduce dropout, whereas slow-to-fast indicators increase it. Constant indicators also increase dropout in the subset of studies promising a small incentive. [25]

This challenges the assumption that any progress bar is reassuring. It does not justify deliberately manipulating progress to move quickly at the start. For an adaptive survey, a changing denominator or repeated apparent stalls could create a similar expectation problem, but that specific transfer remains an inference. **Confidence: High for the older meta-analysis; Medium for this design implication.**

### Recommended Pattern

Display **"Main question 2 of 5"** while the participant answers that question and any associated follow-up. Label the follow-up itself **"Follow-up question"**, or **"Follow-up 1 of up to 2"** when two are possible. Advance the main-question counter once the whole question block is complete.

Keep the submitted main answer visible or easy to revisit. Do not reset or reverse progress when the agent decides to probe. Avoid an exact remaining-question count unless the total is genuinely fixed. A progress indicator should describe the work represented by its label; nesting must not become a way to hide work.

**Competitors:** the documented previews allow researchers to inspect the participant journey, but the sources examined do not verify a consistent competitor convention for counting dynamic follow-ups. Do not treat respondent screenshots or marketing animations as evidence of a platform-wide progress policy. [2][12]

### Exact Upfront Copy

For a survey with a uniform one-follow-up cap:

> "About {measured time range} minutes, including follow-up questions. There are 5 main questions. An AI may ask up to 1 short follow-up after each. You can skip follow-up questions."

For mixed per-question limits:

> "About {measured time range} minutes, including follow-up questions. There are 5 main questions. An AI may ask short follow-up questions depending on your answers. You can skip follow-up questions."

Publish the skip promise only if the product and incentive policy support it. Populate the range from real pilot completions, including generation waits, recording/upload time and follow-ups; do not invent a numerical range before measurement.

**Decision:** test comprehension of total work and perceived honesty, not just whether participants like a progress bar.

## 11. Pacing, Waiting And Probe Wording

**Finding - Low confidence on an abandonment threshold:** no reliable universal number establishes when respondents abandon while waiting for an AI-generated probe. Evidence about perceived waits must not be relabelled as abandonment evidence.

### Case Study: Eight Seconds Is Not A Universal Service-Level Target

A 2022 experiment with **478 US and UK participants** tests response delays of **2, 4, 8 and 16 seconds** in simulated crowd-powered conversational interactions. Sixteen seconds produces more negative feelings; very fast responses can feel implausibly robotic. The study measures waiting perceptions and interaction quality, not an abandonment threshold. [19]

Its simulated interactions differ from Voxpopme's asynchronous video, voice and text workflow. The useful lesson is that response quality and context affect the wait experience, not that Voxpopme should artificially delay replies or accept eight-second generation as safe. **Confidence: Medium for the contextual finding; Low for numerical transfer.** [19]

### Proposed Waiting Behaviour

Acknowledge submission immediately and distinguish processing from the participant's next task. Use **"Reviewing your answer..."** while generation is actually running. If generation fails, show a truthful retry or continue option; preserve the submitted recording or text. Do not use a fabricated typing animation as evidence that a human is present.

Measure end-to-end latency by mode and device, including upload and transcription, with median and tail timings. Instrument retries, exits during generation and exits immediately after a probe. Treat any proposed time target as an engineering hypothesis until linked to observed respondent outcomes.

### Probe Wording

| Situation | Proposed wording | Avoid |
|---|---|---|
| A vague benefit | "What would that help you do?" | "Would this make your life easier?" |
| An unexplained reaction | "What most influenced that reaction?" | Repeating "Why?" after an explanation already exists |
| A broad claim | "Can you describe a specific time that happened?" | Asking for several examples, reasons and comparisons at once |
| A rating | "What most influenced that rating?" | Assuming a low rating means dissatisfaction with a particular feature |

These are proposed neutral patterns, not universally validated scripts. Use one short question at a time; an acknowledgement should not endorse the participant's view. Avoid repetitive praise or a long paraphrase before every probe. The probe-strategy experiment supports varying informational purpose rather than repeatedly requesting explanations. [20]

### Disclose The AI, Without Claiming It Always Improves Answers

A review of **26 articles**, published online in 2024, finds mixed self-disclosure results. Some studies favour disclosure to conversational AI, particularly with reduced judgement on sensitive topics; others find no significant differences. The review does not establish one general causal effect across research settings. [38]

Recommended introduction: **"An AI will read your answers and may ask follow-up questions. A research team will review the responses."** This is a transparency recommendation, not a promise of more candid or more accurate data. Explain recording and data use separately.

## 12. Constraints, Costs, Accessibility And Failure Handling

### Respect The 120-Character Boundary Without Compressing The Research Brief

The **120-character cap is supplied product context**. Apply it to participant question wording, not automatically to private moderator guidance. Confirm whether the same panel restriction also applies to generated written probes before shipping; the brief does not settle that point.

Show the counter before the limit is reached, do not silently truncate, and use an error such as **"Shorten this question to 120 characters or fewer."** Keep essential context visible to participants: moving necessary definitions into a private objective does not solve a short question. Review translated questions against the actual panel's counting rules rather than assuming English wording will fit every language.

### Case Study: Incentives Follow Time, But Not Necessarily Per-Probe Billing

Prolific recommends at least **GBP 9 / USD 12 per hour**, with a minimum of **GBP 6 / USD 8**. Its base reward is fixed per participant; longer actual completion does not automatically change that reward, but underpaying studies may require adjustment. The article is dated July 2026. [30]

This clarifies an important distinction for Voxpopme: more follow-ups can increase the time needed to maintain fair compensation, without the platform literally billing per probe. Other vendors' complete incentive-pricing algorithms were not verified here. Show expected participant time and incentive impact, not an unsupported exact cost per follow-up.

**Illustrative calculation, not a Voxpopme quote:** if an added interaction takes one minute for 100 participants, maintaining GBP 9/hour adds **GBP 15** in participant rewards; at USD 12/hour it adds **USD 20**. This excludes panel/platform fees and assumes the extra minute occurs for every participant.

For planning, estimate total time as base-question time plus the expected number of follow-ups multiplied by their generation-and-response time. Estimate each mode separately; an interaction count alone is not a duration estimate.

### Recommended Failure Contract

**These are proposed safeguards, not a claim of an industry-wide standard. Confidence: Medium.**

| Edge case | Proposed behaviour | What to expose to researchers |
|---|---|---|
| Missing goal | Use question-only clarification; do not invent a substantive research objective | "No additional guidance - the AI will only clarify this question" |
| Vague goal | Offer a more specific suggestion or example; retain explicit researcher choice | Non-blocking guidance-quality warning |
| Contradictory instructions | Flag the conflict before launch; keep the budget and safety policy authoritative | Conflict explanation and editable instructions |
| Short but complete answer | Accept it; brevity alone is not evidence of poor quality | Preview example showing no follow-up |
| Short and incomplete answer | Ask one specific missing-detail question within the budget | Reason for the follow-up |
| Off-topic or low-effort answer | Allow one neutral redirect when useful; then stop or flag under the study's quality policy | Separate quality flag from automatic rejection |
| Refusal, uncertainty or inappropriate content | Respect refusal; do not pursue sensitive material simply to satisfy a signal | Stop/skip outcome and a reviewable flag |
| Agent drift or respondent instructions | Keep researcher policy authoritative; treat respondent content as data, not configuration | Logs and a reportable issue |
| Cap reached or generation failure | Stop, preserve the original answer and continue or offer retry | Cap/recovery outcome, not a hidden extra question |

Bounded intervention is observable in existing tools, but the bounds differ. Qualtrics' clarity validation can prompt once and then permit continuation; Engage's guidance illustrates sufficient-answer stopping and one question at a time. Neither establishes a universal two-probe standard. [7][42]

### Accessibility Requirements

Use **WCAG 2.2** as the baseline, not the developing WCAG 3 draft. The standard covers text alternatives, captions for relevant prerecorded media, programmatically available status messages and keyboard accessibility. [27]

For this product, provide readable written follow-ups in every mode, accessible playback/recording controls, keyboard navigation, announced generation status and a way to recover from recognition mistakes. Caption relevant stimulus media; distinguish that obligation from assuming every participant-created recording is itself published stimulus content. Avoid moving focus unexpectedly when a probe arrives. Review response timeouts and recording limits for accessibility, and offer text or other accommodations where the research objective permits.

**Decision:** the release contract must cover error handling and alternative interaction, not merely the ideal AI-generated conversation.

## 13. What To Hide Or Remove From The Current Setup

**Recommendation - Medium confidence:** remove duplicate behavioural controls from the primary path, but retain effective settings in the summary. The competing examples show that guidance and count controls can coexist; the supplied internal problem is that their meanings currently overlap, not that all expert control is harmful. [8][5]

| Current element | Proposed treatment | Reason |
|---|---|---|
| Question type | Keep as a compact header control | Researchers still need to choose rating versus open formats |
| Stimulus | Keep an **Add image, video or link** action | Important research content should not compete with empty settings |
| Question | Keep and rename **Question for participants** | Makes the participant-facing 120-character field unambiguous |
| Objective | Rename **What do you want to learn?**; keep short and optional | Communicates intent without making researchers write a system prompt |
| Listen for signals | Move to **More guidance** and reframe as useful-answer criteria | Avoids mandatory expected-word checklists |
| Probe hints | Merge into the goal or optional guidance | Removes a second near-duplicate place to brief the same behaviour |
| Max probes | Replace with visible, bounded **Follow-up questions** options | Directly communicates the interaction budget |
| Probe depth, Light/Deep | Remove from the primary card; do not merely rename it | Its distinct effect is not currently explainable in supplied internal testing |
| Survey-wide repeated configuration | Replace with inheritance and visible overrides | Reduces repeated decisions while preserving exceptions |

### Case Application: The Supplied Five-Question Survey

In the internal example, five advertised questions become roughly twelve interactions. Changing labels alone would leave the same underlying burden. The redesigned card should therefore expose the maximum budget and the study summary before the researcher publishes.

For example, five open questions with **Up to 1** each allow at most five follow-ups, or ten question-and-follow-up interactions in total. That is arithmetic under the proposed cap, not a measured average. A mixed survey with rating items set Off would have a different maximum. The respondent introduction should reflect the actual configuration, not a fixed product slogan.

**Decision:** hide optional detail, remove redundant concepts and retain accountability. Do not use progressive disclosure to conceal duration, cost or the effective follow-up policy.

## 14. Evidence Register And The Tests Voxpopme Still Needs

### Key Findings And Confidence

| Key finding | Source | Source type | Date | Confidence and boundary |
|---|---|---|---|---|
| Goals, context and explicit requirements are complementary prompting inputs | OpenAI, Anthropic, Google. [11][10][13] | Vendor technical guidance | Live docs checked Oct 2026; Google updated Apr 2026 | High for guidance; Low for A/B card superiority |
| Non-experts struggle to evaluate prompt robustness | *Why Johnny Can't Prompt*. [28] | Academic, qualitative study | 2023 | Medium transfer to research setup |
| Structured scaffolding can improve perceived understanding | PromptFlow. [14] | Academic, small study | Apr 2026 | Low for speed or output-quality improvement |
| Probe strategy changes relevance, specificity and repetition | Four-probe comparison. [20] | Academic experiment | 2025 | Medium; one topic and limited population |
| AI interviewing can miss required follow-ups | Wuttke et al. [21] | Academic, small controlled study | 2025 | Medium for occurrence; Low for prevalence |
| Conversational surveys can enrich open-ended answers | Xiao et al. [15] | Academic field study | 2020 | Medium; bundled treatment and sample limits |
| Probe placement affects effort and response quality | Hadler. [16] | Academic experiment | 2025 | Medium; cognitive web probing, not LLM count trial |
| Defaults influence choices, with contextual variation | Default-effects meta-analysis. [29] | Academic meta-analysis | 2019 | High for general effect; Low for override-rate prediction |
| More progress display is not always better | Villar et al. [25] | Academic meta-analysis | 2013 | High within analysed surveys; Medium adaptive-survey transfer |
| Explanations do not eliminate AI overreliance | Cognitive-forcing study. [24] | Academic experiment | 2021 | Medium transfer to AI pre-fill |
| Waiting evidence does not establish an abandonment threshold | Abbas et al. [19] | Academic experiment | 2022 | High on study scope; Low numerical transfer |
| AI identity and disclosure have mixed effects | Papneja and Yadav review. [38] | Academic literature review | Online 2024; volume 2025 | Medium; no universal disclosure effect |
| Voice/text differences may reflect participant selection | Glaut synthesis of Human Highway. [37] | Vendor synthesis of external study | Aug 2026 | Low for causal mode comparison |
| Relevant and unnatural probes can both occur in user experience | Conveo authored G2 reviews. [31] | Review site | Aug 2026 reviews | Medium for examples; Low for prevalence |
| Editable guidance and previews are market patterns | Listen, Conveo, Strella, Engage documentation. [8][18][2][36] | Vendor documentation | Checked Oct 2026 | High for documented features, not effectiveness |
| More time may require higher fair compensation | Prolific payment model. [30] | Vendor policy | Jul 2026 | High for Prolific; Medium transfer to other panels |
| Accessibility includes alternatives and status announcements | WCAG 2.2. [27] | Technical standard | Dec 2024 recommendation | High for requirements; product-specific application needed |

The older sources address relatively stable interaction and survey mechanisms. They should inform hypotheses, not override recent product-specific evidence or be represented as tests of current LLM moderators.

### Evidence Gaps And Proposed Tests

**All sample sizes and test designs below are proposals, not published findings or power calculations.**

| Gap | Proposed Voxpopme test | Primary decision metrics |
|---|---|---|
| Free-text, signals or hybrid output quality | Hold the model, questions, answer transcripts and probe budget constant; compare all three instruction formats with blinded researcher scoring | Relevance, new information, neutrality, missed necessary probes, unnecessary probes, stopping accuracy |
| Setup effort and natural objective wording | Counterbalanced usability study with approximately 12-18 novice and experienced researchers; collect their wording before showing A/B/hybrid | Total five-/ten-question setup time, corrections, comprehension, perceived effort |
| One versus two probes | Randomise respondents to no adaptive probe, up to one, or up to two on matched items | Useful information per completed participant and per minute; burden, skips, completion, incentive cost |
| Slider versus bounded choices | Ask users to predict behaviour for short, complete and unexpected answers; then configure matched tasks | Prediction accuracy, cap comprehension, selection time, confidence calibration |
| Global defaults and overrides | Compare inherited defaults with per-question-only setup | Missed settings, unwanted mass changes, override rate, launch errors |
| AI pre-fill acceptance | Seed some plausible but incorrect or leading suggestions in a safe usability environment | Detection, correction, review time, inappropriate acceptance |
| Preview usefulness | Compare no preview, a single example and interactive edge-case preview | Ability to predict failure, configuration changes and later probe quality |
| Rating follow-ups | Randomise a neutral reason-follow-up for comparable rating items | Incremental explanatory value, scale distributions, burden and nonresponse |
| Mode and latency | Compare matched tasks across permitted video/voice/text conditions; instrument actual pipeline delays | Tail latency, exits during waits, answer quality, upload/recognition failures, accessibility issues |
| Honest progress and time estimates | Compare main-question nesting with alternative truthful progress patterns | Expected versus experienced work, perceived honesty, drop-off and completion |
| Terminology across US/UK | Test "follow-up", "probe" and candidate level labels with behaviour-prediction tasks | Meaning, optionality and stopping comprehension, not merely preference |
| Multilingual and unusual answers | Evaluate translated guides and adversarial/contradictory answer cases with appropriate language expertise | Neutrality, translation fidelity, false completion, cap violations, inappropriate sensitive probing |

**Recommended release gates:** no cap violations; no silent question truncation; clear inherited/custom states; reviewed respondent wording; and acceptable probe quality under both complete and difficult answer cases. Numerical acceptance thresholds should be agreed before testing, informed by pilot variance and business risk rather than invented here.

## 15. Surprises And Counter-Evidence

**First, the risk is not only too much probing.** The small AI-versus-human study finds failures to ask required follow-ups, despite explicit moderation instructions. A low count can protect burden while still producing poor coverage. [21]

**Second, a friendlier conversation can take longer and still improve answers.** The Juji field study finds richer responses alongside more time spent. Voxpopme should not equate shorter sessions with better research, or longer sessions with deeper understanding. [15]

**Third, structured support is not automatically a quality improvement.** PromptFlow's positive perceptions come from a very small study without objective output-quality evaluation. Mandatory signals could add effort without establishing better probes. [14]

**Fourth, progress indicators can backfire.** The progress meta-analysis contradicts the common assumption that displaying progress reliably reduces dropout. Honesty about work matters more than the mere presence of a bar. [25]

**Fifth, accountability has a usability cost.** Cognitive forcing reduces overreliance but can be disliked. A confirmation step on every generated field may therefore undermine the simplification it is meant to support. That final design implication is an inference to test, not a demonstrated result for research tools. [24]

**Sixth, AI identity is not necessarily detrimental.** Some disclosure studies favour AI-framed interaction and others find no difference. There is no evidence-based reason here to conceal that an AI generates the follow-ups. [38]

**Decision:** optimise for an explicit trade-off between evidence gained, respondent burden and researcher accountability. Neither maximal automation nor minimal questioning is a sufficient design principle on its own.

## 16. Synthesis: A Clear Contract Beats More Agent Controls

The evidence supports separating three jobs: **what the participant is asked**, **what the researcher needs to understand**, and **how much additional interaction is allowed**. A and B each capture part of this structure, but neither proposed control fully explains the runtime contract.

| Design strategy | Mechanism | Scope and setup burden | Principal trade-off | Evidence and release posture |
|---|---|---|---|---|
| A unchanged | Rich free-text briefing plus effort slider | Flexible context; researcher must articulate several ideas in one field | Expressiveness versus ambiguous effort and buried stopping rules | No direct superiority evidence; change the control before testing |
| B unchanged | Goal plus explicit completion signals and named levels | Scaffolded coverage; potentially more per-question obligations | Precision versus leading/checklist behaviour and vague level semantics | No direct superiority evidence; make criteria optional and preserve participant wording |
| Progressive hybrid | Short goal, optional criteria, explicit cap, inspectable examples | Simple default path with expert detail on demand | Requires careful inheritance, review and fallback behaviour | Best-supported synthesis, Medium confidence; validate before scaling |

The competitor comparison exposes three useful contrasts. **Outset** markets substantial conversational depth, whereas **Qualtrics** structures bounded interventions around text feedback and specific survey/page restrictions. **Listen Labs**, **Strella** and **Conveo** expose guide and follow-up editing, but Conveo's detailed ranges illustrate how expert flexibility can reintroduce configuration complexity. [4][7][8][5][18]

**Engage** makes instructions and manual/simulated testing explicit; **Glaut** emphasises brief-led voice research and personalised probing; **Knit** adds researcher-led mixed-method workflow support. These differ in mechanism, interaction scope and where human effort sits. They should not be treated as interchangeable evidence for an asynchronous written-probe survey. [32][36][6][43]

The non-obvious tension is that **less visible configuration can require more disciplined evaluation**. Removing fields is beneficial only if the remaining goal, budget and stopping policy produce predictable behaviour. Preview and logging then become part of the research-design interface, not optional polish.

**Final recommendation:** prototype the hybrid card; use **Up to 1** as a conservative open-question default and **Off** for pure ratings; allow explicit overrides up to two; remove the unexplained depth slider; and disclose main questions, possible follow-ups and measured time honestly. Choose the final design on **setup comprehension and useful information per respondent-minute**, with neutrality and accessibility as constraints, not on preference ratings alone.

## References

### Model Guidance And Researcher Configuration

1. OpenAI. *Best practices for prompt engineering with the OpenAI API*. Live documentation checked October 2026. [11]
2. Anthropic. *Define success criteria and build evaluations*. Live documentation checked October 2026. [10]
3. Google. *Prompt design strategies*. Updated 28 April 2026. [13]
4. *Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts*. CHI, 2023. [28]
5. *PromptFlow: Supporting Novice Prompt Engineering Through Structured and Visual Scaffolding*. CHI Extended Abstracts, 2026. [14]
6. GOV.UK Design System. *Text input*. [41]
7. OpenAI. *Reasoning models* and API temperature FAQ. [40][33]

### Interviewing, Respondent Experience And Human-AI Interaction

8. *Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probes*. 2025. [20]
9. Wuttke et al. *AI Conversational Interviewing: Transforming Surveys with LLMs as Adaptive Interviewers*. 2025. [21]
10. Xiao et al. *Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions*. 2020. [15]
11. Hadler. *Response Burden and Response Quality in Web Probing*. Survey Research Methods, 2025. [16]
12. *When and why defaults influence decisions: a meta-analysis of default effects*. 2019. [29]
13. Villar, Callegaro and Yang. *Where am I? A meta-analysis of experiments on the effects of progress indicators for web surveys*. 2013. [25]
14. *To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making*. 2021. [24]
15. Abbas et al. *Understanding User Perceptions of Response Delays in Crowd-Powered Conversational Systems*. 2022. [19]
16. Papneja and Yadav. *Self-disclosure to conversational AI: a literature review, emergent framework, and directions for future research*. Online 2024; journal volume 2025. [38]
17. Antoun et al. *Design Heuristics for Effective Smartphone Questionnaires*. 2018. [34]

### Competitor Documentation, Reception And Operating Standards

18. Outset. Product interviews page, homepage and FAQ. [4][9][35]
19. Listen Labs. *Study Design 101* and company description. [8][46]
20. Conveo. *Setting Up Your Topic Guide*, *Detailed Topic Guide Settings* and *Testing Your Study*. [18][3][12]
21. Qualtrics. *Conversational Feedback*. [7]
22. Strella. *Setting Up Your Question Guide*, *Previewing and Publishing Your Guide* and *What Is an AI-Moderated Interview?* [5][2][39]
23. Glaut. Product documentation. [6]
24. CloudResearch Engage. *Using the AI Assistant for Follow-Up Instructions*, *Adding Follow Up Instructions* and *Test Drive Interview Questions*. [32][42][36]
25. Knit. Research workflow description, verified September 2026. [43]
26. G2. Authored Conveo reviews, including August 2026 entries. [31]
27. Pearson. *AI moderated interviews: methodological error amplified*. March 2026; opinion. [26]
28. Glaut. *Voice vs. Text AI Interviews: Which Produces Better Insights in 2026?* August 2026; vendor synthesis. [37]
29. Prolific. *Prolific's payment model*. July 2026. [30]
30. W3C. *Web Content Accessibility Guidelines 2.2*. December 2024 Recommendation. [27]

## References

1. *conveo.ai*. https://conveo.ai/docs/llms.txt
2. *Previewing and Publishing Your Guide | Question Guide – Strella Education Hub*. https://www.strella.io/education-hub/tutorial/previewing-and-publishing-your-guide
3. *Detailed Topic Guide Settings*. https://conveo.ai/docs/setup-to-launch/detailed-topic-guide-settings
4. *AI-Moderated Interviews for Qualitative Research at Scale | Outset*. https://outset.ai/platform/interviews
5. *Setting Up Your Question Guide | Question Guide – Strella Education Hub*. https://www.strella.io/education-hub/tutorial/setting-up-your-question-guide
6. *Glaut AI-moderated interviews (AIMIs)*. https://glaut.com/product
7. *Conversational Feedback - Qualtrics*. https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/conversational-feedback/
8. *Study Design 101*. https://docs.listenlabs.com/setup-to-launch/study-design-101
9. *FAQ | Outset*. https://outset.ai/resources/faq
10. *Define success criteria and build evaluations*. https://platform.claude.com/docs/en/test-and-evaluate/develop-tests
11. *Best practices for prompt engineering with the OpenAI API | OpenAI Help Center*. https://help.openai.com/en/articles/6654000-guidelines-for-prompting-large-language-models
12. *Testing Your Study*. https://conveo.ai/docs/setup-to-launch/testing-your-study.md
13. *Prompt design strategies  |  Gemini API  |  Google AI for Developers*. https://ai.google.dev/guide/prompt_best_practices
14. *PromptFlow: Supporting Novice Prompt Engineering Through Structured and Visual Scaffolding | Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems*. https://dl.acm.org/doi/full/10.1145/3772363.3799168
15. *Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions*. https://dl.acm.org/doi/fullHtml/10.1145/3381804
16. *Response Burden and Response Quality in Web Probing. An Experiment on the Effects of Probe Placement and Format*. https://ojs.ub.uni-konstanz.de/srm/article/view/8295/7678
17. *How to Design AI Interview Prompts and Discussion Guides*. https://glaut.com/post/ai-moderation-prompt-discussion-guide
18. *Setting Up Your Topic Guide*. https://conveo.ai/docs/setup-to-launch/setting-up-topic-guide.md
19. *dl.acm.org*. https://dl.acm.org/doi/pdf/10.1145/3555765
20. *Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probes*. https://arxiv.org/html/2503.08582v1
21. *AI Conversational Interviewing: Transforming Surveys with LLMs as Adaptive Interviewers*. https://aclanthology.org/2025.latechclfl-1.17v1.pdf
22. *Listen Labs Products | Read 0 Reviews on G2*. https://www.g2.com/sellers/listen-labs
23. *Guidelines for Human-AI Interaction - Microsoft Research*. https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction
24. *To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making: Proceedings of the ACM on Human-Computer Interaction: Vol 5, No CSCW1*. https://dl.acm.org/doi/10.1145/3449287
25. *Where am I? A meta-analysis of experiments on the effects of progress indicators for web surveys.*. https://psycnet.apa.org/record/2013-40239-005
26. *AI moderated interviews: methodological error amplified - Carl J. Pearson, PhD*. https://carljpearson.com/ai-moderated-interviews-methodological-error-amplified/
27. *Web Content Accessibility Guidelines (WCAG) 2.2*. https://www.w3.org/TR/WCAG22
28. *Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts | Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems*. https://dl.acm.org/doi/10.1145/3544548.3581388
29. *(PDF) When and why defaults influence decisions: a meta-analysis of default effects*. https://www.researchgate.net/publication/330600525_When_and_why_defaults_influence_decisions_a_meta-analysis_of_default_effects
30. *Prolific's payment model | Prolific Research*. https://researcher-help.prolific.com/en/articles/445230-prolific-s-payment-model
31. *Conveo Reviews 2026: Details, Pricing, & Features | G2*. https://www.g2.com/products/conveo/reviews
32. *Using the AI Assistant for Follow-Up Instructions – Engage*. https://engage-help.cloudresearch.com/hc/en-us/articles/49686420319636-Using-the-AI-Assistant-for-Follow-Up-Instructions
33. *FAQ | OpenAI API*. https://platform.openai.com/docs/faq
34. *Design Heuristics for Effective Smartphone Questionnaires | Social Science Computer Review*. https://dl.acm.org/doi/abs/10.1177/0894439317727072
35. *The AI-Moderated Research Platform | Outset*. https://outset.ai/
36. *Test Drive Interview Questions – Engage*. https://engage-help.cloudresearch.com/hc/en-us/articles/31077753629972-Test-Drive-Interview-Questions
37. *Voice vs. Text AI Interviews: Which Produces Better Insights in 2026?*. https://www.glaut.com/post/voice-vs-text-ai-interviews
38. *Self-disclosure to conversational AI: a literature review, emergent framework, and directions for future research | Personal and Ubiquitous Computing | Springer Nature Link*. https://link.springer.com/article/10.1007/s00779-024-01823-7
39. *What Is an AI-Moderated Interview? | Strella*. https://www.strella.io/blog/what-is-ai-moderated-interview
40. [
  Reasoning models | OpenAI API
](https://developers.openai.com/api/docs/guides/reasoning)
41. *Text input – GOV.UK Design System*. https://design-system.service.gov.uk/components/text-input
42. *Adding Follow Up Instructions – Engage*. https://engage-help.cloudresearch.com/hc/en-us/related/click?data=BAh7CjobZGVzdGluYXRpb25fYXJ0aWNsZV9pZGwrCJTsxj5WHToYcmVmZXJyZXJfYXJ0aWNsZV9pZGwrCJTtboUwLToLbG9jYWxlSSIKZW4tdXMGOgZFVDoIdXJsSSJEL2hjL2VuLXVzL2FydGljbGVzLzMyMjU2MjU3NjE3MDQ0LUFkZGluZy1Gb2xsb3ctVXAtSW5zdHJ1Y3Rpb25zBjsIVDoJcmFua2kH--fb1ac38e6ca3495d825eb0e52c0b7225bca5b2b3
43. *Knit | The AI-Native Research Agency | Quant + Qual*. https://goknit.com/
44. *Announcing The Forrester Wave™: Experience Research Platforms, Q1 2026*. https://www.forrester.com/blogs/announcing-the-forrester-wave-experience-research-platforms-q1-2026
45. *Listen Labs • Trusted AI Research for Leading Brands*. https://listenlabs.com/
46. *listenlabs.ai*. https://listenlabs.ai/index.md
47. *Engage: AI-Moderated Qualitative Interviews at Scale*. https://www.cloudresearch.com/products/ai-survey-platform