Index of every page on this site: llms.txt
All skills
deanpeters avatar

/survey-design-advisor

@476ae12
Run it yourself

Decide whether a survey is the right tool, then shape it. Use when you're about to send a survey and want to learn something real instead of collecting biased, unusable answers.

Use this Skill: https://skilld.dev/gh/deanpeters/product-manager-skills/survey-design-advisor

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈50 tokens for metadata: the name and description. ≈3.1k when used: this file.

Description uses 2.5% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈201 characters, or 2.5%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Survey Design Advisor

Purpose

Decide whether a survey is the right way to learn what you want to know, and if it is, design one that produces answers you can use.

Surveys are cheap to send and easy to misuse. They are good at one thing: measuring how common something is, across a group you can reach, when you already know what to ask. They are poor at discovering why, finding problems you haven't thought of, or predicting what people will do. A survey answers the questions you wrote, in the words you chose, from the people who replied. All three are biased, and the data still looks like a number.

The sentence to keep in your head: use interviews to find out what to ask, and a survey to find out how many.

Input

Works best with: What you want to learn, from whom, and what decision the answer informs.

Also useful: The people you can reach and how (email list, in-app, partners), a draft survey, and prior research or usage data on the same question.

Anything supplied with the invocation itself (text after the skill name, a pasted context dump, or an appended ARGUMENTS: line) counts as answers already given. Use it and skip whatever it covers; don't re-ask.

Arriving empty-handed? That works too. The skill starts with the question you're trying to answer and what you'll do with the result, then walks the decision from there. If a survey is the wrong tool, it says so and points you to the right one.

Example invocations:

  • Should we survey customers about a routing add-on? We want to know if they'd pay.
  • Review this survey draft before I send it to 400 plant engineers.

Key Concepts

When a Survey Fits, and When It Doesn't

You want to... Best tool Why
Find out why people behave a way, or what problems exist Interviews (discovery-interview-prep) A survey can't follow up or notice what you didn't ask
See what people actually do Usage data (usage-data-investigation) Behavior beats stated behavior
Measure how common a known thing is across a group Survey This is its job
Test willingness to pay A priced test or a real commitment Stated intent overstates behavior
Compare known options on a defined scale Survey With care for order effects

If the answer to "what will we do differently depending on the result?" is "nothing," don't send it.

Stated Is Not Done

What people say they'll do and what they do diverge, and surveys capture the first. "Would you use this?" gets yes from people who never will. Ask about past behavior ("When did you last rebuild your schedule by hand?") rather than future intent ("Would you use a tool that...?").

Bias Lives in Four Places

  • Question bias. Leading ("How much do you love..."), loaded, double-barreled (two questions in one), or full of jargon. Neutral wording, one idea per question.
  • Scale and order bias. Early options and positive anchors get picked more. Randomize options where order is meaningless, and use balanced scales.
  • Sample bias. The people who answer are the ones who like you, have time, or were reachable. A survey sent to your newsletter describes your newsletter.
  • Non-response bias. Silence is data: the busiest and the unhappiest often don't answer.

Sample: Who, and How Many

Define who you need to hear from before the question list. Then ask where they are and whether you can reach them. A small, well-chosen sample of the right people beats a large one of whoever replied. Say what the result can claim: "of the 140 plant engineers who responded from our partner lists" is honest; "the market" is not. Don't promise statistics a response count can't carry.

Length and Question Types

Short wins: every extra question loses people, and the last ones get rushed answers. Put the most important questions first, keep it under a few minutes, and use closed questions for measurement and one or two open questions for the things you didn't think to ask. A required open text box at the end is where the real finding often hides.

Test It Before You Send It

Have five people from the target group take it aloud and tell you what each question meant to them. Misread questions show up in five minutes and ruin a thousand responses.

Facilitation Source of Truth

Use workshop-facilitation as the interaction protocol: heads-up, one question per turn, numbered options, and best-guess mode on request.

Default entry mode: Guided · set in pm-skills.config.yaml (library-wide default or a per-skill override); the user's own choice in the session always wins. On the first turn, offer all three modes, then open in the default without waiting for a pick: Guided asks the first question, Context dump invites the paste, Best guess asks for a one-sentence minimum and drafts with every assumption labeled.

Anti-Patterns (what this is NOT)

  • Not a survey generator. It decides the method and shapes the design; the questions are yours to own.
  • Not a statistics tool. It won't compute significance on a thin response set.
  • Not a substitute for interviews. If you don't know what to ask, talk to people first.

Application

This skill asks up to 4 adaptive questions, then recommends a design (or a different method).

Question 1 of 4: What do you want to learn, and what will you do with it?

Agent asks:

"What do you want to find out, and what decision will the answer change?"

Listen for the method hidden in the question. "Why do customers churn?" is an interview question. "What share of accounts use feature X?" is usage data. "How many plants run aging controllers?" is a survey (or a purchased list). If no decision depends on it, say so and stop.

Question 2 of 4: Is this about how common, why, or what they'll do?

Agent asks:

"Which is closer to what you need?

  1. How common is something (how many, how often, which of these)
  2. Why something happens, or what problems exist (reasons, causes, unknowns)
  3. What people will do or pay (future behavior, willingness to pay)
  4. Not sure"

Route: 1 → survey fits. 2 → recommend interviews, and offer a short survey after to size what they found. 3 → warn that stated intent overstates behavior; recommend a priced or committed test (pol-probe-advisor) with a survey only as a supplement. 4 → help them name it.

Question 3 of 4: Who do you need to hear from, and can you reach them?

Agent asks:

"Who exactly should answer, and how will you reach them?

  1. Our own customers (email, in-app)
  2. Prospects or a market segment (a list, panel, or partner channel)
  3. People through partners or resellers
  4. I'm not sure who I need"

Use it to name the sampling bias (own customers = engaged customers; partners = partner-served) and what the results can and cannot claim. Ask how many they can realistically reach.

Question 4 of 4: How long and what shape?

Agent asks:

"How much of their time can you ask for, and do you already have a draft?"

Fit the design: under 3 minutes, 8-12 closed questions plus one open; most important first; neutral wording; past behavior, not future intent. If a draft exists, critique it first: quote each leading, loaded, or double-barreled question and say why.

Final Step: Recommend the design

## Survey Recommendation: [what you're learning]

**Verdict:** [Survey fits / Interviews first / Usage data first / Priced test instead]
**Why:** [one or two lines tied to their answers]

### If a survey: the design
- **Who:** [target and reach, and what the results can claim]
- **Questions:** [about N, closed vs open, order, past-behavior framing]
- **Bias controls:** [neutral wording, randomized options, balanced scales]
- **Sample and what it can't say:** [size realistic, non-response caveat]
- **Test first:** five people from the target group, read aloud

### Questions to fix (if a draft was supplied)
| Question | Problem | Rewrite (option to accept, adapt, or reject) |

### Next
1. [skill or action]
2. [skill or action]

Examples

Example A: SaaS, "would they pay?"

User: "We want to know if customers would pay for the Route Optimizer. Let's survey the base."

Agent (Q2): "Which is closer: how common something is, why it happens, or what people will do or pay?" → 3 (will pay).

Agent: "Stated willingness to pay overstates real behavior, and a survey to your own customers reaches your most engaged ones. A survey can tell you how many feel the rescheduling pain. It can't tell you they'll pay. Here's what I'd do:"

Verdict: Interviews first, then a priced test. Ask 6-8 dispatchers about the last time they rebuilt the day (past behavior). Then offer the add-on at a real price to a small group and count who signs up (pol-probe-advisor). If you still want a survey, use it to size the pain ("In the last two weeks, how many times did you rebuild the day by hand?"), not willingness to pay.

Example B: Industrial, "how many plants?"

User: "How many mid-size plants run controllers older than ten years? We need to size the retrofit market."

Agent (Q2): → 1 (how common). "A survey fits, with a caution about who you can reach."

Agent (Q3): "Partners reach plants they already serve, so the results describe partner-served plants, not the market." → 3 (through partners).

Verdict: Survey fits, scoped honestly. About 10 closed questions, controller age and count first, one open question on what stops replacement. Send through partners and say the result speaks to partner-served plants. Test with five maintenance engineers. Pair it with the market-sizing skill (tam-sam-som-calculator) rather than extrapolating.


Common Pitfalls

Pitfall 1: Surveying to Discover

Symptom: A survey sent to find out what problems exist. Consequence: You get answers to the questions you wrote and miss the ones you didn't. Fix: Interview first, then survey to measure what you found.

Pitfall 2: Asking About the Future

Symptom: "Would you use...?" "Would you pay...?" Consequence: Optimistic yeses that don't turn into behavior. Fix: Ask about past behavior, or test with a real commitment.

Pitfall 3: Leading and Double-Barreled Questions

Symptom: "How satisfied are you with our fast, reliable support?" Consequence: Two questions, one answer, and a nudge. Fix: One neutral idea per question.

Pitfall 4: Surveying Your Friends

Symptom: Sent to the newsletter or the most engaged customers. Consequence: The data describes your fans. Fix: Name the sample, and say what the result can't claim.

Pitfall 5: Claiming More Than the Sample Supports

Symptom: "The market wants X," from 60 replies. Consequence: A decision rests on a number that looks bigger than it is. Fix: Report counts, the reach, and the non-response.

Pitfall 6: No Test Run

Symptom: Sent without anyone reading it aloud first. Consequence: A thousand responses to a question nobody understood. Fix: Five people from the target group, read aloud.


References

Related Skills

External Frameworks

  • Past behavior over stated intent: Rob Fitzpatrick, The Mom Test.
  • Survey bias (question, order, sampling, non-response): standard survey methodology, for example Don Dillman, Internet, Phone, Mail, and Mixed-Mode Surveys.

Provenance

Distilled from practitioner experience designing and reviewing customer surveys for B2B and industrial products.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts7d3 checks · Risk SAFE
  • Gen Agent Trust Hub7d

    The survey-design-advisor skill is a purely instructional tool providing methodological guidance for research and survey design. It contains no executable code, network operations, or sensitive file access patterns.

  • Socket7d

    No alerts

  • Snyk7d

    Risk: LOW · No issues

Signed by skilld at 476ae12. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last week.

Activeupdated last week
argument-hint
[what you want to learn, and from whom]
type
interactive
theme
discovery-research
Other metadata
intent
Guided triage from a research question to a survey decision. Checks whether a survey is the right method at all, then fits the question types, sample, length, and bias controls to what you need to learn, and flags the leading questions and sampling traps that make survey data look more certain than it is. Recommends a design, or recommends interviews or usage data instead.
best_for
[
  "Deciding whether a survey, interviews, or usage data fits the question",
  "Writing questions that measure instead of lead",
  "Choosing who to ask and how many, and what the results can claim",
  "Reviewing a survey draft before it goes out"
]
scenarios
[
  "We want to know if customers would pay for a routing add-on. Should we survey them?",
  "Review my survey. I think half the questions are leading",
  "We need to size how many plants run aging controllers. Survey or something else?",
  "How many responses do we need, and who should get it?"
]
estimated_time
10-20 min

README badge

README badge for deanpeters/product-manager-skills/survey-design-advisor