≈596 tokens on demand. Your agent reads this file only when SKILL.md points to it.
Cognitive Bias Checklist
Purpose: Bias detection and prevention across study design, interview execution, analysis, and reporting.
Contents: design biases, execution biases, analysis biases, review checklist, report-language checks.
Study Design Biases
| Bias |
Check |
Mitigation |
| Confirmation bias |
Are questions neutral and falsifiable? |
Include disconfirming questions |
| Sampling bias |
Are channels diverse? |
Recruit across segments and channels |
| Self-selection bias |
Are only enthusiastic users responding? |
Include passive and dissatisfied users |
| Professional participant bias |
Is prior research participation screened? |
Filter or flag frequent participants |
Interview Execution Biases
| Bias |
Sign |
Mitigation |
| Social desirability |
Overly positive answers |
Ask about behavior, not virtue |
| Leading bias |
Interviewer cues steer answers |
Start open, keep reactions neutral |
| Primacy effect |
First participant dominates memory |
Aggregate before concluding |
| Recency effect |
Last participant dominates memory |
Review all data together |
| Hawthorne effect |
Artificial behavior under observation |
Use naturalistic context where possible |
Analysis Biases
| Bias |
Check |
Mitigation |
| Cherry-picking |
What evidence was excluded? |
Code all data systematically |
| Pattern recognition bias |
Is the pattern only intuitive? |
Multi-reviewer challenge |
| Hindsight bias |
Does this feel "obvious" only after the fact? |
Document hypotheses in advance |
| Anchoring |
Are early interpretations still unchallenged? |
Seek contradictory data |
| Conformity bias |
Did the team align too quickly? |
Individual review before synthesis |
Prevention Protocol
- Use neutral questions.
- Capture exact participant words.
- Separate observation from interpretation in notes.
- Search for contradicting data before finalizing insights.
- State sample limits and confidence levels in reports.
Dangerous Report Language
| Risky phrasing |
Better phrasing |
| "Everyone said..." |
"8 of 8 participants said..." |
| "Users prefer..." |
"Most participants in this study preferred..." |
| "Clearly..." |
"The data suggests..." |
| "As expected..." |
"This was consistent with the prior hypothesis..." |
