/field

Research Calibration System (DISTILL)

Purpose: Track research quality, recommendation adoption, and method effectiveness over time. Contents: DISTILL workflow, adoption thresholds, calibration rules, question effectiveness, journal format, ecosystem sharing.

DISTILL Workflow

TRACK -> ASSESS -> REFINE -> SHARE

What To Track

• method effectiveness by context
• question types that produce rich data
• recommendation adoption
• insight accuracy after shipping
• participant segment usefulness
• bias detection and correction rate

Adoption Thresholds

Assessment Triggers

Calibration Rules

1. Require 3+ studies before adjusting method weights.
2. Cap each adjustment at +/-0.15 per cycle.
3. Apply 10% decay per quarter toward defaults.
4. Explicit user method preference overrides calibration.
5. For studies with < 3 insights, record only; do not update weights.

Question-Type Calibration

Journal Entry Format

## YYYY-MM-DD - DISTILL: [Study Type]

**Studies assessed**: N
**Overall adoption rate**: X%
**Key insight**: [description]
**Calibration adjustment**: [method/question: old -> new]
**Apply when**: [future scenario]
**reusable**: true

<!-- EVOLUTION_SIGNAL
type: PATTERN
source: Field
date: YYYY-MM-DD
summary: [research methodology insight]
affects: [Field, Echo, Spark, Voice]
priority: MEDIUM
reusable: true
-->

Per-Recipe Behavior (SKILL.md excerpt)

Behavior notes per Recipe:

• interview: Define research questions → author guide → design screener. Includes AI-moderation fit evaluation.
• usability: Test planning and task scenario design. Apply SUS/SEQ/CASTLE benchmark thresholds.
• analysis: Thematic analysis, coding, and affinity mapping. Bias check required.
• persona: Generate personas from research data. Disclose WEIRD bias and prepare Cast handoff.
• journey: Journey mapping + JTBD switch interview analysis. Includes Echo[demand] handoff determination.
• survey: Quantitative survey design — item authoring, scale selection, sample-size calculation, order-bias control, Cronbach's α validation. For usability cognitive walkthrough use Echo; for production KPI tracking events use Pulse; for operational NPS/CSAT feedback pipelines use Voice.
• diary: Longitudinal behavioral study — study length, ESM prompt frequency, self-report bias mitigation, fatigue management, media capture. For passive in-product telemetry use Pulse; for single-session cognitive walkthrough use Echo; for retrospective feedback mining use Voice.
• cards: IA validation — open / closed / hybrid card sort, tree testing, first-click testing, dendrogram and similarity-matrix analysis. For UI comprehension walkthrough use Echo; for post-launch navigation analytics use Pulse; for post-launch findability complaints use Voice.
• multi: Multi-engine research-design generation (see Multi-Engine Mode section + reference/tri-engine-research.md for the full SCOPE → PREFLIGHT → FAN-OUT → NORMALIZE → CLUSTER → SCORE → GROUND → SYNTHESIZE → PRESENT flow). Critical difference from Judge: divergent methodologies are NOT auto-low-value — triangulation is the discipline's quality lever.

Secondary Thresholds (SKILL.md excerpt)