Synthetic Demand Calibration
Purpose: Validate synthetic user demands against real user data to measure and improve generation accuracy.
Calibration Framework
Why Calibrate
Synthetic demands generated by Plea are hypotheses, not facts. Without calibration against real-world data, they risk:
- Over-representing articulate user segments
- Missing culturally-specific needs
- Generating technically-flavored demands that real users would never express
- Confirming existing team biases rather than challenging them
Calibration Sources
| Source | Agent | Signal Quality | Use When |
|---|---|---|---|
| Real user feedback | Voice | High | Post-launch, ongoing product |
| Session replay patterns | Trace | High | Behavioral evidence available |
| Interview transcripts | Field | High | Research cycle completed |
| Support tickets | Voice | Medium | Support data accessible |
| NPS/CSAT comments | Pulse | Medium | Survey data available |
Calibration Process
- Generate: Plea produces demand set for target feature area
- Match: Compare synthetic demands against real data from Voice/Trace/Field
- Score: Rate each demand on the calibration matrix
- Tag: Apply confidence tags based on match results
- Adjust: Refine persona channeling for low-match segments
Calibration Matrix
| Match Level | Tag | Meaning | Action |
|---|---|---|---|
| Direct match | [validated] |
Real users voiced the same demand | High confidence, proceed |
| Partial match | [supported] |
Real data implies similar need | Medium confidence, refine wording |
| No match (plausible) | [hypothesis] |
No real data yet, but realistic | Flag for research validation |
| No match (implausible) | [synthetic-only] |
Likely an artifact of AI generation | Review for removal or revision |
Calibration Thresholds
- Healthy: ≥60% of demands are
[validated]or[supported] - Acceptable: 40-59% match rate — increase Field/Voice input
- Low confidence: <40% match rate — trigger recalibration with fresh Voice/Trace data
Recalibration Triggers
- New Voice feedback batch available
- Trace detects behavioral shift ≥5%
- Product pivot or major feature launch
30 days since last calibration
Integration with Cast
When calibration reveals persona accuracy issues:
- Feed match results back to Cast via FUSE mode
- Low-match personas may need Cast EVOLVE or RETIRE
- High-match personas strengthen Cast confidence scores