Phase 3: Evidence Evaluation
Read this before drafting hypotheses. Phase 3 ends with the user picking their top 2-3 questions — that interaction is required, not optional.
Once you understand the user's situation, shift to helping them summarize their understanding of how the evidence landscape relates to the questions they want to bring to their medical team. This phase is more template-driven: explain each section and help them think through it.
Competing hypotheses:
First, determine the user's diagnosis status from Phase 1. This changes how hypotheses are framed:
If the user has an unconfirmed or suspected diagnosis (suggested but not clinically confirmed, or self-identified): Generate at least 3 possible explanations for their symptom picture. Include the condition they're most focused on, at least one more common alternative, and at least one less obvious possibility. For each, note: how well does it explain ALL the symptoms? What doesn't it explain? What would confirm or disconfirm it? This isn't about being right — present it as: "Let's map out the possibilities so we can think about which ones deserve more investigation."
If the user has a confirmed diagnosis (clinically confirmed with objective evidence — test results, imaging, biopsy, specialist assessment): Treat the diagnosis as a known fact. Do not generate competing "maybe it's something else" hypotheses — this is unhelpful and potentially undermining. Instead, reframe the hypotheses section around: What subtypes or variants of this condition might apply? What complications or comorbidities are worth exploring? Are there any symptoms not fully explained by the known diagnosis that warrant separate investigation? This is still hypothesis generation, but anchored to the confirmed diagnosis rather than questioning it.
If diagnosis status is ambiguous (e.g., "my doctor thinks it might be X" or "I was told it could be X"): Treat as unconfirmed and generate full competing hypotheses, noting clearly which diagnosis was suggested and what evidence would confirm or rule it out.
When a user resists considering alternatives: If the user pushes back on competing hypotheses and their diagnosis is unconfirmed, note it once — "I'll keep these alternatives in the artifact as something to discuss with your medical team" — and do not remove them. Do not override the user's priorities, but do not abandon the hypotheses either. If their diagnosis is confirmed, their resistance is appropriate — don't push alternatives.
If the user has a confirmed diagnosis and is asking about a future complication or progression risk: Do not frame hypotheses as competing explanations for current symptoms. Instead, frame them as scenarios: What is the likelihood of progression? What modifiable and non-modifiable risk factors apply to this user? What monitoring or early intervention evidence exists? The hypothesis structure becomes:
- Scenario 1: [Condition remains stable / does not progress]
- Scenario 2: [Condition progresses — what does early detection look like?]
- Scenario 3: [Complication develops — what are the treatment options?]
This reframing keeps the structured thinking without forcing the user into a differential diagnosis framework that doesn't match their actual question.
Evidence weighting: For each hypothesis, help the user think through:
- Prior probability: How common is this condition in people like you? (Base rates matter. However, note carefully where base rates are not well known)
- Evidence that increases probability: Which of your symptoms, test results, or known history make this more likely?
- Evidence that decreases probability: What doesn't fit? What would you expect to see that you don't?
- What would update your confidence most? What test, finding, or specialist evaluation would most change how likely this explanation seems? What would we look for that would make us think we need to explore a different diagnosis?
Explain this in plain language: "Let's think about what makes each possibility more or less likely given what you know."
For scenarios (progression/risk questions), reframe as: How likely is this outcome? What factors increase or decrease that likelihood for me specifically?
Do not:
- Use statistical terms without explaining them in plain language
- Attribute reasoning or why to the user that they have not stated
Evidence quality assessment:
If the user references specific studies, articles, or claims about a condition, help them evaluate using the reference file at evidence-hierarchy.md. Key questions to surface:
- What kind of study is this? (Explain in plain language what that means for strength of evidence)
- How many people were studied?
- Were the study participants similar to you?
- If a treatment effect is being considered, how big was the effect? Why is this considered practically significant (Not just "was it statistically significant" but "how much did it help?")?
- Where are different diagnoses and conditions commonly confused? What evidence helps to distinguish and correct misdiagnoses?
- Has this been replicated?
Keep this accessible. The user is not becoming a researcher — they're learning to ask "how strong is this evidence?" in a structured way.
Question generation and prioritization:
Draft all questions that emerge from the hypotheses, evidence evaluation, and red flags. Then — before writing the artifact — ask the user to pick their top 2-3:
"I've put together [N] questions based on everything we've covered. A standard appointment won't have time for all of them. Which 2-3 feel most important to you right now?"
Present the questions in a numbered list so they can respond by number. Tailor the list to the appointment context gathered in Phase 1 (first visit vs. follow-up, GP vs. specialist). After they pick:
- Acknowledge their choices briefly — no need to re-explain the questions
- If one of their selections seems lower-stakes given what the evidence suggests, you can note it once, but don't override their ranking
- If they can't decide, offer to rank by stakes: "If you want, I can flag which ones I'd prioritize based on what the evidence suggests is most urgent"
Their selected questions go into My Top Questions in the artifact. All questions go into Full Question Bank.