Evidence Hierarchy: A Plain-Language Guide
This reference helps you explain study types to someone who is not a researcher, is probably tired, and needs to make decisions about their health. Use clear language. Avoid jargon unless you define it immediately.
Why this matters
Not all evidence is equal. A single person's story about what worked for them is real, but it tells you different things than a study of 5,000 people. When someone is evaluating health information, knowing what kind of evidence they're looking at helps them weigh it appropriately.
Source rules
Peer-reviewed publications only. Only cite evidence from peer-reviewed scientific and medical literature: journals, systematic review databases (Cochrane), and clinical guidelines from recognized bodies. Do not cite preprints (medRxiv, bioRxiv, SSRN, or similar) as evidence. Preprints have not passed peer review and should be excluded entirely, even if they appear in search results.
Non-academic web sources (Mayo Clinic, WebMD, NHS, Healthline, patient advocacy sites, etc.) may be used for plain-language context or patient experience data, but must not be cited as evidence for a clinical claim. If a web source references a study, find and cite the original study instead. If the original cannot be found or accessed, note that the claim could not be traced to a primary source.
The hierarchy (strongest → weakest for clinical decision-making)
Systematic reviews and meta-analyses
What it is: Researchers gather all the existing studies on a question and analyze them together. Why it's strong: Combines evidence across many studies, reducing the chance that one flawed study drives conclusions. Can reveal patterns no single study shows. Watch for: Quality varies: a meta-analysis of bad studies is still bad evidence. Check whether the authors assessed study quality (look for terms like "risk of bias assessment" or "GRADE"). Also check how many studies were included and whether they were actually studying the same thing. If a non-Cochrane systematic review does not report a risk-of-bias assessment, flag it with ⚠️ — it should be treated as lower confidence than a Cochrane review even though the study type is the same. Plain-language version for the user: "This is a study of studies — researchers looked at everything published on this question and combined the results. It's generally the strongest evidence available, but it's only as good as the studies it includes."
Clinical practice guidelines
What it is: Recommendations developed by a panel of clinicians and methodologists, synthesizing the available evidence and adding expert consensus to produce guidance for clinical practice. Major producers include NICE (UK), ACP, WHO, and specialty societies (e.g., American Headache Society, ACR). Why it's useful: Represents the current standard of care as understood by a field. When a guideline is recent and methodology is transparent, it's a high-value source for understanding what clinicians are expected to do and why. Watch for: Guidelines age badly. A guideline more than 5-10 years old may not reflect current evidence, so always check whether an update has been issued. Quality also varies: the best guidelines use GRADE methodology to explicitly rate the evidence behind each recommendation; others are based primarily on expert consensus with limited transparency. Check whether the guideline reports its evidence-grading methodology. Always flag guidelines older than 5-10 years with ⚠️. Plain-language version for the user: "This is an official set of recommendations from a medical organization about how doctors should approach this condition. It tells you what the standard of care is supposed to be — but check when it was published, because guidelines don't always keep up with new research."
Randomized controlled trials (RCTs)
What it is: Participants are randomly assigned to either receive the treatment or not. Neither they nor (ideally) their doctors know which group they're in. Why it's strong: Randomization means the groups should be similar in every way except the treatment, so differences in outcomes are more likely caused by the treatment itself. Watch for: Sample size matters. A 30-person RCT is much weaker than a 3,000-person RCT. Also check who was in the study — if a drug was only tested on 25-year-old men, the results may not apply to a 60-year-old woman. Look at effect sizes, not just whether results were "statistically significant." A statistically significant result can still be clinically tiny. Plain-language version for the user: "This is the gold standard for testing whether a treatment works. People were randomly put in groups so the comparison is fair. But check how many people were in the study and whether they're similar to you."
Cohort studies
What it is: Researchers follow a group of people over time to see what happens. They compare people who were exposed to something (a treatment, an environmental factor) to people who weren't. Why it's useful: Good for studying things you can't or shouldn't randomize (you can't randomly assign people to smoke). Can track long-term outcomes. Watch for: Because people aren't randomly assigned, the groups might differ in ways that affect the outcome. Researchers try to control for this statistically, but they can't control for things they didn't measure. Plain-language version for the user: "Researchers followed people over time and compared groups. It's useful but not as clean as a randomized trial because the groups might have been different in ways that affect the results."
Case-control studies
What it is: Starts with people who have a condition and compares them to people who don't, looking backward for differences in exposure or history. Why it's useful: Good for rare conditions where you can't wait for cases to accumulate prospectively. Relatively fast and inexpensive. Watch for: Relies on people accurately remembering their past, which is unreliable. People who are sick tend to search harder for explanations, which can introduce bias. Plain-language version for the user: "Researchers compared people who have the condition to people who don't, looking back at their histories for differences. It's a starting point for understanding causes, but memory is unreliable and there are other biases to watch for."
Case series and case reports
What it is: Detailed descriptions of one patient or a small group of patients with a condition or response to treatment. Why it's useful: Can identify new conditions, unusual presentations, or unexpected treatment responses. Often the first signal that something exists. Watch for: No comparison group. What happened to this patient may not happen to anyone else. Cannot establish cause-and-effect. Most vulnerable to publication bias: unusual cases get published, typical cases don't. Plain-language version for the user: "This is a detailed description of what happened to one person or a few people. It can be an important early signal, but it can't tell you whether the same thing would happen to you. There's no comparison group."
Clinical framework papers and consensus statements
What it is: A paper or document that synthesizes existing evidence into a structured clinical tool — a diagnostic framework, a list of red flags, a classification system — often developed by a working group or expert panel. Distinct from a guideline (which tells clinicians what to do) and from a systematic review (which aggregates study results). Examples: the SNNOOP10 red flag framework for headache, the Rome criteria for functional GI disorders, the ACR classification criteria for rheumatoid arthritis. Why it's useful: These are often the frameworks clinicians actually use in practice. Understanding them helps a patient understand how their clinician is thinking. When validated in subsequent studies, they carry meaningful evidential weight. Watch for: The framework itself may be based on expert consensus rather than primary research — check whether it cites underlying studies. Validation studies matter: a framework that has been prospectively tested is more trustworthy than one developed purely from expert opinion. Note whether the framework was developed by an independent group or by a body with a potential interest in a particular classification. Plain-language version for the user: "This is a structured tool that clinicians use to organize their thinking — like a checklist of warning signs or a set of criteria for a diagnosis. It's based on synthesized evidence and expert agreement, not a single study. Check whether it has been tested in real patients."
Expert opinion and clinical experience
What it is: What experienced clinicians believe based on their practice, even without formal studies. Why it's useful: For conditions with very little research, clinical experience may be the best available guidance. Experienced clinicians recognize patterns across many patients. Watch for: Subject to all human cognitive biases. Clinicians remember dramatic cases more than routine ones. Their patient population may not represent the full picture. Can perpetuate outdated practices. Plain-language version for the user: "This is what experienced doctors think based on treating patients, not based on formal studies. It matters — especially for conditions without much research — but it's also subject to the same biases all humans have."
Key concepts to explain when relevant
Effect size vs. statistical significance
"Statistically significant" means the result probably isn't due to random chance. But it doesn't tell you how big the effect is. A medication might have a statistically significant effect on pain but only reduce it by 0.3 points on a 10-point scale — real, but maybe not worth the side effects. Always ask: how much did it help?
Number needed to treat (NNT)
How many people need to take a treatment for one person to benefit. A higher NNT indicates that treatment is less effective. NNT of 2 = very effective (treat 2 people, 1 benefits). NNT of 100 = modest (treat 100 people, 1 benefits beyond what would happen without treatment). Useful for putting treatment benefits in perspective.
Base rates
How common a condition actually is in the relevant population, or how common a certain scenario is when looking across a population. Base rates also shift with context: if you already have a known risk factor for a condition, the base rate among people like you may be meaningfully higher than the general population figure. Base rates can help contextualize rare conditions and likely explanations: rare conditions are still rare even when your symptoms appear to match. This isn't a reason to dismiss the possibility, but it is a reason to consider the more common explanations too and let the evidence guide you.
Generalizability
Do the study participants look like you? Many clinical trials have historically underrepresented women, older adults, people of color, and people with multiple conditions. If a study population doesn't match you, the results still have value, but the confidence that they apply to your specific case is lower.
Publication bias
Studies with positive results get published more than studies showing no effect. This means the published evidence can overstate how well something works. Systematic reviews that look for unpublished studies are more trustworthy on this dimension.