Skip to content
Learn Motion
ExploreHow it worksMembership
Log in
Learn Motion

How Doctors Judge Whether a Medical Study Can Be Trusted

1The Clinical Question and Why Study Design Follows From It2Randomization, Allocation, and the Logic of Comparison3Blinding, Follow-Up, and Who Actually Got Analyzed4Reading the Result: Effect Size, Uncertainty, and Significance5Applicability: Does This Result Fit My Patient?6Combining Studies and Forming a Verdict
The Clinical Question and Why Study Design Follows From It

When the Design Cannot Answer the Question

3 / 3
The example shows why this is a capability problem, not an execution problem. The study followed 200 patients who took a supplement and found that 80 percent felt better. The implied question is whether the supplement improves symptoms, which is a therapy question, and a therapy question requires a comparison between treated and untreated people followed forward. The case series has no comparator, so the improvement could be natural recovery, a placebo effect, or simply that people tend to feel better over time. No statistical adjustment can create the missing comparison group. This is why mismatch is fatal: the design cannot produce the structure the question requires, no matter how carefully the study was run. Keep this separate from risk of bias, which asks whether a capable design was executed well.
0:00 / 0:00

The structural test

For any study, state the question it claims to answer, then state the structure that question requires. A therapy question requires a comparison group and forward follow-up. A harm question requires either forward follow-up of exposed and unexposed people or a backward look from outcome to exposure. A diagnosis question requires the test and a reference standard applied to the same patients. If the design cannot produce that structure, the mismatch is fatal and the result cannot answer the question regardless of how well the study was run.

A mismatch in practice

A study reports that among 200 patients who took a supplement, 80 percent reported feeling better after three months, and concludes the supplement is effective. The question implied is a therapy question: does the supplement improve symptoms? The structure required is a comparison between treated and untreated groups followed forward. The design is a case series with no comparator, so the improvement could reflect natural recovery, placebo effect, or regression to the mean. The design cannot answer the question, and this is fatal, not a minor limitation.

Mismatch is not the same as bias

Risk of bias asks whether a design that could answer the question was executed well. Design-question mismatch asks whether the design could answer the question at all. A perfectly conducted case series is still unable to answer a comparative therapy question. Keep these two judgments separate: the first is about execution, the second is about capability.

Previous3 / 3Next

Learn Motion

Generate a course. Learn it properly.

Operated by Wuhan Daoyin Technology Co., Ltd.

Contact: [email protected]
Privacy PolicyTerms of Service

© 2026 Learn Motion