The Limitations That Most Often Undermine a Prediction | AI Learning Guide

Look at the list as a checklist you can run against any score. Is the comparison group thin, or drawn from a single disease area or era? Does the candidate sit in a different chemical family, mechanism, or patient population than the training drugs? Were its properties measured with the same assays and standards as the training data, or is the model reading a distorted description? And is the history itself representative, or does it contain only the drugs someone earlier chose to test? That last one is the hardest to see, because the blind spot comes from human decisions rather than from the algorithm. What ties all four together is that none of them appears in the output. A score built on almost nothing looks identical to a well-supported one, so the context has to be checked by hand.

0:00 / 0:00

Auto-play

The recurring sources of unreliability

A thin or narrow comparison group: too few comparable past drugs, or all of them from one disease area or era.
A mismatch between training drugs and the candidate: a different chemical family, mechanism, or patient population.
A shifting measurement: candidate properties recorded with different assays, instruments, or standards than the training data.
A non-representative history: the record contains only drugs that were actually tested, so it inherits the blind spots of earlier human choices.

A score produced from a thin comparison group looks exactly like one produced from a rich one. The output carries no warning about the quality of the evidence behind it, so that context has to be checked separately.