Two Ways a Design Can Be Hard to Trust | Visual AI Learning Guide

Look at the two sides of this comparison. On the left, the design process leaves a visible trail: a target, a binding site, a set of properties, a trade-off. Someone can read that trail and ask whether the reasoning holds. On the right, the trail is missing — a molecule appears, but no one can say why. Now notice that the diagram separates this from a second question. Explainability is about whether the reasoning can be stated. Reproducibility is about whether the result can be regenerated. A process can be fully explainable and still not reproducible, or reproducible and still opaque. They are independent, and each one can block a drug on its own.

0:00 / 0:00

Auto-play

When a molecule is proposed by a model rather than by a chemist reasoning from known biology, two separate questions arise. They are often lumped together as 'the black-box problem,' but they are different and they fail for different reasons.

Explainability asks: can anyone say why this molecule was proposed? A transparent design process leaves a chain of reasoning — this target, this binding site, this property, this trade-off. A black-box process produces a molecule without a legible reason. The molecule may still be excellent, but the developer cannot explain the design choices, and a regulator cannot evaluate whether the reasoning was sound or lucky.

Reproducibility asks: if the same input is run again, do you get the same result? This is a practical auditing problem. If a model's output depends on random seeds, on a specific version of the software, or on data that cannot be shared, then no one else can regenerate the result. Regulators cannot audit it, and other scientists cannot build on it. A result that cannot be reproduced is, for regulatory purposes, not a result at all.