← All publications

From the literature

Excel versus Knight: when 72% is just a well-groomed opinion

Finance loves turning the future into probabilities. Knight does not forbid the exercise. He simply asks us not to confuse a useful estimate with an objective measurement.

Sometimes all it takes is a "%" symbol for an assumption to look as though it has spent time in a laboratory. This is precisely the kind of conceptual slippage Knight wants to avoid. He distinguishes probabilities that can be calculated or inferred from statistical classes from those that merely express our confidence in a judgement.

The reference class problem

In a perfectly defined lottery, probability is a property of the device. In a sufficiently homogeneous portfolio of policyholders, a past frequency provides a usable estimate. But in a one-off strategic decision (launching a technology, acquiring a company, anticipating the adoption of a product), the situation is rarely so tidy.

Knight puts his finger on the central difficulty, the one statisticians would later call the reference class problem. To calculate a frequency, you have to define a group of "similar" cases. Yet this group never exists ready-made. It is the analyst who decides what counts as comparable, and that decision is already a judgement.

Take a familiar assumption: "projects of this type succeed in 40% of cases". Which projects, exactly? Those in the same sector? Of the same size? Led by teams with the same experience, in the same economic climate, with the same financing model? The broader the class, the larger the sample but the less relevant it is. The narrower the class, the more relevant it is but the fewer cases it contains. At the limit, the perfectly relevant class contains only one element: the project itself. And a frequency over a sample of one is not a frequency.

What the number means

This does not mean models should be abandoned. On the contrary, scenarios, sensitivity analyses and subjective probabilities are useful for disciplining reasoning and making disagreements explicit. But their usefulness depends on how clearly three things are distinguished: what is observed, what is assumed and what is merely judged plausible.

A model becomes misleading when it blurs this boundary. A 72% probability of success implicitly suggests that 720 out of 1,000 identical projects would succeed. But do 1,000 identical projects really exist? If not, the figure is the numerical translation of an opinion. That is not illegitimate; it is simply something else, and that something else is not used in the same way.

The danger is compounded by an effect that has since been well documented: apparent precision breeds confidence. A rate stated to the decimal point seems more reliable than a wide range, when the opposite is often true. An honest range declares its ignorance; a decimal conceals it.

A question to ask in committee

For an investment committee, good Knightian practice comes down to one question, asked systematically for every important assumption: where does this number come from?

The possible answers are not of equal worth. Historical data on the same asset. A class of comparables built by the analyst, in which case you need to know which one. A causal model based on physical or contractual relationships. An expert consensus, which is an aggregated opinion and not a measurement. A management judgement, which remains legitimate but is not neutral when that management is leading the project. Each answer implies a different use of the number and a different response if reality departs from it.

The paradox is that acknowledging uncertainty often makes the model more useful. You stop defending an artificial precision and can focus the analysis on the few variables that genuinely deserve monitoring, a test, a decision milestone or an exit option. The aim is no longer to make people believe the future has been tamed. It is to prepare a decision that will survive the fact that it has not been.

Frank H. Knight, Risk, Uncertainty, and Profit (1921), chapter VII. Editorial adaptation, not a quotation.