Confidence should follow the evidence.
Too much trust in AI is a risk and too little is a waste, so test what people believe and what they rely on separately.
By Think Assets
More trust is not the aim
A good trial of AI does not simply leave people more confident. It helps them place confidence in the right contributions, for the right reasons. An adviser can be sure of a total that reconciles and still be unsure of the forecast assumption behind it. One confidence score for both hides the distinction that matters.
Confidence is not reliance
Confidence is what someone believes about a statement or a judgement. Reliance is what they go on to use. A reviewer can voice doubt and accept the recommendation anyway, or voice confidence and redo all the preparation. So a test needs both: stated confidence tied to particular questions, and what people did in practice, whether they accepted, rejected, checked or asked for more evidence.
Look both ways
Too much confidence can turn a tentative assumption into what looks like a settled conclusion. Too little can leave sound analysis unused, send questions upwards that did not need to go, or use up capacity on repeated checks. Neither is a label to attach because someone agreed or disagreed with the AI. The reference point is the evidence and the task, never the answer the AI preferred.
Agree what sound judgement looks like
Some statements have a plain reference: the calculation reconciles or it does not. Others involve interpretation, and professionals can reasonably disagree. So the firm's senior people set the criteria, the right actions and the reasoning expected, before anything is run. Disagreement is recorded, and so are the limits of the evidence. Where a question allows more than one defensible answer, no single correct answer is forced on it.
Make the comparison a fair one
Include cases where the AI is right and cases where it is not, assumptions that are supported and assumptions left open, and cases where the expert's first view deserves to change. Match the cases, or the groups of people, so that nobody is remembering a file or learning as they go. Record which versions of the tool and the sources were used, and keep some past files back for the test. If asking for a confidence score changes how people work, the design has to allow for that. How many cases to run depends on the decision at stake and the uncertainty the firm can accept.
Keep the conclusion in proportion
A useful result separates better decisions, justified confidence and proper use of the AI. It reports the whole effort of review and the failures that matter, not only an average score. An AI's own statement of how sure it is cannot be read as a measured probability. It has to be tested before anyone leans on it.
At Think Assets this is the proof inside the third of our four steps, Show what it is worth, and it shapes the second, Design the way of working: where to rely on the AI, and where to look. The finance worked example shows the distinction on a piece of due diligence, where every balance agrees to its schedule and one rests on an estimate.
See the finance worked exampleRead the thinking on calibrated confidence