An AI trial is a snapshot, not a permanent verdict.
When to look again at what is delegated and what experts review, as the tools, the sources and people's habits change.
By Think Assets
Both sides of the arrangement change
A firm can reject a tool after one trial, or go on using a way of working long after it was last tested. Neither decision should become permanent by default. What the AI can do changes. The source material changes. Experts form new habits. The question is when those changes are large enough to look at the way of working again.
Agree what prompts a review
When a way of working is adopted, write down what the task covers, the version of the tool and the instructions it was given, the sources it needs, its known limits, and who is responsible for review. Agree what would justify another test: a significant change to the tool, a new kind of document, a different group of users, a change in what the decision requires, or signs that the work is taking unexpected effort or producing unexpected results. A date in the diary helps, but it should not be the only prompt.
Watch the tool and the habits together
Better extraction can make more of the AI's preparation worth using. Familiarity can also make people less alert to its qualifications. And a poor early experience can leave a team rebuilding work that a later version now does dependably. So look at what people do with the AI's work: where they accept it, reject it, look further or do it again. Then ask whether the evidence still supports each of those habits.
Test the change in question
A focused test answers one question. If a new way of finding documents keeps track of their versions more reliably, find out whether the combined review is better for it, and whether that supports delegating more. Keep a fixed set of past files so that results can be compared over time, and add new ones, so that familiarity with the old files is not mistaken for general ability. Client files stay in the firm's environment, used only with the permissions they require.
Protect the judgement worth keeping
Changing the way of working changes which skills people practise. Consider whether experts can still challenge the AI, read evidence of a kind they have not met before, and recognise a case that falls outside what was tested. Continuing advice looks at who does what and what follows from it, not only at the tool's latest published score.
Turn each review into a decision
Each review should end in a decision about investment or about the work: keep the arrangement, delegate more, refine a handover, look into a concern, or run a focused comparison. Quality of service, the whole effort of delivery and usable capacity sit beside the cost of making the change and running it. A technical improvement supports investment only if the firm can make worthwhile use of it. Record the reason, and what will prompt the next review.
This is the fourth of our four steps, Keep it right: we stay alongside the firm's leadership as AI changes. In the real estate worked example, the evidence that mattered sat in a side letter and not in the lease summary. The sources can matter as much as the tool.
See the real estate worked exampleSee how we work with a firm