For every other category a requisitioner states the requirement and procurement tests it against the market. When the department buys for itself, that role is absent: requisitioner and reviewer are the same person. What is missing at the start, then? The requirement. What lies on the table instead is the vendor's material, because it is the only paper in the room.
These five questions are deliberately about you rather than about the market. If you cannot answer them, no shortlist will save you.
1. What does this step cost us today?
Not the total departmental budget. The specific step you are proposing to change, expressed in hours per case multiplied by cases per year. If the answer is a range spanning an order of magnitude, you do not yet know enough to evaluate an offer, because you cannot tell a good business case from a bad one.
Getting this number is unglamorous work: sitting with the people doing the task and timing it, several times, across easy and hard cases. It also tends to be the single most valuable week of the whole project, because it frequently reveals that the expensive step is not the one everybody assumed.
2. What exactly arrives, and in what condition?
Vendors demonstrate on clean inputs. Your reality includes the supplier who sends a photograph of a delivery note, the one whose PDF is a scan of a fax, and the one who changed their layout last quarter without telling anyone. Ask for a proof of concept on a sample you choose — including the twenty per cent that are awkward — rather than on the sample the vendor brings.
3. Who decides, and what are they allowed to decide?
Almost every failed implementation I have seen traces back to an approval rule that was never written down and turned out to be contested. Before automating a decision, establish who currently owns it, what their limit is, and what happens when they are on holiday. If two people give different answers, you have found something more valuable than a software feature.
Automating a disputed process does not settle the dispute. It encodes it, at speed, in a place where nobody can see it.
4. How will we know it is wrong?
Every automated step needs a detection mechanism, and the good ones are boringly concrete: a sample checked manually each week, a threshold above which a human always looks, a monthly reconciliation. Ask the vendor how their system behaves when it is uncertain. If the answer is that it is not uncertain, that is the answer to your question about the vendor.
5. What happens when we want to leave?
Where does your data live, in what format can you get it out, and what does the process look like on the day you decide to switch. For every other category that clause is already in the template. For consumption-based AI services the template does not exist yet, because the category is younger than the rulebook — so the exit has to be written in by hand.
Warning signs in a vendor conversation
- The demonstration uses their data and cannot be repeated with yours.
- Accuracy is quoted as a single percentage with no description of the test set.
- The business case depends on headcount reduction the business has not agreed to.
- Nobody can explain what happens to an ambiguous case.
- The implementation plan contains no process work, only configuration.
The uncomfortable possibility
Work through these five honestly and a certain number of projects will not survive the exercise. A step that costs less than assumed, a process that needs a decision rather than a system, a document flow that would be better fixed at its source with the supplier.
That is a good outcome, arrived at cheaply. The projects that do survive tend to be noticeably better specified than the ones that skipped the questions — and they are much easier to defend when someone asks, two years later, what the money bought.