There is an irony in procurement departments buying procurement software badly, and it is common enough to be worth naming. The discipline that exists to run rigorous selection processes tends, when buying for itself, to start from the vendor's framing rather than its own requirement.
These five questions are deliberately about you rather than about the market. If you cannot answer them, no shortlist will save you.
1. What does this step cost us today?
Not the total departmental budget. The specific step you are proposing to change, expressed in hours per case multiplied by cases per year. If the answer is a range spanning an order of magnitude, you do not yet know enough to evaluate an offer, because you cannot tell a good business case from a bad one.
Getting this number is unglamorous work: sitting with the people doing the task and timing it, several times, across easy and hard cases. It also tends to be the single most valuable week of the whole project, because it frequently reveals that the expensive step is not the one everybody assumed.
2. What exactly arrives, and in what condition?
Vendors demonstrate on clean inputs. Your reality includes the supplier who sends a photograph of a delivery note, the one whose PDF is a scan of a fax, and the one who changed their layout last quarter without telling anyone. Ask for a proof of concept on a sample you choose — including the twenty per cent that are awkward — rather than on the sample the vendor brings.
3. Who decides, and what are they allowed to decide?
Almost every failed implementation I have seen traces back to an approval rule that was never written down and turned out to be contested. Before automating a decision, establish who currently owns it, what their limit is, and what happens when they are on holiday. If two people give different answers, you have found something more valuable than a software feature.
Automating a disputed process does not settle the dispute. It encodes it, at speed, in a place where nobody can see it.
4. How will we know it is wrong?
Every automated step needs a detection mechanism, and the good ones are boringly concrete: a sample checked manually each week, a threshold above which a human always looks, a monthly reconciliation. Ask the vendor how their system behaves when it is uncertain. If the answer is that it is not uncertain, that is the answer to your question about the vendor.
5. What happens when we want to leave?
Where does your data live, in what format can you get it out, and what does the process look like on the day you decide to switch. This is standard practice for every other category your department buys. It gets skipped surprisingly often when the subject is AI, apparently because the technology feels too new to have an exit.
Warning signs in a vendor conversation
- The demonstration uses their data and cannot be repeated with yours.
- Accuracy is quoted as a single percentage with no description of the test set.
- The business case depends on headcount reduction the business has not agreed to.
- Nobody can explain what happens to an ambiguous case.
- The implementation plan contains no process work, only configuration.
The uncomfortable possibility
Work through these five honestly and a certain number of projects will not survive the exercise. A step that costs less than assumed, a process that needs a decision rather than a system, a document flow that would be better fixed at its source with the supplier.
That is a good outcome, arrived at cheaply. The projects that do survive tend to be noticeably better specified than the ones that skipped the questions — and they are much easier to defend when someone asks, two years later, what the money bought.