Black-and-white close-up of a motherboard

AI IMPLEMENTATION

Evaluating AI features: a practical framework for UK SMEs

Most UK SMEs we work with have been pitched four AI features by their existing software vendor in the last year — and many of those features will be quietly deprecated by 2027. Here’s the three-question framework we use with clients to decide which AI features to actually invest time in, and which to politely ignore.

The three-question framework

Three questions, asked in order, before you turn on any new AI feature in a product you already pay for. Question 1: who else in your industry uses this feature, and what did it replace for them? Question 2: what’s the workflow when the AI is wrong, and who handles that workflow? Question 3: who owns the data the feature trains on, processes, and stores?

These are not technical questions. They’re commercial and operational. We’ve found they cut the noise from vendor pitches by about 80% — most pitched features fail one or more of them, and the failures are exactly where the long-term cost lives.

Question 1: who else uses this feature?

Pitched AI features fall into two categories. Mature ones — like email summarisation, meeting transcripts, basic copilots — have a year or more of customer references in your industry. New ones — anything launched in the last six months — typically have references that are case studies the vendor wrote, not workflows your peers actually rely on.

We’re not snobbish about new features. But before adopting one as a load-bearing part of an SME workflow, find two non-vendor references in your sector. If the vendor can’t introduce you, that is information. The feature may be excellent technically and not yet suitable as something your team plans around.

Question 2: what’s the workflow when the AI is wrong?

Every AI feature has a non-zero error rate. The right question is what the workflow looks like when it errors. Does someone notice? How? Who fixes it? Is the fix in the system, or do they fix it in their head and move on? The last option is the dangerous one. Silent fixes accumulate into systemic distrust of the feature.

A practical example: an AI-suggested invoice category that’s wrong 2% of the time. If the workflow is ‘finance reviews and corrects every entry anyway’, the feature is useful (it speeds up the easy 98%). If the workflow is ‘the entries go straight into accounts and finance reviews quarterly’, the feature is dangerous because the 2% errors compound over the year. Same feature, completely different verdict, depending on the workflow around it.

Question 3: who owns the data?

Most AI features in vendor products send data to the vendor’s model provider, sometimes within the EU/UK and sometimes not. For UK SMEs subject to UK GDPR, this is not optional knowledge. The right answers are documented in the vendor’s data processing addendum (DPA) and they should be checked before turning the feature on, not after. A surprising number of AI features in popular SaaS tools default-on a setting that processes customer data outside the UK and the EEA.

If you’re in a regulated sector — finance, healthcare, legal services — this is the question that most often disqualifies a feature outright. The vendor’s standard DPA is rarely sufficient. You will need a bespoke addendum or a vendor that supports UK-only data residency. Both are increasingly available, but neither is automatic.

A worked example: an HR copilot pitch

A vendor pitches a copilot inside your HR system that drafts performance review summaries from manager notes. Question 1: two competitors in your sector use a comparable feature, references available — pass. Question 2: the workflow when the AI is wrong is that the HR manager edits the draft before it’s sent to the employee — pass, because errors are caught before they propagate. Question 3: drafts are processed in a US data centre and the DPA addendum is generic — fail.

The verdict isn’t ‘don’t use it’. It’s ‘don’t use it until the data residency is fixed.’ That’s a different conversation with the vendor. We help SME clients navigate exactly these conversations as part of AI implementation engagements — not always to add features, often to evaluate ones already pitched.

Vendor pitched you an AI feature?

We run 30-minute triage calls on adoption decisions, free.