Home — AI Practice
AI in quality engineering,
with the guardrails on
We use frontier language models daily, and we are standardised on Anthropic’s Claude. This page sets out exactly where the models help, where they do not, and what governs their use on a client engagement.
Claude is our
standard model.
Anthropic’s Claude is the default model across our engineering work — scenario generation, framework scaffolding, failure triage and analysis.
The choice is deliberate rather than incidental. Three properties matter for the work we do: long-context reasoning that can hold an entire test framework and a specification at once; strong instruction-following, which is what makes generated Gherkin usable rather than merely plausible; and a vendor whose published position on safety and evaluation is one we can put in front of a bank’s risk committee without flinching.
We are not dogmatic about it. Some clients mandate a specific model estate — a bank standardised on a cloud vendor’s AI services, a programme with an existing enterprise agreement — and we work inside whatever governance the client already operates. Our tooling is written to be model-agnostic at the interface for exactly that reason.
Where a client leaves the decision to us, we choose Claude.
Six jobs. Not
“AI-powered testing”.
There is a great deal of noise in this space. Here is the specific, unglamorous list of tasks where a language model measurably beats the alternative in our practice — and nothing beyond it.
- Scenario generation from acceptance criteria
- Feed the model the specification, the domain glossary and the existing suite; get back candidate Gherkin covering paths a human under deadline pressure tends to skip. Every candidate is reviewed against the risk model before it enters the suite. On a live e-commerce replatform this accelerated scenario generation by 40%.
- Test data synthesis
- Generating realistic, structurally valid, non-production data — card numbers that pass Luhn, addresses that satisfy the validator, HL7 messages that parse. This replaces the copied-from-production dataset, which is the single most common privacy failure we find in test environments.
- Failure triage across large result sets
- A 2-million-transaction execution produces more failure output than anyone reads. Clustering failures by likely root cause turns four hours of log reading into twenty minutes of confirmation. The engineer still confirms; the model just orders the queue.
- Framework scaffolding
- Page objects, fixtures, API clients, boilerplate. Mechanical work with a known-correct shape. Reviewed like any other pull request, because generated code is still code you have to own.
- Requirement gap analysis
- Reading an epic and asking what a tester would ask — what happens on partial failure, on timeout, on a duplicate submission. On one card platform migration, requirements arrived materially incomplete; structured interrogation of the specification is how those gaps get closed before they become defects.
- Telemetry and trace summarisation
- Turning a wall of APM output into a ranked account of what changed between two executions. It complements the automated comparison tooling we build; it does not replace the measurement.
Nothing a model writes
reaches a client unreviewed.
If you are engaging a quality firm, the last thing you want is unverified output entering your test suite. Our rules are short, absolute, and we will put them in the engagement documentation.
Human review is mandatory
Every generated scenario, every generated line of framework code, is reviewed by the engineer who owns it before it is committed. No exceptions, no “it looked right”.
Client data does not go into a model
No production data, no personally identifiable information, no credentials, no client source code — unless the client has explicitly authorised it in writing and the destination is inside their own approved model estate.
Generated coverage is measured, not assumed
A model will happily generate forty scenarios that all exercise the same path. Coverage is verified against the risk classification, the same as any hand-written test.
The suite must run without the model
Models assist authorship. They are never in the execution path. A test that only passes because a model was available at runtime is not a test.
Provenance is recorded
We track which artefacts were AI-assisted, so a client can audit it later and so we can tell honestly whether the approach is actually paying for itself.
Generation sits inside the process. It is never the whole process
An unreviewed generated test is worse than no test. It produces the green tick without the assurance, and somebody downstream will trust it.
The honest limits.
A short list, because the gap between what this technology is sold as and what it does is where most disappointment lives.
- It does not replace a test engineer. It replaces the typing. The judgement about what is worth testing is the entire job, and that judgement is still ours.
- It does not find what nobody described. A model reasons over the specification you give it. The defect nobody wrote down is still found by an engineer who has seen the failure mode before.
- It drifts toward the obvious case. Generated suites over-represent the happy path. The partial refund, the token that expires mid-checkout, the fulfilment split that strands a line item — those come from the risk model, not the generator.
- It does not do performance engineering. A model can help draft a JMeter plan. It cannot tell you your connection pool exhausts at 4,000 concurrent sessions. Only measurement does that.
- The percentage gains are real but bounded. Forty percent faster scenario authorship is a genuine, measured result. It is not the same as forty percent less testing, and we will not present it that way.
This is the subject
of our doctoral research.
Our principal’s PhD research is in machine learning applied to software testing, with post-graduate study in AI and machine learning at Caltech and a data science specialisation alongside it.
That matters less as a credential than as a habit. We read the evaluation methodology before we accept the benchmark. We are enthusiastic about the tooling and sceptical about the claims, in roughly that order — which is, after all, the disposition we are hired for.
Trademark note. Claude and Anthropic are trademarks of Anthropic PBC. Innova IT Ltd is an independent consultancy. References on this site describe the tools we use in our own engineering practice and do not imply any partnership with, sponsorship by, endorsement by, or affiliation with Anthropic PBC.
Bring us the release you are
least comfortable shipping.
A short, specific conversation is usually enough to tell whether we can help. No discovery deck, no bench to feed — just an engineer who has broken systems like yours on purpose, many times.
Typical first engagement: a two to three week assessment producing a workload model, an instrumented baseline and a ranked list of what will fail first.