A few months ago I would have said my job is to check code. Write tests, cover edge cases, sit at the end of the pipeline and decide: through or back. That is no longer my job. This is not a complaint. It is a finding.
The classic QA model works as long as the output is deterministic. You put in an input, you expect a defined output, you test for it. But when the system under test is an agent, every run looks a little different. assert output == expected stops being your tool. You no longer test a result. You test a system that produces results.
From the gate to the architecture
The question shifts. Not: is this output correct? But: is this workflow able to guarantee quality at all? That is a structural question, and it gets answered before the first line of code is written.
So QA helps design the phases, defines the gates and writes the instructions that govern the process. Not alone, but as part of the team — as the one who keeps watching whether the architecture actually holds.
QA no longer sits at the quality gate. QA guards the architecture.
Benchmark the process, not the output
Once the workflow is the product, you compare workflows. A flow without review that produces four times as many reverts is a measurable statement. The numbers that matter:
- Time to done — from plan to merge.
- Defect escape rate — what slips past the QA gate.
- Workflow adherence — were the phases actually followed?
- Revert rate — how often the result gets undone.
A skipped review or a merge without sign-off is not cosmetics. It is a quality incident. Track it like one.