Validation Documentation for an AI-Built Quality System

The Situation

A life-sciences compliance and technology consultancy decided to replace the commercial quality management system it had used for years with one our team would build in-house. I ran the program and led its validation documentation.

A quality management system, or QMS, is where a company in a regulated industry keeps the documents that prove it does its work properly: its procedures and work instructions, staff training records, audits, deviations and corrective actions, all under version control with electronic signatures. Regulators and clients audit against it. Because so much depends on it, the system itself has to be validated before it can be used: written evidence that it does what it is supposed to, reviewed and signed by the people accountable for quality.

The client validates under its own procedure, aligned with GAMP 5 and with Part 11 electronic signatures. That means an ordered set of documents, from the plan and requirements through design, testing, traceability and release, each with a named author, reviewer and approver. And every one of those documents has to hold up when an auditor reads it.

Sixty Days

Once notice went to the old vendor, the deadline was fixed. The team had roughly 60 days to build the new system, move the controlled documents across and get off the old one, with validation running alongside rather than after.

Normally a validation package is written by a validation team over months, once the software has settled down. Here the software was being built with AI, fast, and changing daily, and the documents had to keep pace with it. In September the validation work met almost daily, and every ruling from those sessions went into the controlled documents the same day, through the same meeting-to-records process I use to run projects inside the repository.

The Package

I led the validation documentation and took part in writing every document in it. The package came to more than 80 controlled documents, roughly 600 pages, covering the full lifecycle:

Part of the package	What it does
Quality and project plan	How the system will be validated and by whom
System requirements	What the system must do, as numbered, testable requirements
Design and configuration specifications	How the system meets each requirement
Test plan and test cases	How each requirement will be proved
Test, qualification and summary reports	Where the results are recorded
Traceability matrix	The thread from every requirement to its design and its test
Tool qualifications	Proof that every tool in the build chain is fit for its use
Data migration and retirement plans	Moving controlled documents off the old system, then retiring it
Release notes and release request	The documents that close the package

Our technical lead wrote the technical documents with me: architecture, configuration specifications, the installation guide and the code review plan. Approval, test execution and signatures belong to the client’s validation and QA leads.

Turning Edits into a Reusable Method

AI developer tools helped draft the package inside the repository, where the application code, plans and supporting documents were available together. Much of my time went into reading those drafts line by line, checking them against the implementation and the client’s procedures, and directing the next revision.

As that work accumulated, I started treating an edit as more than a correction to one document. If the same problem could appear in the next draft, it needed to become a writing convention the tools could follow.

For example:

• A draft described completed work as something still to come. I corrected the sentence and added an instruction to check the wording against what had actually happened. Later drafts and reviews could use the same check.
• A reference needed its proper title and identifier. Instead of accepting a name copied from another draft, I checked it against the client’s document index and made that verification part of the drafting instructions.
• An edit affected more than the page in front of me. When we changed a term or corrected an explanation, I had the tools find the other places it appeared, carry the change through, and repair the references between documents.

I collected those conventions in an agent skill: a reusable set of instructions kept alongside the work. It covered how to structure a document, write plainly, check references, and apply changes across the package. Each new round of editing added to it, so the next task began with the lessons from the previous ones.

That was the useful part of the method. The work I put into a difficult draft became something the rest of the package could benefit from. The tools could use the skill to prepare a new document or review an existing one, while I continued to check the substance and refine the instructions as we worked.

Every substantive change also got a dated record of what changed and which decision caused it. The documents themselves cite only other controlled documents, which is what a reviewer needs, while the full history of how each one was written stays complete beside them.

How the Package Kept Up With the Code

Traceability that is generated. Requirements, design claims and test cases live in a typed evidence graph, and the traceability matrix is generated from it. Our technical lead built the generator. My part was keeping the documents bound to it as they changed, so a requirement could not quietly lose its test.

Tests and test documents from the same source. I wrote the end-to-end journey designs for the automated Playwright suite and then turned them into the formal system test case documents. The test cases a validation reviewer reads and the automated tests the developers run come from one description, so they can’t drift apart.

Tool qualification as a script. Every tool in the build chain, from the test runner and linter to the PDF validator, database and hosting, has to be qualified for its intended use. I wrote a script that runs each qualification plan and writes a report with checksums of everything it touched, leaving the pass or fail field blank for a person to complete.

The Wider Process Work

The same work fed the client’s process documentation. I drafted an agentic software development lifecycle, iterated its lifecycle diagrams through several versions, and recorded the SOP working sessions. Those became input to the client’s SOP for AI-assisted development, which a colleague wrote, and to a concept for qualifying AI agents as tools.

The Result

We built the system inside the window, moved the controlled documents onto it, and retired the old one. The validation package was written alongside the build and handed to the client’s validation and QA leads, who own its review, execution and approval.