A project can produce hundreds of test pages and still fail to demonstrate what happens when an interface loses a confirmation or a batch restarts after interruption. Assurance quality depends on the relevance and credibility of evidence, not document quantity. For GMP automation, the work starts with intended use, process risk and the functions that must perform reliably.
CSV and CSA discussions are useful when they improve that reasoning. They become misleading when used to promise less responsibility, automatic acceptance of supplier testing or exemption from applicable pharmaceutical requirements. This article connects specification, commissioning, testing and release through a practical engineering lifecycle.
Establish the applicable framework before choosing terminology
[REGULATORY REQUIREMENT] EU GMP Annex 11 addresses computerized systems, Chapter 4 documentation and Annex 15 qualification and validation. At the review date, the operative Annex 11 and Chapter 4 texts remained the 2011 versions. The 2025 consultation proposals were not operative replacements. In the United States, assess relevant drug CGMP requirements and the applicability of Part 11 to electronic records and signatures.
[GUIDANCE] FDA's final Computer Software Assurance guidance of February 2026 concerns software used in medical-device production or quality management systems. It supersedes the September 2025 final guidance. Its scope must not be expanded into a universal replacement for pharmaceutical GMP computerized-system validation. Risk-based thinking can inform engineering while the applicable requirements remain explicit.
[GUIDANCE / STANDARD] ISPE GAMP 5, second edition, provides industry good-practice guidance; ASTM E2500-25 addresses science- and risk-based specification, design and verification of manufacturing systems. Neither is legislation. Identify which frameworks the project adopts and how they support its actual intended use.
Define intended use in operational terms
Describe what the system controls, records, calculates and presents, and which decisions depend on it. Include users, operating modes, equipment boundaries, interfaces and relevant failure conditions. “SCADA for production” is too broad to support an assurance strategy. The functions controlling a process phase and those displaying maintenance statistics may require different evidence.
Identify the consequences of failure before selecting tests. Consider product quality, patient safety, data integrity and operational continuity, with attention to dependencies and detectability. A read-only display may still influence a consequential operator decision; a small interface may carry the only record of material consumption.
[QRM] Use risk assessment to focus knowledge and effort. Record assumptions and uncertainty, involve appropriate subject-matter expertise, and revisit the assessment when design or evidence changes. A numerical risk score is a decision aid, not proof that a function works or permission to ignore an applicable requirement.
Write requirements that can be verified
A requirement should describe the intended outcome, relevant conditions and acceptance basis. Separate user need from an implementation preference unless a genuine constraint requires a specific design. Link consequential requirements to their rationale and risk so that reviewers understand why the verification matters.
For example, require the system to preserve and reconcile the defined production transaction after an assessed communication interruption. Then specify the expected state, record completeness and duplicate behaviour. “The interface shall be reliable” cannot guide a meaningful acceptance decision.
Maintain traceability at a useful level. Connect requirements, design decisions, risks, verification evidence and unresolved issues without building a matrix that merely repeats document titles. Traceability should help answer whether each consequential intended use is supported by evidence and whether changes have been assessed.
Assess the supplier and the actual evidence available
Evaluate supplier competence, development and configuration practices, testing, defect management, cybersecurity support and lifecycle arrangements in proportion to the system. Distinguish a standard product, configured functionality and project-specific code. The assurance approach should reflect what is known about each and how it affects intended use.
Request evidence relevant to the proposed version and configuration. A generic quality certificate or demonstration cannot establish that a site-specific recipe, interface or report is correct. Determine whether supplier records identify test conditions, expected outcomes, actual results, deviations and configuration identity sufficiently for their intended use.
Plan evidence reuse early with the supplier. Agree responsibilities, access, witness points and record expectations before FAT. Reconstructing missing test context after delivery can consume more effort than performing the work properly once. Supplier evidence remains subject to assessment; it should neither be accepted automatically nor repeated automatically without a reason.
Connect commissioning, FAT, SAT and qualification
Commissioning establishes engineering functionality and readiness. FAT evaluates agreed functions before shipment or site deployment; SAT addresses the installed environment and site interfaces. Qualification and validation activities establish the required documented assurance for the intended use within the applicable framework. These activities can share evidence when its quality, scope and conditions are suitable.
Define which evidence can be reused and which site-specific conditions need additional verification. Shipping, installation, network configuration, identity services and connected equipment can alter behaviour. A successful FAT on a simulated interface does not prove the delivered site transaction without assessment of the differences.
[GEP] Avoid both extremes: repeating every test merely because its heading changes, or declaring commissioning equivalent to qualification without evaluating evidence. Use a planned acceptance rationale that identifies what was demonstrated, under which configuration, and what remains to be established at the next stage.
Select testing methods according to the question
| Method | Useful application | Evidence considerations |
|---|---|---|
| Scripted verification | Defined consequential sequence or acceptance criterion | Clear prerequisites, expected outcomes and actual observations |
| Exploratory or scenario-based testing | Interactions, usability and behaviour not fully captured by a fixed path | Purpose, scope, tester, configuration, observations and issues |
| Automated testing | Repeatable calculations, mappings or regression checks | Suitability of the test mechanism, controlled inputs and interpretable results |
| Engineering inspection | Architecture, configuration and design constraints | Competent review, defined criteria and disposition of findings |
Unscripted does not mean undocumented or without an objective. Automated does not mean intrinsically trustworthy. Choose methods that can produce credible evidence for the assessed function, and combine them where necessary. There is no universal GMP percentage of scripted tests or mandatory number of test cases.
Challenge critical functions and abnormal conditions
Normal execution is only part of the evidence. Include invalid inputs, unauthorized actions, missing prerequisites, interrupted communications and recovery where consequential. Examine boundaries and state transitions, because defects often occur when a process moves between otherwise well-tested states.
For control functions, verify the relationship between command, permissive, output and feedback. For records, verify completeness, attribution, time and retrieval. For interfaces, verify confirmation, duplicate handling and reconciliation. For recipe execution, verify version identity, permitted adjustments and restart behaviour.
Plan failure simulations so they are controlled and safe. Use representative environments or approved methods that do not expose live production to unnecessary hazards. Explain any limitations of simulation and how remaining uncertainty is addressed. A test should not create a new uncontrolled process risk merely to demonstrate robustness.
Make automated evidence reviewable
Automated checks can efficiently compare configuration, verify calculations or exercise repeatable interface scenarios. Define the expected result independently of the implementation where practical. A test that copies the same logic as the application can reproduce the same defect and still pass.
Control test data and identify the software and configuration under test. Retain outputs that support interpretation, including failures and relevant logs. Assess the suitability of tools used to generate or evaluate evidence according to their role and risk. A green dashboard without a clear test population and execution identity is weak assurance.
Use regression testing to address credible change impact. Select affected functions and dependencies based on the change assessment; do not confuse a large unchanged test suite with complete coverage. Where a test mechanism changes, assess whether its previous and future results remain comparable.
Control prerequisites and test data
Test results are interpretable only when the starting conditions are known. Identify equipment state, configuration, user role, connected services and relevant data before execution. A failed prerequisite can invalidate a later observation even if the final screen resembles the expected result. Record deviations from planned conditions and assess their effect on evidence.
Use test data that represent meaningful process and record conditions, including invalid, missing and boundary values where relevant. Protect production data if they are used in a controlled test environment. Define how test records are identified and prevented from contaminating operational records during deployment.
Assess environment differences explicitly. A simulator may establish control logic while omitting network latency, instrument behaviour or infrastructure dependencies. A site test may establish integration without covering every internal calculation. State what each environment can demonstrate and combine evidence to address the intended use. This makes limitations visible and helps avoid unjustified conclusions from otherwise successful tests.
Manage defects as engineering information
Record observed failures and unexpected behaviour with enough context to reproduce and assess them. Separate the symptom, likely cause, consequence and disposition. A defect categorized as low severity still needs a justified resolution or acceptance; a category label alone does not close the issue.
Assess effects on other functions and on evidence already collected. A library correction or time-service change can invalidate assumptions beyond the original failed test. Define the necessary re-verification and retain the relationship between the defect, correction, configuration and subsequent evidence.
For accepted residual issues, identify the rationale, operational controls, owner and closure conditions where applicable. Ensure users know limitations relevant to their tasks. Do not hide unresolved defects in a test appendix while presenting the release summary as an unconditional statement of readiness.
Worked example: interrupted recipe transfer
An illustrative project transfers an approved recipe from a supervisory application to a controller. The risk assessment identifies incorrect version selection, partial transfer and ambiguous confirmation as consequential failures. The team defines a requirement that execution uses the confirmed complete recipe instance and preserves its identity.
Supplier FAT evidence demonstrates normal transfer using the proposed software version. Site testing adds identity integration, the installed communication path and an interrupted transfer. The controller must not execute an incomplete or unconfirmed recipe; the interface must show the actual outcome and support the approved recovery workflow.
An exploratory session then examines operator recovery from the displayed error. It reveals that two similar commands have unclear meanings, leading to a design correction. Scripted verification confirms the critical state behaviour, while the scenario session contributes usability evidence. The release assessment uses both, with their configuration identities and defect dispositions, instead of treating one testing style as inherently superior.
Define release as an accountable decision
A release package should explain the intended use, applicable requirements, assessed risks, evidence coverage and remaining limitations. Confirm that the installed configuration matches the assessed baseline and that necessary procedures, training, maintenance and support are ready. An application can pass functional tests yet remain unsuitable for use if recovery or account administration has no operational owner.
Review deviations and unresolved issues according to their consequences. Identify which prevent release and which can be accepted with justified controls. The decision belongs to the site's authorized roles within its quality system, supported by engineering and process expertise.
- Consequential requirements have credible, traceable evidence.
- Site-specific interfaces and dependencies are verified.
- Defects and test limitations have explicit dispositions.
- Records, backup and recovery arrangements are usable.
- Users and support teams understand the released configuration.
- Change and ongoing review responsibilities are assigned.
Maintain assurance after initial release
Changes, incidents and operational experience can alter the assurance basis. Assess updates to controller code, recipes, interfaces, infrastructure, security controls and reports according to their effects. Preserve configuration identity and verify affected functions after implementation.
Periodic review should consider performance, incidents, changes, access, support status and continued fitness for intended use. Derive its scope and timing through the applicable framework and site assessment; do not invent a universal annual interval. Retirement also needs assurance that retained records remain accessible and that dependencies are removed deliberately.
Use the automation URS article to establish verifiable requirements, and legacy migration for major lifecycle changes. The Automation & Digital Systems hub connects assurance with the wider engineering scope.
Primary references and status
Reviewed 23 September 2026: European Commission, EudraLex Volume 4; 21 CFR Part 11; FDA Computer Software Assurance, final February 2026; ICH Q9(R1); ISPE GAMP 5, second edition; ASTM E2500-25. Publisher information establishes edition and scope; proprietary methods and tables are not reproduced. The testing matrix and example are original GuideGxP recommendations.
