EU AI Act Security Testing: What Article 9 Requires

The EU AI Act is not another data protection framework. It is a product safety regulation for AI, structured around the model that governs medical devices, machinery, and toys in the European market. Providers of high risk AI systems must complete a conformity assessment before placing the system on the market, maintain a documented risk management system throughout the lifecycle, and be able to demonstrate compliance to notified bodies and market surveillance authorities.

For security teams, this reframes AI security testing from best practice to legal obligation. Article 9 explicitly requires it. Article 15 explicitly names the categories of attacks the system must be resilient against. Notified bodies will read the security testing evidence during conformity assessment. Market surveillance authorities can request it post-market. What was optional in 2024 is legally required in 2026.

What EU AI Act Article 9 Actually Requires

Article 9 defines the risk management system as a continuous iterative process, not a one-off assessment. Providers must plan, document, and run this process across the entire lifecycle of the high risk AI system. The Act sets out four operative steps.

Article 9(2)(a). Identification and analysis of known and reasonably foreseeable risks. Providers must identify risks to health, safety, and fundamental rights that the AI system may pose when used in accordance with its intended purpose. For AI systems, this covers both traditional cybersecurity risks (data breach, unauthorised access) and AI-specific risks (adversarial manipulation, model integrity failure, discriminatory output patterns).

Article 9(2)(b). Estimation and evaluation of risks. Identified risks must be estimated and evaluated in terms of severity, likelihood, and impact. This is where quantitative security assessment feeds directly into the risk register: a demonstrated adversarial vulnerability, a demonstrated prompt injection path, a demonstrated data poisoning susceptibility all become documented risk entries.

Article 9(2)(c). Evaluation of other risks emerging from post-market monitoring. Post-market surveillance data feeds back into the risk management system. New attack techniques, disclosed vulnerabilities, and observed misuse patterns must be evaluated and integrated.

Article 9(2)(d). Adoption of appropriate risk management measures. Providers must adopt technical and organisational measures to eliminate or reduce risks. Measures must balance mitigation with the intended purpose and the state of the art.

Article 9(6). Testing throughout development and before market. This is the operative clause for security testing. Testing must occur throughout the development process and, at minimum, before the AI system is placed on the market or put into service.

Article 9(8). Testing against predefined metrics. Testing must be performed against prior defined metrics and probabilistic thresholds appropriate to the intended purpose. Ad hoc testing without documented metrics does not satisfy the requirement.

Article 9(9). Testing in real world conditions. For certain high risk AI systems, testing in real world conditions is permitted subject to specific safeguards under Article 60. This is relevant for AI systems where lab testing cannot fully replicate deployment.

Vulnerable groups. Article 9(9) also requires specific consideration of whether the system is likely to adversely affect persons under 18 or other vulnerable groups. Security testing must account for these populations where the system's intended purpose touches them.

What Counts as High Risk AI Under the Act

Not every AI system falls under Article 9. The Act applies risk management obligations to systems classified as high risk under Article 6, which points to two categories.

Annex I high risk AI systems. AI used as a safety component of, or itself constituting, a product covered by Union harmonisation legislation listed in Annex I. Machinery, medical devices, in vitro diagnostics, toys, radio equipment, marine equipment, civil aviation, and others. The August 2, 2026 deadline applies to a subset of these; the full Annex I obligation activates on August 2, 2027.

Annex III high risk AI systems. AI systems used in specific enumerated contexts including: biometric identification (excluding verification), critical infrastructure, education and vocational training, employment and worker management, access to essential services (public benefits, credit scoring, insurance underwriting, emergency response dispatch), law enforcement, migration and border control, and administration of justice and democratic processes.

The classification exercise is itself a compliance task. Recent enterprise research from appliedAI covering 106 enterprise AI systems found 18% clearly classified as high risk and 40% falling into unclear classification zones, most commonly around critical infrastructure use, employment decisions, and product safety borderlines. Misclassifying an in-scope Annex III system as out of scope is the cheapest way to walk into a Tier 2 penalty of up to EUR 15 million or 3% of turnover.

Practical classification test. Ask three questions in sequence. First, does the system fall under one of the eight enumerated Annex III categories? Second, is the system used in the EU market (regardless of where the provider is established)? Third, does the intended purpose materially influence a decision affecting individuals? Two or more yes answers make it likely high risk and require Article 9 compliance.

Security Testing Categories That Satisfy Article 9

The mapping above breaks Article 9's requirements into specific testing categories. Each category corresponds to a distinct assessment discipline and produces distinct evidence.

AI threat modeling. Systematic identification of attack surfaces, data flows, trust boundaries, and adversarial motivations. Threat modeling produces the input list for subsequent security testing and creates the documented risk analysis Article 9(2)(a) requires. Frameworks such as MITRE ATLAS provide standardised AI threat taxonomies that map cleanly to Article 9 documentation requirements.

Adversarial robustness testing. Testing the AI system's behaviour under adversarial inputs. For predictive models, this includes evasion attacks, transfer attacks, and physical adversarial examples. Article 15(5) names this category explicitly as "inputs designed to cause the AI model to make a mistake".

Prompt injection and jailbreak assessment. For LLM based systems, testing whether user inputs can override system instructions, extract system prompts, bypass safety controls, or execute unintended actions. Direct prompt injection, indirect prompt injection via retrieved content, and jailbreak techniques all fall under Article 15(5) resilience requirements. Our prompt injection testing guide covers the full methodology.

Data poisoning simulation. Testing whether the training data pipeline is resilient to contamination. Article 15(5) explicitly names training data manipulation as a required control category. For providers using continuous learning or fine-tuning pipelines, this is a live risk that Article 9 documentation must address.

Model poisoning and supply chain testing. Verifying integrity of pre-trained components, base models, and third party model artefacts. Article 15(5) names pre-trained components used in training as a required protection category.

Confidentiality attack testing. Testing membership inference, model inversion, training data extraction, and other attacks that violate confidentiality of training data or system prompts. Article 15(5) names confidentiality attacks explicitly.

AI red teaming. Adversarial simulation exercises against the AI system in production-like conditions. Red teaming synthesises the categories above into a scenario-based assessment and typically produces the highest-signal evidence for Article 9 documentation. Our LLM red teaming approach walks through the methodology.

Traditional application security testing. AI systems are software systems. Web application penetration testing, API penetration testing, and infrastructure testing remain necessary. Article 9 does not replace conventional security testing; it adds AI-specific requirements on top.

Compliance Timeline and Deadlines by System Type

The Act's phased implementation means different obligations apply to different system categories on different dates. Compliance planning depends on correctly identifying which timeline applies.

What Notified Bodies and Auditors Will Actually Check

Conformity assessment under Article 43 of the EU AI Act happens through one of two paths. Internal control (self-assessment) is permitted for most Annex III systems. Third party assessment by a notified body is required for biometric identification systems and some others. In both paths, the security testing evidence lives in the technical documentation package required by Article 11 and Annex IV.

Documented risk register. Every identified risk from threat modeling and vulnerability assessment recorded with severity, likelihood, mitigation, and residual risk assessment. Notified bodies read the risk register as the entry point to the whole package.

Testing evidence with named metrics. Article 9(8) requires testing against predefined metrics. Evidence must show what was tested, what metrics applied, what pass thresholds were set, and what results were achieved. Ad hoc pentest reports without documented metrics do not satisfy this.

Adversarial testing coverage. Article 15(5) categories must each be addressed: adversarial examples, data poisoning, model poisoning, confidentiality attacks, model flaws. Coverage of all five categories with test evidence is expected. Absence of any category is a documented gap the notified body will question.

Post-market monitoring plan. How Article 9(2)(c) risks will be identified and evaluated post-deployment. Plans that only reference generic security monitoring without AI-specific detection do not satisfy the AI-specific risk framing of the Act.

Named test provider credentials. The Act does not require testing by any specific type of provider, but notified bodies commonly ask who performed the testing, what their credentials are, and whether they are independent from the AI development team. Independent third party testing is generally the strongest evidence category.

Traceability from test to control. Every mitigation measure claimed under Article 9(2)(d) should trace back to specific test findings. If a control claim has no corresponding test evidence, or if a finding has no corresponding remediation, the notified body will flag it.

Framework alignment where relevant. Where the provider operates under ISO 42001, NIST AI RMF, or SR 11-7 (US financial services), Article 9 evidence can be integrated into existing frameworks. The Act explicitly permits this in Article 9(10).

How SecurityWall Supports Article 9 Compliance

SecurityWall's AI and LLM security testing service is structured for the evidence categories Article 9 and Article 15 explicitly require. Each engagement produces documented, framework-mapped output calibrated to satisfy notified body review and internal conformity assessment.

Threat modeling to Article 9(2)(a). Systematic threat modeling using MITRE ATLAS and OWASP AI Security taxonomies, producing a documented risk register that plugs directly into your Article 9 documentation.

AI red teaming to Article 15(5). Adversarial simulation covering all five named attack categories: data poisoning simulation, model poisoning verification, adversarial example generation, confidentiality attack testing, and model flaw discovery. Our LLM red teaming methodology walks through the specific techniques applied.

Prompt injection and jailbreak testing for LLM systems. Direct and indirect prompt injection assessment, jailbreak resistance testing, system prompt extraction attempts, and safety control bypass verification.

Traditional pentest for the underlying software stack. Web application, API, and infrastructure testing against the systems supporting the AI model. Article 9 does not remove the requirement for conventional cybersecurity.

Post-market monitoring design. Recommendations for AI-specific security monitoring that feed the Article 9(2)(c) requirement without duplicating existing SOC infrastructure.

Framework alignment. Reports mapped to Article 9 sub-paragraphs, Article 15(5) attack categories, and where relevant, ISO 42001 controls or ISO 42001 AI maturity assessment outputs.

Concrete pricing is quoted in the 15 minute scoping meeting because it depends on model architecture, scope, and framework coverage. What is standard across every engagement: independent testing, documented metrics, five-category adversarial coverage, and framework-mapped output.

Frequently Asked Questions

Does Article 9 apply to my AI system? Article 9 applies to providers of high risk AI systems classified under Article 6. This includes systems used as safety components of Annex I products (medical devices, machinery, etc.) and systems used in Annex III enumerated contexts (biometric identification excluding verification, critical infrastructure, education, employment, essential services, law enforcement, migration, justice). Deployers of high risk AI have separate obligations under Article 26. If your AI touches employment decisions, credit scoring, insurance underwriting, or education outcomes, assume Article 9 applies until you can defensibly rule it out.

Is a penetration test enough to satisfy Article 9? A single penetration test alone is not enough. Article 9 requires a continuous iterative process across the lifecycle including threat modeling, risk analysis, testing against defined metrics, adversarial category coverage (Article 15(5)), and post-market monitoring. A pentest is one component of the evidence package, not the whole package. That said, a well-scoped AI penetration test with framework mapping to Article 9 sub-paragraphs and Article 15(5) categories is the strongest single evidence artefact you can produce.

What are the penalties for non-compliance? Prohibited AI violations reach EUR 35 million or 7% of global annual turnover, whichever is higher. Non-compliance with high risk AI obligations (including Article 9) reaches EUR 15 million or 3%. Supplying incorrect or misleading information to authorities reaches EUR 7.5 million or 1%. These are among the largest regulatory penalties in EU law, exceeding GDPR headline maximums.

When exactly is the deadline? For high risk AI systems listed in Annex III, the binding enforcement date is 2 August 2026. Prohibited AI has been enforceable since 2 February 2025, and general purpose AI models since 2 August 2025. Product-embedded high risk AI (Annex I) reaches enforcement on 2 August 2027. A European Commission proposal in November 2025 to defer certain deadlines to 2027 was not enacted; treat August 2026 as operative.

How does Article 9 interact with GDPR, DORA, and NIS2? Article 9 does not replace existing frameworks; it adds AI-specific obligations. If personal data is processed by the AI system, GDPR obligations apply in parallel including Data Protection Impact Assessment under Article 35. If the AI system is deployed within a financial services provider, DORA ICT risk management obligations apply. If the deployer is an NIS2 essential or important entity, NIS2 penetration testing requirements apply to the underlying systems. Article 9 documentation can be integrated with existing frameworks under Article 9(10).

Who performs the conformity assessment? For most Annex III high risk AI systems, providers self-assess through internal control (Article 43(2)). For biometric identification systems and some others, third party assessment by a notified body is required (Article 43(1)). In both paths, the security testing evidence lives in the technical documentation required by Article 11 and Annex IV. Notified bodies are designated by EU member states and listed in the NANDO database.

Related reading:

• LLM Penetration Testing Guide
• LLM Penetration Testing: Scope, Cost, Providers
• LLM Security and AI Red Teaming
• OWASP Top 10 for Agentic AI 2026
• Prompt Injection Testing Guide
• AI Maturity Assessment for ISO 42001
• ISO 42001 AI Compliance
• NIS2 Penetration Testing Requirements