Directive (EU) 2024/2853 classifies software and artificial intelligence systems as “products” subject to strict liability. This fundamentally changes the risk profile for software houses: in the event of damage caused by a vulnerability, the manufacturer is liable even without fault, unless they can demonstrate that the defect was not discoverable given the state of technical knowledge at the time the product was placed on the market. A penetration test is the tool that produces this demonstration in a documented and defensible form.

This article is part of the mini-guide on Directive (EU) 2024/2853. For the general framework, consult the directive hub, or delve into the chapters on digital products and software liability.

When a software product is considered defective

The directive defines a product as defective if it does not provide the safety that the public is legitimately entitled to expect, also taking into account applicable cybersecurity requirements. For software, this means that an exploitable vulnerability—even if not yet exploited—may be sufficient to qualify the product as defective at the time of release.

A penetration test, performed before placing the product on the market and after every substantial modification, allows for documentation that the product’s defenses have been systematically verified and that known or reasonably discoverable vulnerabilities have been addressed. Without this documentation, the manufacturer is left to prove the absence of defects without concrete evidence.

Burden of proof and technical documentation

The directive introduces a mechanism for access to evidence that may compel the manufacturer to produce technical documentation during litigation. If that documentation does not exist or is incomplete, the defect is presumed. If, however, it exists and is up to date, the manufacturer can invoke the “state of technical knowledge” defense: at the time of release, the defect was not discoverable using available methods.

A dated penetration test report, signed by an independent team and correlated with the tested software versions, is the most direct form of this documentation. Automated analysis is not enough: manual and creative activity that simulates the behavior of a real attacker is required.

Liability when an external attacker intervenes

The directive maintains the manufacturer’s liability even when damage is caused jointly by a product defect and the action of a third party—for example, an attacker exploiting an unpatched vulnerability. The presence of a hacker in the causal chain does not exonerate the manufacturer.

The penetration test is specifically designed to simulate this scenario: if a vulnerability is discoverable using standard offensive techniques, the manufacturer cannot claim it was unforeseeable. Documenting that the test was performed—and that the vulnerabilities found were corrected—concretely reduces exposure to claims for physical or psychological damages or data destruction.

Penetration test or vulnerability assessment: which one does the directive require?

A vulnerability assessment identifies known vulnerabilities through automated scans and comparisons with public databases. It is useful for continuous monitoring but does not produce the evidence required by the directive in litigation.

The penetration test adds the manual and offensive component: an expert tester verifies whether vulnerabilities are actually exploitable in the specific context of the product, chains multiple weaknesses to simulate a real attack, and documents the path taken. It is this depth of analysis—and its traceability—that makes the penetration test the appropriate tool to meet the evidentiary requirements set by the directive.

For software products and web applications, ISGroup’s Web Application Penetration Testing covers this need using OSSTMM and OWASP methodologies, structured reports, and remediation support.

Useful resources

• Guide to Directive (EU) 2024/2853 — the complete framework on strict liability for products, including software and AI.
• Digital products and the EU directive — how the directive applies to software, apps, and integrated digital components.
• Software liability — analysis of specific obligations for software manufacturers.
• Vulnerability assessment and the EU directive — when VA is sufficient and when a penetration test is needed.

Frequently Asked Questions

• How often should a penetration test be performed to be useful for the purposes of the directive?
• The directive does not set a specific frequency, but the criterion is a “substantial modification”: every release that changes security-relevant functionality requires a new test. A test performed on a previous version does not cover vulnerabilities introduced subsequently.
• Must the penetration test be performed by an external party?
• The directive does not explicitly mandate this, but in litigation, a report produced by an independent team carries significantly more evidentiary weight than an internal analysis. The tester’s independence strengthens the credibility of the documentation.
• Does the penetration test also cover artificial intelligence systems?
• Yes, to the extent that the AI system is classified as a product under the directive. For AI components integrated into software, the test must include model-specific attack vectors—such as prompt injection or input manipulation—in addition to traditional application vulnerabilities.
• What happens if the penetration test identifies vulnerabilities that are not corrected before release?
• The report documents the vulnerabilities known at the time of release. If the manufacturer chooses to release the product anyway without fixing those vulnerabilities, the documentation becomes evidence against them: it proves that the defect was known and was not addressed.