AI Act Article 15: Accuracy, Robustness and Cybersecurity — Security & GDPR article by Rui Codex

AI Act Article 15 requires high-risk AI systems to reach an appropriate level of accuracy, robustness and cybersecurity, and to keep that performance stable over their lifecycle. Providers must declare accuracy metrics in the instructions for use, build fallbacks for faults, control feedback loops in systems that keep learning, and defend against data poisoning and adversarial inputs.

Key takeaways

  • Article 15 treats accuracy, robustness and cybersecurity as lifecycle requirements, not one-time test results.
  • Accuracy levels and metrics must be declared in the instructions for use, so they can be checked against real behaviour.
  • Robustness depends on fallbacks, monitoring and controlled handling of systems that keep learning after release.
  • Cybersecurity under Article 15 covers attacks on training data, pre-trained components, inputs and the model itself.

What does Article 15 require?

Article 15 sets three performance requirements for high-risk AI systems: an appropriate level of accuracy, robustness and cybersecurity, consistent across the whole lifecycle. The text is available in full on EUR-Lex, Regulation (EU) 2024/1689, and it should be read alongside the rest of the Regulation, especially the instructions-for-use and risk management provisions.

The article does not prescribe a single metric or threshold. It asks providers to choose levels that fit the intended purpose and to declare them. That makes the requirement a documentation and engineering task as much as a testing task.

This article is general information and not legal or tax advice. Whether a given system is high-risk, and which dates apply to it, must be checked against the official text and with qualified counsel.

Rui Codex offers technical implementation support for teams building AI systems, and the AI Act technical measures service page describes that work. Rui Codex does not provide legal, tax or compliance advice, audits or certification.

How do you measure accuracy in practice?

Start by defining accuracy for the specific task, because a single score rarely describes a system's behaviour. A classifier used for invoice routing needs different metrics from a model that ranks candidates for a credit decision. Choose metrics that reflect the costs of each error type, then fix them before measuring.

Build a test set that reflects production inputs, not only clean benchmark data. Include edge cases, rare classes and the input quality your users actually produce. Record the dataset version, the evaluation date and the model version together, so results can be reproduced later.

Tip: Write the accuracy target and the metrics into the instructions for use before the first evaluation run, so the threshold is not adjusted to fit the results.

Where accuracy measurements fail

Aggregate scores hide subgroup failures. A model can look accurate overall while performing poorly for a language, region or input format that is a small part of the test set. Break results down by the groups and conditions that matter for the use case.

Offline accuracy also drifts. Input distributions change after deployment, so a score taken at launch describes a system that no longer exists. Plan periodic re-evaluation with a fixed sample and a documented trigger for retraining or review.

How do you design for robustness?

Robustness means the system keeps behaving sensibly when something goes wrong: malformed input, a failed upstream service, a missing field or a model output outside the expected range. Article 15 refers to resilience against errors, faults and inconsistencies, and to technical redundancy such as backup or fail-safe plans.

The practical starting point is a failure list. For each component, write down what happens when it returns garbage, times out or stops responding, and decide the fallback: a rule-based path, a human review queue, or a safe refusal.

  • Validate inputs before they reach the model, and reject or flag values outside known ranges.
  • Set confidence or consistency checks on outputs, and route low-confidence cases to a person.
  • Log inputs, outputs and fallback events with enough detail to investigate a failure later.
  • Test the fallback path as often as the main path, because it is rarely exercised in normal operation.

Each fallback has a trade-off. Human review improves safety but adds latency and cost. A rule-based fallback is predictable but may be less accurate than the model, and teams must document that difference.

Systems that keep learning

Article 15 also addresses systems that continue to learn after they are placed on the market. Feedback loops can amplify biased outputs when the system's own decisions become its next training data. Controls include freezing the model between approved releases, reviewing new training batches, and monitoring outcome distributions over time.

Tip: If the system learns from user corrections, store those corrections separately and approve them before they reach training, instead of letting them flow in automatically.

Which attacks must cybersecurity measures address?

Cybersecurity under Article 15 is about resilience against unauthorised third parties who try to alter the system's use, outputs or performance by exploiting vulnerabilities. The Regulation names examples of attacks that technical measures should address, including data poisoning, model poisoning, adversarial examples, confidentiality attacks and model flaws.

Data poisoning targets the training set, for instance by injecting records that shift decisions. Model poisoning targets pre-trained components reused in the build, so their provenance matters. Adversarial examples are inputs crafted to cause a wrong output, and confidentiality attacks try to extract training data or model details.

  1. Control the provenance of training data and pre-trained models, with checksums and a recorded source for each artefact.
  2. Restrict who can change training pipelines, prompts, thresholds and deployment configurations.
  3. Rate-limit and monitor inference endpoints to detect probing patterns.
  4. Keep an incident response plan that covers model rollback, not only server recovery.

No single control covers all these attacks. Input filtering can reduce adversarial inputs but will not stop a poisoned training set. Access control protects pipelines but does not detect a subtly manipulated dataset that passed review. Layering controls and testing each layer is the realistic approach.

Teams that already run secure cloud and development practices can extend them to the AI layer. The security and compliance engineering articles on the Rui Codex blog cover the baseline controls that AI pipelines inherit.

Hypothetical examples of Article 15 gaps

Here are two hypothetical scenarios:

In the first, a team evaluates a document classifier on a clean dataset and reports one overall score in its documentation. The score is never broken down by document type, so a weak result on scanned handwritten forms goes unnoticed until users complain. The fix is a subgroup table and a documented limit for unsupported formats.

In the second, a team connects an AI model to an automated approval step. When the upstream data service fails, the model receives empty fields and approves cases by default. A validation rule and a manual review fallback would have stopped the bad decisions. Testing the failure path before launch would have revealed the gap.

Neither scenario describes a real organisation. They show how Article 15 requirements translate into ordinary engineering decisions, and how gaps appear when those decisions are skipped.

Gaps like these are often found during integration work, where data flows between systems that were never designed to fail together. Rui Codex works on system integrations, and the failure handling between connected systems is usually where robustness is tested first.

Frequently Asked Questions

Does Article 15 apply to every AI system?

Article 15 applies to high-risk AI systems as defined in the AI Act, not to every AI system. Whether a system is high-risk depends on its intended purpose and the categories in the Regulation. Check the official text on EUR-Lex and seek qualified advice for a classification decision.

Must accuracy be a single number?

No, Article 15 does not require a single number. Providers should declare relevant accuracy metrics and levels that match the intended purpose. Metrics should be broken down by important conditions so that users can understand where the system performs well and where it does not.

What is the difference between robustness and accuracy?

Accuracy describes how often outputs are correct under expected conditions. Robustness describes how the system behaves when conditions are unexpected, such as faulty inputs or failed components. A system can be accurate on clean data and still fail badly when a dependency breaks.

Is a human review step enough to meet the cybersecurity requirement?

A human review step is not enough on its own. Cybersecurity under Article 15 requires technical measures against attacks such as data poisoning and adversarial inputs. Human review can be one layer, but it should sit alongside access control, provenance checks and monitoring.

How often should performance be re-checked?

Re-check performance whenever the model, data or use context changes, and at regular intervals defined in your plan. Use a fixed evaluation sample so results are comparable over time. Record a documented trigger that starts a review when performance falls below the declared level.

Conclusion

Article 15 turns accuracy, robustness and cybersecurity into lifecycle obligations that touch testing, documentation, engineering and operations. The practical work is to declare metrics, design fallbacks, control learning loops and defend the data and model pipeline. Teams that start with a failure list and a clear evaluation plan will find the requirements easier to meet. To discuss your project, request a project consultation at the Rui Codex contact page or email info@ruicodex.com.

Sources

Tags: AI Act Article 15 AI compliance robustness accuracy cybersecurity high-risk AI AI governance

Need Help Implementing This?

Our team can help you put these insights into practice. From AI automation to custom software development, we build solutions that deliver real results.

Send a project request