Is healthcare AI high risk under the EU AI Act?

Five questions to help medical practices understand AI risk and the importance of a system's intended purpose.

The short answer: Not all healthcare AI is classified as high risk under the EU AI Act. The classification depends on what a system is intended to do and whether it meets the relevant legal criteria.

Note for Swiss practices: The EU AI Act does not automatically apply. Existing Swiss data protection and professional obligations remain relevant.

Imagine two patients contacting a medical practice.

The first wants to book a routine appointment. An AI assistant helps them navigate the booking process using the practice's existing rules.

The second describes their symptoms to an AI system designed to assess how urgently they need medical attention in an emergency healthcare setting.

Both systems use AI to interpret information. But their intended purposes, potential consequences and legal classifications can be very different.

The EU AI Act recognises these distinctions through its risk-based approach. Understanding them is an important step when considering AI for your practice.

1. What determines an AI system's risk classification?

The EU AI Act establishes different requirements for different categories of AI.

Some AI practices are prohibited. Certain applications are classified as high risk and are subject to additional requirements. Other systems may be subject to specific transparency obligations or other applicable legislation without being classified as high risk.

The starting point is the system's intended purpose: the use for which the provider has designed it.

For example, an assistant designed to answer administrative questions serves a different purpose from software designed to support a medical diagnosis.

A system's classification depends on whether its intended purpose and characteristics meet the criteria established in the legislation, not simply on the fact that it uses AI or operates in healthcare.

2. Which healthcare AI applications can be classified as high risk?

Two provisions of the EU AI Act are particularly relevant to healthcare.

  • Certain applications listed in Annex III
    These include AI systems intended for emergency healthcare patient triage, where the technology helps determine how urgently patients need medical attention.

  • Certain regulated products covered by Annex I
    These can include AI-enabled medical devices, such as qualifying diagnostic software, where the relevant legal conditions are met, including the applicable third-party conformity-assessment requirement.


Not every AI application used in healthcare, or every AI-enabled medical device, is automatically classified as high risk.

The specific purpose and legal classification of the individual system determine which requirements apply.

3. Is patient-facing AI automatically high risk?

No. An AI system does not become high risk simply because it communicates directly with patients.

An assistant designed to provide approved information about opening hours, services or appointment availability is not automatically classified as high risk.

However, being outside the high-risk category does not mean that an application is unregulated or risk-free. Other requirements, including relevant transparency and data-protection obligations, may still apply.

For medical practices, the important distinction is between the system's legal classification and the responsibilities that arise from using it in a healthcare environment.

We'll explore transparency, patient data and human involvement in more detail in Article 3.

4. Why does the system's intended purpose matter so much?

Two systems may look similar to a patient while performing fundamentally different tasks.

Consider an AI assistant that receives the message: “I need an appointment for a skin concern.”

An administrative assistant might identify the appointment request and help the patient navigate a booking process using predefined practice rules.

A system designed to evaluate the patient's symptoms and determine the urgency of medical attention has a different intended purpose.

The distinction is not necessarily how the patient interacts with the technology. It's what the AI is designed to determine and how its output is used.

This is why a product's intended purpose, rather than its appearance or general capabilities, is central to understanding its classification.

5. What if a practice uses AI beyond its intended purpose?

An AI system should be used within the purpose and conditions established for it.

For example, a product intended to answer administrative questions should not be assumed to be suitable for assessing symptoms or recommending treatment simply because it can generate responses to medical questions.

Using a system beyond its intended purpose can introduce new risks. In certain circumstances, substantially modifying a system or changing its intended purpose can also affect legal responsibilities under the EU AI Act.

Before extending an existing system to new tasks, establish whether the proposed use is supported by the provider and whether it changes the system's classification or applicable requirements.

For applications involving clinical decisions or potentially high-risk uses, seek appropriate legal and clinical advice.

What this means for your practice

When considering an AI application, start by describing its intended purpose in one sentence:

“This AI system is designed to [perform a specific task] using [relevant information] for [a defined purpose].”

Then consider whether the task is administrative or clinical, how the output will be used and whether the provider's documented intended purpose matches your planned application.

These questions provide a starting point for understanding the system's classification and identifying where further assessment may be necessary.

The important question isn't simply whether AI is being used in healthcare. It's what the technology is designed to do.

What comes next?

Legal risk classification is only one part of introducing AI into a medical practice. Even applications that are not classified as high risk need to be considered carefully when they interact with patients.

Next article: Can AI improve the patient experience without compromising trust?

References


This article provides general information and does not constitute legal or clinical advice. The classification of an individual AI system requires an assessment of its intended purpose and the applicable legal provisions.