What are we going to talk about in this article? For the curious, here is the definition of an AI system: “A machine-based system designed to operate with varying levels of autonomy, that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers from the input it receives how to generate outputs such as predictions, content, recommendations or decisions that can influence physical or virtual environments.”
The European Union is adopting the world's first rules on artificial intelligence (the AI Act). The AI Act defines the framework that developers, providers, distributors, importers and even certain users (in particular you, entrepreneurs using an AI system in your business) must follow.
The AI Act clearly says "no, thank you" to systems that could place you on a socially undesirable list, manipulate you or monitor you without authorisation. It lays down rules for high-risk systems, including transparency obligations.
Definition of an AI system
The AI Act classifies AI systems into categories according to their level of risk
The challenge of categorisation
“If classifying AI into risk categories were a competition, it would be an Olympic sport.”
Correctly classifying individual systems into a risk category will be crucial in practice, but also frequently problematic. Deciding which box to place your AI creation in is your responsibility as the provider. Non-compliance of an AI system with any of the provisions can in principle result in a fine of up to EUR 15,000,000, or, in the case of a company, up to 3% of its total worldwide annual turnover for the preceding financial year.
Helpfully, within 18 months of the Act entering into force the Commission will issue guidelines with examples of which activities are considered high-risk, which should make the Act easier to interpret.
Obligations of high-risk developers
Developers of high-risk AI systems must, for example, establish a risk management system (= a continuous, iterative process planned and run throughout the entire lifecycle of a high-risk AI system, requiring regular systematic review and updating), ensure data governance, including model evaluation, prepare technical documentation (before placing the system on the market and keep it updated), design systems for event logging (so-called logs), provide instructions for use, enable human oversight, and ensure appropriate levels of accuracy, robustness and cybersecurity. High-risk AI systems should bear the CE marking so that they can move freely within the internal market.
What is model evaluation? It is the testing of models with regard to their tendency and ability to manipulate and deceive.
What should this look like for high-risk AI systems? The whole process should take place in the following steps:
Registration obligation under Article 51
- At the European level - the obligation to register at the European level applies to the systems listed in Annex III, with the exception of point 2, which is registered at the national level, and even where the provider has concluded that its system is not high-risk. Deployers that are public authorities or agencies, or that act on their behalf, will also register in this European database, in principle before they start using the systems. Records of systems used for law enforcement, migration, asylum and border-control management will not be public.
- At the national level - high-risk systems and their safety components in the area of critical infrastructure (digital infrastructure, road transport, and the supply of water, gas, heating and electricity) / point 2 of Annex III /.
Obligations of developers of general-purpose artificial intelligence (GPAI)
A general-purpose AI model is “an AI model, including one trained on a large amount of data using self-supervision at scale, that displays significant generality and is capable of competently performing a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications.” This does not apply to AI models that are used before being placed on the market for research, development and prototyping activities.
Developers of general-purpose AI models must provide technical documentation, instructions for use, comply with the Copyright Directive and publish a summary of the content used for training.
Natural persons interacting directly with AI must be informed that they are communicating with AI, unless this is obvious from the circumstances and context. AI outputs (images, videos, audio) must be marked in a machine-readable format; this does not apply to an assistive function for standard editing. Where AI content forms part of an evidently artistic, creative, satirical or fictional work or programme, users who are not acting in the course of personal, non-professional activity must appropriately disclose that it is such an AI work.
If a GPAI model is assessed as posing a systemic risk, which we expect for systems such as ChatGPT and Gemini, it must also carry out model evaluations, monitor and report serious incidents and ensure cybersecurity. A list of GPAI models with systemic risk will be published.
When do we talk about systemic risk? When a system has a high-impact capability or the Commission has so decided. Where the value of FLOPs is greater than 10^25, systemic risk is presumed and the provider bears the burden of proving otherwise. These values and indicators will change in practice through Commission acts.
Are you integrating artificial intelligence into your system or product?
If you develop your own system into which you have integrated an AI system, then under the Act you will be considered a “downstream provider”, and some provider obligations will also apply to you, depending on the level of risk of your system.
If you integrate a GPAI system into your system and it does not qualify as high-risk, as a downstream provider you must still meet the general requirements for transparency, security and data protection, but you will not have to comply with the specific requirements of the Regulation intended for high-risk systems.
If, however, the integration changes the intended purpose of the system and therefore its classification to a high-risk system, you will be considered a provider of a high-risk system with all the related obligations.
You are also considered a (new) provider if you place your name or trade mark on a high-risk system or make a substantial modification to a high-risk system. The existing provider should cooperate with you and make information available in order to achieve a conformity assessment. The original provider may, however, expressly exclude by contract the conversion of its system into a high-risk system.
A product manufacturer with an AI safety component is also considered a provider of a high-risk system if the AI system is placed on the market together with the product, or put into service under the name or trade mark of the product manufacturer.
At least the transparency information
As a downstream provider you should have at least the GPAI transparency information available: information on the characteristics and limitations of the original GPAI (with an exception for open source under the conditions set out below), in accordance with Annex IXb – a general description of the model, including its architecture and number of parameters, the manner and format of inputs and outputs, and a description of the components of the model and the process for its development, including the technical means (e.g. instructions for use, infrastructure, tools) needed to integrate the general-purpose AI model into AI systems, the modality (e.g. text, image, etc.) and format of inputs and outputs and their maximum size (e.g. the length of the context window, etc.), as well as information on the data used for training, testing and validation.
What are the open source exemptions under the AI Act?
The obligations laid down in this Regulation do not apply to AI systems released under free and open source licences, provided they are not placed on the market or put into service as high-risk AI systems or AI systems covered by Title II (prohibited systems) and Title IV (transparency obligations).
This rule supports open development and innovation in the field of artificial intelligence by allowing developers and organisations to share AI systems under open licences without having to meet the regulatory requirements specified for high-risk systems.
Even where the AI Act applies to open source, certain exemptions are also provided, for example from the obligation to appoint an authorised representative. If the open source model does not pose a systemic risk and has published the relevant information enabling access to the model, its use, modification and distribution, its parameters, including weights, information on the model's architecture and on the use of the model, no representative needs to be appointed.
Logically, there is also no obligation to conclude a written agreement between the provider of a high-risk system and a third party with integrated open source AI.
Here, however, I must point out that classifying a system as open source need not be entirely without complications. Open source does not mean that a licence to use cannot be granted for a fee. Free and open source (so-called FOSS) means free, open-source software. The freedom lies in the user's ability to use the software for any purpose, to modify it and to distribute it, even for a fee, which is made possible precisely by access to the source code. The definitions of "free" and "open" software, commercial use, and the role of licences regularly give rise to debate in practice.
At this point I must also recall that ChatGPT is definitely not open source. OpenAI only aspired to open source in its early days but gradually changed its approach, something for which one of its co-founders, Elon Musk, criticises it today.
By when should you be prepared?
Get to know the deadlines in the timeline:
AI law is our specialisation. We monitor the development of the AI Act, the related code of conduct, practical guidance, model documentation and Commission acts on your behalf. If you want to be AI compliant, we are here for you.
What can we help you with, for example?
- Setting up internal rules for the use of an AI system
- An AI-use policy for employees
- Employee training (meeting the AI literacy obligation under Article 4g of the AI Act)
- Negotiating licence agreements
- Setting up general terms and conditions for the use of your AI work
- Registering in the EU database
- Assessing whether your activity falls under the AI Act
- Properly setting up processes with regard to copyright
- We are also certified for IP Scan and can advise you on protecting your copyright when using or developing AI tools.
Pavla Nečasová specialises in business consulting, GDPR, photography law (copyright, licences, protection of personality rights) and artificial intelligence law.
In the field of photography law she publishes in photography magazines, lectures and founded the blog fotopravo.cz
During her time at the Supreme Court she gained invaluable experience with appellate proceedings and legal argumentation, which she draws on in practice.