The EUIPO has published the second phase of its research on Automated Content Recognition (ACR) technologies

On September 22, 2022, the European Union Intellectual Office (EUIPO), through its European Observatory on Infringements of Intellectual Property Rights, published a discussion paper entitled "Automated Content Recognition: IP Enforcement and management use cases". The study represents the second phase of a project already started in 2019, which led to the publication of its first phase named "Automated Content Recognition: Existing technologies and their impact on IP" in November 2020.

The ability of computers to recognise content

There is no universally accepted, standardised definition of "Automated (or "Automatic") Content Recognition (ACR)" technologies. However, over the last two decades, the ability of computers to recognise content has become crucial for a wide range of applications (and this is not a list of the capabilities of HAL 9000 in "2001: A Space Odyssey").

Simply put, ACR technologies are all those technologies that are able to identify and recognise the type of content stored in a digital file, or content that can even be placed in an offline environment (e.g. in a shop) and recognised by a camera, through a computed "automatic" process, without the intervention of human review. Some of the most useful contexts in which these technologies are used are digital media (such as Merpel's extensive use of Shazam to recognise his favourite music), e-commerce or cloud computing. They are also commonly used to moderate illegal or harmful online content on e-commerce marketplaces, social media or even video sharing platforms, and to automatically classify content stored in the cloud. This versatility of use has led a number of technology companies to develop solutions and invest heavily in this area (see below). 

In addition to these purposes, ACR technologies are also used to support the protection and management of intellectual property rights online (and offline?) – e.g. among others, to analyse brand awareness by recognising images of branded products posted on social media or online marketplaces. The Observatory has worked intensively on this research, which led to the publications of the above discussion papers.

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", September 2022

Scope

The aim of both studies was to raise public awareness of what ACR technologies are per se (Phase 1) and how they intersect with IP protection and management (Phase 2). In addition, Phase 2 also analyses the perspective of the various stakeholders involved in their implementation, such as IP owners and intermediaries like e-commerce marketplaces, social media and content sharing services. The recently published discussion paper also looks at the potential impact and opportunities that the technologies may also have for general users and law enforcement authorities. 

Implementing ACR-based solutions for intellectual property protection and management

The Observatory's research analysed the following ACR technologies:

• Hashing, i.e. the generation of a short string of characters (a 'hash') that is assigned to the file as a unique identifier.
• Watermarking, i.e. the ability to add information and embed it directly into a product, signal or digital file.
• Fingerprinting, i.e. extracting and analysing recognisable features/information of a digital content (audio or video) to identify and assign some of its unique inherent characteristics.
• AI-based or enhanced solutions such as computer vision, especially logo recognition, character recognition and object recognition, used to recognise content online and offline.

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 1: Existing technologies and their impact on IP", page 6

The recently published second phase focuses on the concrete implementation of these technologies by rights holders, intermediaries and law enforcement authorities in the protection and management of intellectual property rights by analysing five use cases:

1. Solutions to detect IP-infringing listings on e-commerce marketplaces; 
2. Smartphone solutions to detect genuine or counterfeit products; 
3. Solutions to recognise 3D printing files and 3D-printed products; 
4. Solutions to protect and manage copyright and neighbouring rights on content-sharing services; 
5. Solutions to identify live streams of IP-protected content. 

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", List of use cases, page 5

Use Case 1: The detection of IP-infringing listings on e-commerce marketplaces

The first use case focuses on how AI-based technologies (i.e. logo, character or object recognition) and invisible watermarking/image fingerprinting are used by some e-commerce marketplaces (e.g. Amazon Brand Registry, Alibaba IP Protection Platform, eBay VeRO) or third-party vendors (e.g. Red Points, BrandShield, Smart Protection among others) to detect potential IP-infringing listings online. The above solutions can be combined with contextual information (e.g. keywords usually associated with IP-infringing listings) and behavioural data (e.g. sellers' history) to improve IP-violating offer detection. The use case shows how ACR technologies are part of broader solutions for dealing with IP infringing listings, which may include automatic removal, notice and take down or even human review of entries to confirm their IP-violating nature.

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", List of some ACR technologies with the types of content they can recognise, page 13

Use Case 2: Smartphone solutions to detect genuine or counterfeit products

The second use case addresses the potential of using portable devices such as smartphones as wearable hardware for implementing ACR technologies. It explains how this could provide law enforcement authorities (LEAs) with portable tools to detect genuine and counterfeit products during searches or on-site inspections. Although these integrated solutions are not yet available to LEAs, some companies are developing ACR-based authentication solutions for smartphones with applications to capture an image of a product to be authenticated. The application can then use AI-based content recognition to analyse one or more specific features of the product (e.g. its pattern, trade mark or design) to determine whether it is genuine or counterfeit. The development of integrated solutions that effectively serve the purpose of LEAs will likely require the use of a dedicated AI-based content detection solution that could be fed with information from a variety of IP owners to detect potentially IP-infringing products.

© Entrupy, "How to use Entrupy Luxury Authentication", 2021, Vimeo

Use Case 3: Solutions to recognise 3D printing files and 3D-printed products

The third use case provides an overview of ACR-based solutions for marking, tracking and identifying the computer-aided design (CAD) and the finished 3D printed products. It's well known that 3D printing consists of two main phases (and related implications from an IP perspective - e.g. licensing and distribution of files and finished products): the creation of the digital design of the object and the printing with a 3D printer. In both processes, ACR solutions can be used to detect CAD files and printed products.

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", Different steps of the 3D printing process, page 29

Use Case 4: Solutions to protect and manage IPRs on content-sharing services

The fourth use case focuses on the concrete implementation of fingerprinting-based solutions in content-sharing services, such as social media or video-sharing platforms (also defined as "Online-Content-Sharing Services Providers - OCSSPs). This use case analyses the impact of these technologies on the monitoring, management and moderation of the large amount of copyright-protected content shared through these services. This issue has also raised the question of how to strike the right balance between the respective interests and rights of intellectual property owners on the one hand and amateur creators, users and content sharing services on the other. 

In this context, Article 17 of Directive EU/2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market provides for a specific liability regime requiring online content sharing service providers to make a communication to the public within the meaning of the Directive and to obtain an authorisation from the respective rights holders for the content uploaded by their users to their services. Paragraph 4 of the Article requires service providers, in cases where authorisation is not obtained, to demonstrate that they have used their best efforts to obtain authorisation and to ensure the unavailability of certain works previously identified by rightholders in order to avoid liability for copyright infringement.

The use case analyses the internal solutions of content sharing services (i.e. YouTube's Content ID as well as Meta's Rights Manager) as well as purchased or outsourced third-party ACRs (e.g. Audible Magic).

The use case also looks at ACR-based solutions for image and photo content and the opportunities these technologies offer for the publishing sector.

Finally, the use case focuses on ACR solutions to support copyright management with regard to the application of exceptions and limitations on the internet (in light of the European Commission's Guidance on Article 17 of Directive (EU) 2019/790 on Copyright in the Digital Single Market, 2021 and Case C-401/19 (26/04/2022, C-401/19, Poland v Parliament and Council, EU:C:2022:297, § 85-86).

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", Example of a fingerprint-based ACR and content management workflow, page 38

Use Case 5: Solutions to identify live streams of IP-protected content

The last use case analyses ACR solutions that could prevent illegal live streaming of sporting events, concerts or other IP-protected content via illegal IPTV services or open web streams, or in legitimate services that offer live streaming capabilities to their users.

In this use case, the use of a combination of forensic watermarking and fingerprinting is crucial to identify the authorised stream of live content used as the source of the illegal stream and to identify and block such streams.

The use case also focuses on cooperation between all actors (intermediaries, IP owners and final users) involved in the removal of live streams of infringing content. Due to the time-sensitive nature of live stream content, IP owners typically seek near-immediate blocking by the intermediary upon notification and close cooperation with relevant intermediaries to develop optimised notification and action processes.

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", Forensic watermarking, live streaming ecosystems and data flows, page 58

© EUIPO, "Automated Content Recognition: Discussion Paper – Phase 2: IP enforcement and management use cases", Fingerprinting, live streaming ecosystems and data flows, page 59

Conclusion

The EUIPO's discussion papers show what ACR technologies are and how they can support IP rights management and protection in concreto. The five use cases reported on provide an overview of the challenges and opportunities that ACR technologies can offer in specific IPR infringement contexts, both in online and offline environments. The analysis focuses on the fact that the actors and providers involved in the potential use of ACR are not just third-party technology developers, but also content sharing services and/or e-commerce marketplaces per se and their direct cooperation with IP owners, general users and law enforcement authorities in the fight against counterfeiting and piracy. Finally, the discussion paper highlights the limitations of these implementations from the perspective of their technical accuracy and from a legal perspective in certain contexts, such as the application of copyright exceptions and limitations - where human intervention may still be necessary (e.g. use case 4).