Use of large language models in the patent industry: A risk to patent quality?

Large language models (LLMs) such as ChatGPT have been hailed as potentially revolutionary for the legal industry. Lord Justice Birss himself recently made headlines for praising the usefulness of ChatGPT for writing court judgments. LLMs are undeniably a major technological advance with the potential to make a significant impact on the legal industry. We are now awash with companies claiming to provide LLM software capable of drafting, prosecuting and challenging patents. The data protection concerns with LLM have been extensively discussed elsewhere and represent a legal and contractual challenge as opposed to technological one. This post will instead focus on the potential short to medium term impact of LLMs on the patent profession.

Large language models (LLM)

Large language models (LLM), such as ChatGPT, are a particular type of Generative AI that manipulate and generate text. LLMs are trained on vast quantities of text data from the internet. These tools are capable of producing intelligible and superficially intelligent text in response to user prompts. PatKat is reliably informed by experts in the field that LLMs are a rare example of a true paradigm shift. The current capabilities of LLM would have been unimaginable in the AI field as little as 8 years ago. Remarkably, ChatGPT and its cohorts are now capable of passing the famous Turing test, such that text generated by ChatGPT cannot be readily identified as being written by an AI as opposed to a human, as any teacher struggling to identify AI-assisted essays will tell you. 

LLMs can often produce sensible answers to questions for which an internet search would provide an answer. The most obvious use case for LLMs is therefore chat-bots. LLMs are also good at reformatting text into certain styles whilst retaining the original meaning, making them useful for summarising information in long-form text (as may be required in UK court judgements). However, there is also considerable excitement that LLM may revolutionize many types of service industry, including the legal profession. After all, ChatGPT is reportedly capable of passing the US bar exam.

Software for the patent industry before LLMs

Various software solutions for patent search, drafting and prosecution have been available and marketed for some time. However, any solution pre-dating LLMs would have had severely limited functionality compared to LLMs. Software predating LLMs could have provided pre-drafted boilerplate language, searched for exact words and phrases within text and assessed the correspondence of text to a predetermined format. However, without the use of a LLM, they would not have been able to generate meaningful text describing or providing nascent verbal reasoning. By contrast, LLMs have the remarkable capacity to read and "understand" text, to the extent that they can rephrase, summarise and extrapolate meaning from pre-existing text. No software tool predating LLMs had anything close to this ability. If you are looking for a tool capable of generating meaningful verbal reasoning in the form of a patent draft or office action response, anything predating LLMs may therefore be reasonably ignored. 

LLMs for patent drafting and prosecution

Superficially, patent drafting and prosecution therefore seems to be an ideal use case for LLMs. Patent drafts and office actions follow a distinct format and style. In terms of data, patents and patent office correspondence are also freely publicly available on Google Patents and the respective patent registers. 

Turing test

However, the first thing you notice as soon as you look at current LLM solutions for patent drafting and prosecution, are the limitations to the technical field in which the new LLM tools can be used. The majority of LLM tools currently advertised for use in patents are focused on mechanical and software inventions. This is not surprising given that mechanical inventions represent an easier use case. Particularly, understanding a mechanical invention generally does not require the complexity of data interpretation needed in the biotech and pharmaceutical field. However, the focus on mechanical and software inventions is probably also a reflection of the subject-matter specialisms of the software developers producing the tools. Whilst data (and particular figure/graph) interpretation does represent a particular challenge to LLMs such as ChatGPT, there is no reason to view this problem as insurmountable. OpenAI is already makes moves towards providing this functionality. To this Kat, it should therefore not be long before LLM can also be applied outside of the mechanical field.

However, even ignoring the subject-matter bias of current tools, LLM tools for patent drafting and prosecution still have some considerable limitations. LLM are undeniably very good at providing generic text on a topic for which the internet provides extensive guidance, and for which a deep understanding of complex specialist technical issues is not required. LLM are similarly very good at summarising and reformatting text whilst retaining the pre-existing meaning of the text, e.g. from a bullet point list into paragraphs. However, both of these tasks are far removed from the detailed technical verbal reasoning required for patent drafting and prosecution. 

LLM are thus good at providing long-form novelty or inventive step arguments, provided that they are spoon-fed the argumentative points in the form of prompts. LLM may also be used to draft claims when given a detailed description of the essential features of a mechanical invention (and vice versa). Left to its own initiative to determine and claim the inventive concept of an invention, or to devise its own novelty or inventive step argument in view of cited prior art, a LLM will also be able to provide text that makes sense, is error free and superficially responds to an Examiner's objections. However, on closer inspection, this text will often be extraordinarily generic and lacking in substance. The problem is that the current LLMs used for patent work will still lack the deeper understanding of how the world works within specialist technical fields, and thus will fail to produce a sophisticated reasoned response or to deduce and capture the inventive concept of an invention. 

The limitations of LLM in patents is at least partly due to the relatively limited data available for patents, particularly if we consider the even further limited amount of relevant data available within each respective technical field. Compared to the vast majority of use scenarios demonstrating the abilities of LLMs (such as passing the bar exam), patents are still a very niche area for which there is limited data and guidance on the internet, either with respect to the law itself or the technical subject matter within each field.

At best, therefore, an LLM-generated draft will most likely resemble the work product of a patent attorney working outside of their technical field who is out of their depth with respect to the subject-matter. However, this does not of course mean that LLM tools don't have anything to offer the patent industry. 

Are LLMs a risk to patent quality?

Even though a LLM work-product may currently be of lower quality to that produced by an attorney, the ability of LLMs to produce superficially acceptable patent work products may still make the use of LLM attractive in certain sectors and technical fields. The use of LMMs may have the significant advantage of reducing both the time and cost of patent work. The LLM may therefore be beneficial to applicants with very large patent portfolios, particularly within the mechanical field (for which the technical specialism is less of an issue), or where keeping the application pending is more important than achieving grant. LLM tools may also be useful for firms focussed on cost-cutting and time saving, effectively replacing or supplementing for expensive partner time in the manner historically done with trainee billable hours. 

Patent offices may therefore soon find themselves flooded with AI-generated drafts, responses and third party observations (TPOs) that appear to respond to an Examiner's objections and satisfy patentability requirements, whilst failing to make any substantive points or claim anything of substance. The industry should therefore be alive to the risk that the use of LLM may increase overall work-load for patent offices (and costs for applicants) whilst lowering the overall quality of patent applications and prosecution responses. 

Final thoughts

LLM possesses the remarkable ability to generate meaningful text in a user-stipulated style. LLMs thus have the potential to be revolutionary for the legal field, including the patent industry. However, as things currently stand, LLM tools still have some considerable limitations. Whilst we are still some way away from AI that is capable of replacing qualified attorneys, it nonetheless seems likely that the use of LLM tools to assist attorneys will soon be considered routine. It will therefore be essential for the patent industry to ensure patent quality is maintained. Whilst LLM tools are still in their infancy, patent professionals need to be aware of the current limitations of these tools, and transparent about their use with clients. 

Acknowledgements: Thanks to Mr PatKat for his AI insights and expertise.