Use of large language models in the patent industry: A risk to patent quality?

Large language models (LLMs) such as ChatGPT have been hailed as potentially revolutionary for the legal industry. Lord Justice Birss himself recently made headlines for praising the usefulness of ChatGPT for writing court judgments. LLMs are undeniably a major technological advance with the potential to make a significant impact on the legal industry. We are now awash with companies claiming to provide LLM software capable of drafting, prosecuting and challenging patents. The data protection concerns with LLM have been extensively discussed elsewhere and represent a legal and contractual challenge as opposed to technological one. This post will instead focus on the potential short to medium term impact of LLMs on the patent profession.

Large language models (LLM)

Large language models (LLM), such as ChatGPT, are a particular type of Generative AI that manipulate and generate text. LLMs are trained on vast quantities of text data from the internet. These tools are capable of producing intelligible and superficially intelligent text in response to user prompts. PatKat is reliably informed by experts in the field that LLMs are a rare example of a true paradigm shift. The current capabilities of LLM would have been unimaginable in the AI field as little as 8 years ago. Remarkably, ChatGPT and its cohorts are now capable of passing the famous Turing test, such that text generated by ChatGPT cannot be readily identified as being written by an AI as opposed to a human, as any teacher struggling to identify AI-assisted essays will tell you. 

LLMs can often produce sensible answers to questions for which an internet search would provide an answer. The most obvious use case for LLMs is therefore chat-bots. LLMs are also good at reformatting text into certain styles whilst retaining the original meaning, making them useful for summarising information in long-form text (as may be required in UK court judgements). However, there is also considerable excitement that LLM may revolutionize many types of service industry, including the legal profession. After all, ChatGPT is reportedly capable of passing the US bar exam.

Software for the patent industry before LLMs

Various software solutions for patent search, drafting and prosecution have been available and marketed for some time. However, any solution pre-dating LLMs would have had severely limited functionality compared to LLMs. Software predating LLMs could have provided pre-drafted boilerplate language, searched for exact words and phrases within text and assessed the correspondence of text to a predetermined format. However, without the use of a LLM, they would not have been able to generate meaningful text describing or providing nascent verbal reasoning. By contrast, LLMs have the remarkable capacity to read and "understand" text, to the extent that they can rephrase, summarise and extrapolate meaning from pre-existing text. No software tool predating LLMs had anything close to this ability. If you are looking for a tool capable of generating meaningful verbal reasoning in the form of a patent draft or office action response, anything predating LLMs may therefore be reasonably ignored. 

LLMs for patent drafting and prosecution

Superficially, patent drafting and prosecution therefore seems to be an ideal use case for LLMs. Patent drafts and office actions follow a distinct format and style. In terms of data, patents and patent office correspondence are also freely publicly available on Google Patents and the respective patent registers. 

Turing test
However, the first thing you notice as soon as you look at current LLM solutions for patent drafting and prosecution, are the limitations to the technical field in which the new LLM tools can be used. The majority of LLM tools currently advertised for use in patents are focused on mechanical and software inventions. This is not surprising given that mechanical inventions represent an easier use case. Particularly, understanding a mechanical invention generally does not require the complexity of data interpretation needed in the biotech and pharmaceutical field. However, the focus on mechanical and software inventions is probably also a reflection of the subject-matter specialisms of the software developers producing the tools. Whilst data (and particular figure/graph) interpretation does represent a particular challenge to LLMs such as ChatGPT, there is no reason to view this problem as insurmountable. OpenAI is already makes moves towards providing this functionality. To this Kat, it should therefore not be long before LLM can also be applied outside of the mechanical field.

However, even ignoring the subject-matter bias of current tools, LLM tools for patent drafting and prosecution still have some considerable limitations. LLM are undeniably very good at providing generic text on a topic for which the internet provides extensive guidance, and for which a deep understanding of complex specialist technical issues is not required. LLM are similarly very good at summarising and reformatting text whilst retaining the pre-existing meaning of the text, e.g. from a bullet point list into paragraphs. However, both of these tasks are far removed from the detailed technical verbal reasoning required for patent drafting and prosecution. 

LLM are thus good at providing long-form novelty or inventive step arguments, provided that they are spoon-fed the argumentative points in the form of prompts. LLM may also be used to draft claims when given a detailed description of the essential features of a mechanical invention (and vice versa). Left to its own initiative to determine and claim the inventive concept of an invention, or to devise its own novelty or inventive step argument in view of cited prior art, a LLM will also be able to provide text that makes sense, is error free and superficially responds to an Examiner's objections. However, on closer inspection, this text will often be extraordinarily generic and lacking in substance. The problem is that the current LLMs used for patent work will still lack the deeper understanding of how the world works within specialist technical fields, and thus will fail to produce a sophisticated reasoned response or to deduce and capture the inventive concept of an invention. 

The limitations of LLM in patents is at least partly due to the relatively limited data available for patents, particularly if we consider the even further limited amount of relevant data available within each respective technical field. Compared to the vast majority of use scenarios demonstrating the abilities of LLMs (such as passing the bar exam), patents are still a very niche area for which there is limited data and guidance on the internet, either with respect to the law itself or the technical subject matter within each field.

At best, therefore, an LLM-generated draft will most likely resemble the work product of a patent attorney working outside of their technical field who is out of their depth with respect to the subject-matter. However, this does not of course mean that LLM tools don't have anything to offer the patent industry. 

Are LLMs a risk to patent quality?

Even though a LLM work-product may currently be of lower quality to that produced by an attorney, the ability of LLMs to produce superficially acceptable patent work products may still make the use of LLM attractive in certain sectors and technical fields. The use of LMMs may have the significant advantage of reducing both the time and cost of patent work. The LLM may therefore be beneficial to applicants with very large patent portfolios, particularly within the mechanical field (for which the technical specialism is less of an issue), or where keeping the application pending is more important than achieving grant. LLM tools may also be useful for firms focussed on cost-cutting and time saving, effectively replacing or supplementing for expensive partner time in the manner historically done with trainee billable hours. 

Patent offices may therefore soon find themselves flooded with AI-generated drafts, responses and third party observations (TPOs) that appear to respond to an Examiner's objections and satisfy patentability requirements, whilst failing to make any substantive points or claim anything of substance. The industry should therefore be alive to the risk that the use of LLM may increase overall work-load for patent offices (and costs for applicants) whilst lowering the overall quality of patent applications and prosecution responses. 

Final thoughts

LLM possesses the remarkable ability to generate meaningful text in a user-stipulated style. LLMs thus have the potential to be revolutionary for the legal field, including the patent industry. However, as things currently stand, LLM tools still have some considerable limitations. Whilst we are still some way away from AI that is capable of replacing qualified attorneys, it nonetheless seems likely that the use of LLM tools to assist attorneys will soon be considered routine. It will therefore be essential for the patent industry to ensure patent quality is maintained. Whilst LLM tools are still in their infancy, patent professionals need to be aware of the current limitations of these tools, and transparent about their use with clients. 

Acknowledgements: Thanks to Mr PatKat for his AI insights and expertise. 

Use of large language models in the patent industry: A risk to patent quality? Use of large language models in the patent industry: A risk to patent quality? Reviewed by Rose Hughes on Tuesday, October 03, 2023 Rating: 5


  1. I see the bigger risk in that the mere use of LLMs to draft a patent application could invalidate that patent application. The terms and conditions of many LLMs are remarkably thin on guaranteeing your inputs will be kept confidential.

    The old adage, if you’re not paying for the product, you are the product,comes to mind.

    1. I agree, it seems you have to agree to use your input for future training, your inputs are therefore not confidential, an own goal it would seem

    2. I don't believe any of the current models are training in real time, so while input data could be used after the fact the timescale is currently outside the danger zone for patent filings. I believe one of the players in the patent LLM space are running their own instance of the tool on the cloud so there is no greater privacy risk than using OneDrive or similar.

    3. Hedger, your comment is alarming. It does not matter whether the LLM reproduces your advice on the day you hand over the data or in 20 years. The act of handing over your data to someone who is free to disseminate the data is itself a public disclosure.

    4. You aren't handing your data over to anybody. LLMs can be trained with old data and then subsequently ran on a private instance, of which only the end user has access to. Therefore you aren't disclosing anything to any other human being, its working more like a computer program.

    5. The case law would suggest a confidently agreement is also required. See T0002/09, 4.2:

      “According to the case law of the boards of appeal, information is "available to the public" if only a single member of the public is in a position to gain access to it and understand it, and if said member of the public is under no obligation to maintain secrecy (see T 1081/01, point 5 of the Reasons, affirmed by T 1309/07, point 3.2.1 of the Reasons). Whether or not a member of the public has actually accessed the information is irrelevant (see T 84/83, point 2.4.2 of the Reasons).”

      This is uncharted territory and there are certainly arguments as to whether this old law holds up when applied to LLM servers.

      The comparison with cloud storage is interesting, and arguably this line of case law might be more relevant. T1153/06, headnote:

      “It can be safely concluded that a document stored on the World Wide Web was made available to the public:
      If, before the filing or priority date of the patent or patent application, a document stored on the World Wide Web and accessible via a specific URL
      (1) could be found with the help of a public web search engine by using one or more keywords all related to the essence of the content of that document and
      (2) remained accessible at that URL for a period of time long enough for a member of the public, i.e. someone under no obligation to keep the content of the document secret, to have direct and unambiguous access to the document,
      then the document was made available to the public in the sense of Article 54(2) EPC 1973.
      If any of conditions (1) and (2) is not met, the above test does not permit to conclude whether or not the document in question was made available to the public.
      (See point 6.7.3)”

      Is a cloud storage provider staff member bound by confidence? Google workspace t&cs say:

      “The recipient will only use the disclosing party's Confidential Information to exercise the recipient’s rights and fulfill its obligations under the Agreement, and will use reasonable care to protect against the disclosure of the disclosing party's Confidential Information. The recipient may disclose Confidential Information only to its Affiliates, employees, agents, or professional advisors ("Delegates") who need to know it and who have agreed in writing (or in the case of professional advisors are otherwise bound) to keep it confidential. The recipient will ensure that its Delegates use the received Confidential Information only to exercise rights and fulfill obligations under this Agreement.”

      Not perfect, but perhaps good enough. ChatGPT has nothing even approaching this, plus they can update terms at any time.

      I am sure someone more clever can argue a path that reconciles the law with LLMs, but until an authority does so, would you risk using an LLM to draft a patent?

  2. Is ChatGPT capable of passing the EQE or FD4? Since the subject-matter is usually mechanical and intended to be accessible to everyone, regardless of technical background, it might have a chance. However, I suspect that the answers would superficially look ok, but on closer inspection would be too generic and lacking in substance for the reasons Rose explains. Might be fun to try it though.

    1. You are forgetting that FD4 is a perfect exam that separates the wheat from the chaff. An AI model that could even come close to passing FD4 is the stuff of science fiction.

  3. Great article Rose as usual. You mentioned how LLMs could be used to replace the work typically done by trainees to keep costs down. I think this is actually a really big concern for our industry. Firms that can replace their trainee work with LLMs operated by skilled attorneys could be significantly more cost-efficient than those that don't. If trainee attorneys can be entirely replaced, how will firms justifying the cost of training the next generation of attorneys?
    To some extent this will be avoided in firms with a strong focus on quality over quantity but the issue will become more prominent as the technology improves.

  4. A lot of the (over) excitement about LLMs seems to reveal something about the perception of the role of a patent attorney, which I think is sometimes shared by some in the profession. Rose is 100% right to say:
    "...LLM are similarly very good at summarising and reformatting text whilst retaining the pre-existing meaning of the text, e.g. from a bullet point list into paragraphs. However, both of these tasks are far removed from the detailed technical verbal reasoning required for patent drafting and prosecution."
    The challenge for us is to make people understand that the job of an attorney is not just to write in a certain style using certain choice words that you don't often find outside of patent specifications. Personally I think a lot of attorneys could do better to use some of those words much less.
    I am sure Rose is also right to say "LLM are thus good at providing long-form novelty or inventive step arguments, provided that they are spoon-fed the argumentative points in the form of prompts." But how useful really is that as a tool? Do examiners really want "long-form" arguments when the same point could be made in a succinct list of bullet points?
    Yes there are certain conventions we have to (or probably ought to) follow but the ability of LLMs is a reminder of the importance of style over substance. Patent attorneys (at the moment) have nothing to fear if substance matters.
    I remember reading a few years ago some survey that CIPA had done about training. One of the top complaints from trainees was about supervisors "correcting" work for style. I am sure an LLM could be made to take in an already good draft specification and turn it into one that the trainee's supervisor would be happier with in terms of matching the style... but wouldn't a better (and cheaper!) solution be a more open mind on the part of the supervisor?

    1. Very thought-provoking comment, especially considering trainee angle.

      Patent attorneys have nothing to fear only if there is honesty from providers of such services, including their limitations which include destroying novelty of future patents. See here, the provider refuses to acknowledge there is a problem -

      If patent attorneys have to compete with ‘cheaper’ services that could invalidate IP rights, that’s a much harder battle to win in absence of the required honesty. Patent attorneys should be afraid of this.

    2. True - cheaper, poor-quality and sometimes dishonest competitors are a challenge we face. It is probably difficult for clients to tell the difference between high-quality work and work that looks superficially like a reasonable patent draft but just hasn't had the thought put in. Actually I think many clients eventually figure it out but only when it is too late...

      In principle the cheaper, poor-quality, AI-based competitors are no different from the cheaper, poor-quality, human competitors in that sense, but there are going to be more of them now, everyone is excited about AI and those who want to believe will believe.

      The exchange with Dolcera is interesting... for my part I would have though the confidentiality issue, though serious, is one that is in principle solvable just as many are happy to store confidential documents on cloud-based services. But Dolcera's combative approach and an apparent failure to understand the concern at all is not really going to do them any favours.

  5. It is interesting to see that Rose and the commenters are looking at LLM=AI from the attorney side.
    It would also be interesting to look at it from the granting authority side.
    I am convinced that the upper management of the EPO is longing to introduce AI in search and examination and push production/productivity to much higher levels.
    The quality issues mentioned on the attorney’s side are no doubt also applicable to the EPO side. At the EPO quantity is already weighing much higher than quality. Bringing in AI would most probably imply a further increase of quantity over quality.
    Shoddy applications written with the help of AI vs. shoddy search and examination with the help of AI would probably lead to more shoddy patents. In the long run, it could be the end of patents as we know them. What good are such patents? By then neither trainees/representatives nor examiners would be needed.
    EPO’s logo is meant to show that a patent is unique, like fingerprints are unique. How can this tie up with an increased use of AI in patent drafting and prosecution?

  6. While the interactions of generative AI with copyright law is outside the scope of Rose’s post, recent developments in this area must be highlighted.
    The 18 Aug 2023 decision Thaler v Perlmutter of the US District Court for the District of Columbia upheld the US Copyright Office’s position that « copyright can protect only material that is the product of human creativity,”.
    This ruling however leaves open the degree of human input necessary for an AI-generated work to qualify for copyright protection.
    Also of note is a class action suit (11 July 2023) against Google in the US District Court for Northern District of California.
    The question of the degree of human input to an AI-generated output required for securing protection may also arise in the case of inventions. This is the case under German inventor remuneration law, where an applicant has to list the inventors and their respective contributions and may include in the list the contribution of an IA system (which reduces the inventors' contributions and the remunerations to be paid by the employer).

  7. Thanks for an interesting article, Rose. Speaking from the attorney side, I think that LLMs will play a huge and increasing role in our industry. I have heard many tech-industry insiders claim - rightly or wrongly - claim that generative AI (of which LLMs are of course just one aspect) constitutes a paradigm shift as great as the widespread adoption of the internet. That may prove to be inaccurate, but if the impact of generative AI is even detectable comparable to the overall impact of the internet then it seems crazy to think that it *won't* have a major impact on the industry.

    My feeling is that patent attorneys are largely "small-c" conservative when it comes to new technology, and my worry is that the speed of development will outpace firms' ability to adapt. GPT-3 really only came to public prominence about a year ago and its abilities today are vastly greater than those then. What will it be in 5 years? And yet how many firms have an "AI" plan - or even plan to have a plan?

    My view is that generative AI will inevitably play a major supporting routine attorney work - drafting boilerplate, writing up case law and summarising documents for sure, and perhaps if the legal framework can be defined, drafting whole specifications also. Certainly we will find small applicants seeking protection based already-filed specifications that have been generated by AI, but those same applicants probably wouldn't be using a quality-focussed firm anyway. Could this even push the overall quality of the industry up, forcing an "evolve or die" approach on firms pushing quantity over quality?

  8. As an in-house attorney who commissions external drafts, this is easy - any firm using AI drafting gets kicked off my approved list. We have had no problem booting firms for bad drafting quality (and disappointingly bad client care, but that's a feature of the profession). The issue is that we need the whole application to be written with understanding, not just of the invention per se, which can be technically complex, but the product it relates to and the broader commercial strategy. This needs attorney time and domain knowledge, which despite internal cost constraints, we take time to build. It is not clear to me how a superficially plausible block of text cranked out of ChatGPT or Dolcera or whatever will replicate that, and none of the boosters, who appear to be unavoidable on LinkedIn, have a convincing answer.


All comments must be moderated by a member of the IPKat team before they appear on the blog. Comments will not be allowed if the contravene the IPKat policy that readers' comments should not be obscene or defamatory; they should not consist of ad hominem attacks on members of the blog team or other comment-posters and they should make a constructive contribution to the discussion of the post on which they purport to comment.

It is also the IPKat policy that comments should not be made completely anonymously, and users should use a consistent name or pseudonym (which should not itself be defamatory or obscene, or that of another real person), either in the "identity" field, or at the beginning of the comment. Current practice is to, however, allow a limited number of comments that contravene this policy, provided that the comment has a high degree of relevance and the comment chain does not become too difficult to follow.

Learn more here:

Powered by Blogger.