OpenAI's large language model (LLM) patents

OpenAI's approach to IP is often cited in the tech industry as an example of a radically new approach to IP. OpenAI has the reputation for protecting its innovations through the use of trade secrets as opposed to patents. However, it appears that this characterisation of OpenAI's strategy is years out of date. This is perhaps not surprising, given that patent applications are not published until 18 months after they are filed, and that OpenAI's major innovation of ChatGPT was first released about 2 years ago. This year, eleven OpenAI patents and patent applications have been published. Intriguingly, OpenAI is not just filing patents, it is pursuing highly accelerated grant of its patents. This strategy implies that OpenAI has the intention to enforce its IP or at the very least wants the ability to do so. 

Who are OpenAI?

OpenAI began life in 2015 as a non-profit company, with the self-proclaimed vision of providing humanity with safe and beneficial AI. The company famously kick-started a paradigm shift in AI tech in 2022, with the release of its large language model (LLM), ChatGPT. ChatGPT was a revolutionary generative AI model designed to interpret and generate human-like text. Critically, both the inputs and outputs of ChatGPT are in natural language, effectively democratising the use of AI. 

PatKat on the hunt for patent news

OpenAI's shift to a more commercial model began in 2019, with the creation of its profit-based subsidiary OpenAI Global, LLC, Since then, Microsoft has invested approximately US$13 billion into OpenAI and owns nearly 50% of its equity. This year, the tech space has been alive with rumours that OpenAI is now planning a shift to a fully profit-based model. Every big tech company also now has their own version of ChatGPT, and there are countless other companies implementing ChatGPT in a vast range of business solutions. 

OpenAI is pursuing speedy patent grant

Fascinatingly, OpenAI is pursuing highly accelerated grant of its IP. A patent can only be enforced once it is granted. However, it can take 3 to 5 years of patent prosecution before national patent offices to achieve grant of a patent. There are also a number of formal prosecution procedures that delay the process. Claiming priority from a first initial filing can provide up to 12 months of additional patent protection, but delays grant of the application by the same amount. Filing an international PCT application reduces costs and can help with efficient multi-jurisdictional prosecution, but will also take longer than pursuing a direct national filing. Therefore, the quickest way to achieve grant of your patent in a particular country is to pursue a direct national filing, possibly even dispensing with a priority claim. Whilst this may have disadvantages in terms of cost and sacrifice of patent term, it can provide you with a granted patent in less than a year. 

All of the currently published applications filed by OpenAI are US cases. Nine of these cases have already been granted. In some cases the time between the filing date and grant was as little as nine or ten months. Furthermore, some of these cases are direct national US filings that do not claim priority. Given the sacrifice of patent term associated with this strategy, OpenAI's approach strongly suggests that quick grant is a priority for them. At the moment, we can only speculate why this may be the case. The most obvious reason would be that OpenAI intends to enforce its patents and/or to use them as negotiating chips in licensing negotiations. It is also clear from the public databases that OpenAI is pursuing global protection, however these patent applications have not yet been published or granted. 

A closer look at the OpenAI patents

Focusing just on the granted US patents, the claim scope achieved by OpenAI is remarkably broad. The patents relate to generalisable use cases for language models. The OpenAI patents relate to the use of large language models for generating and editing text (US11983488B1US11886826B1), images (US11922550B1US11983806B1, US12039431B1), and code (US12008341B2). There are also cases covering methods for integrating language modes with an external API or graphical user interface (US11922144B1US12051205B1)

Importantly, the scope of a patent is determined by the language of the granted claims. To infringe a patent claim, a method or product must possess every feature listed in that claim. For example, US11983488B1, relating to the generation and editing of text, specifically covers: 

10. A method for automatically generating and editing text, comprising: 
    receiving an input text prompt, the input text prompt comprising a null set;
    receiving one or more user instructions;
    determining a set of model parameters based on the one or more user instructions;
accessing the language model using the at least one processor based on the input text, the set of model parameters, and the one or more user instructions;
generating, using the accessed language model, an output text based on the input text, the one or more user instructions, and at least one of a sampling temperature parameter or a nucleus sampling parameter;
    receiving one or more new user instructions;
editing the output text based on the language model and the one or more new user instructions by     replacing at least a portion of the output text; and
optimizing the accessed language model by aligning the language model based on the output text using machine learning;
wherein:
    the language model is optimized through one or more iterative cycles of training based on one or more outcome metrics associated with the output text and one or more datasets; and
the one or more datasets comprise at least one of annotated data, labeled data, enriched data, or demonstration data based on one or more output text.

The claims of the other granted patents can be read on Google PatentsImportantly, it is not necessary to include the code for a model in a patent. The description sections of the patents therefore just broadly describe the architecture and training methods for the models, as well as providing examples of their use. 

Final thoughts

For the past 2 years, OpenAI has therefore been busy filing patent applications for the key use cases of ChatGPT. So much for an IP policy based solely on trade secrets! Ironically, however, ChatGPT itself does not yet appear aware of the shift in its owner's strategy. Today, when asked "Does OpenAI have any patents?", ChatGPT 4o returned: "As of now, OpenAI does not hold any patents. This approach aligns with its broader strategy of promoting transparency and responsible AI development while avoiding the traditional IP model that might hinder collaboration or the ethical use of AI technology". Aside from the clear factual inaccuracy of this statement, PatKat also disagrees with ChatGPT's implication that patenting does not align with a goal of transparent or responsible AI development. Afterall, patents must be published, whereas trade secrets, by definition, are not disclosed. The fact that OpenAI is following other tech companies by patenting their innovations should also come as no surprise. The patent system is a well-established, robust and effective way to protect innovation, and it makes sense for OpenAI to make full use of it. 

Further reading

OpenAI's large language model (LLM) patents OpenAI's large language model (LLM) patents Reviewed by Rose Hughes on Tuesday, August 13, 2024 Rating: 5

3 comments:

  1. In claim 10 showed above, I have noted no less than 6 times the terms “user instructions”, whereby a distinction is apparently made between “original user instructions”, cf. line 3, and “new user instructions”, cf. line 10.

    At least in Europe, I cannot see such a claim coming to grant, as I have never seen the user of a method becoming part of a patented method.

    As nobody knows what the original and new user instructions can represent, I see a major problem of sufficiency and/or of clarity.

    The user instructions are anything but trivial and are clearly determining in the method. Not specifying what those user input represent is thus not acceptable.

    The description might give more, but then an essential feature, the “user instructions” should be added in the claim! Without any explanations about the nature of the user instructions” in the description, there is a massive problem of sufficiency.

    One example might also not be enough to cover such a broad claim.

    What is valid for claim 1 is also valid for claim 1 to the system as I cannot imagine a user being part of patented system.

    I do not expect a lot of patents in the field of AI. It is clear that not only the model, but also the training data has to be disclosed. Data being an important asset in our day and age, it is difficult to imagine that plenty of applicants will be prepared to share this asset.

    ReplyDelete
    Replies
    1. As a follow-up on my previous response, I have checked OpenAI patent publications and it appears OpenAI does not hold published applications outside the US. This is consistent with the strategy depicted by Rose and the fact that the battleground is primarily in the US.
      For OpenAI, the key issue at the moment is not patents but copyright and trade secrets : strive to restrict as much as possible access to the data sets of copyrighted works they use for learning their software. See Hollywood Reporter 24 Sept 2024.

      Delete
  2. @Interested third party

    I fully agree with you as regards the EPO. As to the USPTO, it is actually surprising that the examiner granted a patent claim including such utterly indefinite terms as "user instructions".

    As to applications outside the US, the 18-month period from the priority date (13 March 2023) expires next week (13 September 2024). Thus we will know very soon if there has been any foreign filing.

    ReplyDelete

All comments must be moderated by a member of the IPKat team before they appear on the blog. Comments will not be allowed if the contravene the IPKat policy that readers' comments should not be obscene or defamatory; they should not consist of ad hominem attacks on members of the blog team or other comment-posters and they should make a constructive contribution to the discussion of the post on which they purport to comment.

It is also the IPKat policy that comments should not be made completely anonymously, and users should use a consistent name or pseudonym (which should not itself be defamatory or obscene, or that of another real person), either in the "identity" field, or at the beginning of the comment. Current practice is to, however, allow a limited number of comments that contravene this policy, provided that the comment has a high degree of relevance and the comment chain does not become too difficult to follow.

Learn more here: http://ipkitten.blogspot.com/p/want-to-complain.html

Powered by Blogger.