Full speed ahead for DeepMind's AI patent applications

AI art

Last summer, IPKat reported on DeepMind’s published patent applications directed to broad aspects of machine learning technology (IPKat post here). The post received considerable interest from the artificial intelligence (AI) community (see full discussion here). Since then, the number of published DeepMind patent applications has been steadily rising. A full list of their published international patent applications (now numbering 25) can be seen here. For some of DeepMind's earlier applications, the international search and examinations have also been published - giving an indication of the scope of patent claims for which DeepMind may achieve grant.

Patent basics - What are "international patent applications"

Patent applications must be granted before they can be enforced. To achieve grant, an applicant for a patent must convince the patent office that the invention for which they seek protection is novel and inventive. Some types of subject-matter (e.g. computer programs per se) may also be excluded from patentability.

Applicants wishing to protect their inventions in a number of different countries will often choose to file an international patent application. The applicant can use the international patent application to enter the national phase in their chosen jurisdictions (e.g. Europe, US, China, Japan etc.). In the national phase, the applicant must then convince the national patent office of each jurisdiction that the application satisfies the requirements for patentablity in that jurisdiction.

The international application allows the applicant to delay the choice of the jurisdiction in which they wish to pursue patent protection (including the costs associated with doing so). An international search will also be carried out, giving the applicant an initial idea as to the prior art that may be cited against the application by the national patent offices. Applicant's can also optionally request international preliminary examination (IPE), to get an idea of the objections to patentability that they may face in national phase.

DeepMind requested international preliminary examination for a number of their international applications. The international preliminary examination reports (IPER) have now been published.

So what is the preliminary verdict on the patentability of DeepMind’s applications?

WO2018048934 (Generating Audio Using Neural Networks, see full file here) has been deemed novel and inventive by the international preliminary examination authority (IPEA): see here, page 2. The Examiner's reasoning can be seen here. The Examiner observed that the claimed method had the technical effect of decreasing the computational requirements to generate audio waveform compared to previously known methods such as CA 2810457 (D1). D1 is a Canadian patent application relating to a convolution neural network (CNN) for speech recognition filed by Gerald Penn et al. from the Department of Computer Science at the University of Toronto. The Examiner indicated that arriving at DeepMind's method would have required a skilled person to "drastically [depart] from the paradigm used in D1.

WO2018048934 may therefore be expected to be granted by the European Patent Office (EPO) without the need for DeepMind to make any further substantial amendments to the scope of the claims. The deadline for this application to enter the European Regional phase is 6 April 2019. If no additional issues are encountered, DeepMind may be granted the patent within a couple of years.

WO2018064591 (Generating video frames using neural networks, see full file here) has now entered the EP regional phase, i.e. it is now in the hands of the European Patent Office. DeepMind took the unusual step of entering the EP regional phase early (5 months before the January 2019 deadline). The majority of applicants wait until the deadline for entering the EP regional phase. Applicants generally request early entry when they wish to achieve a quick grant. Given that DeepMind has purportedly indicated that their patent strategy is defensive, why the hurry?

In the international phase the claims were found novel, but lacking in inventive step in view of the ArXiv papers Junhyuk et al. (D1, also published at NIPS 2015) and Oord et al. (D2). DeepMind's claimed invention uses a Pixel-RNN to predict the next frame in a video sequence.  The Examiner argued that this was obvious in view of prior art describing the pixel-RNN for static images (D2), and the problem of predicting the next frame in a video (D1). Particularly, the Examiner argued that the technical effect of DeepMind's claimed method is that "differences in spatial dependency between the pixel in a frame, ordered according to the pixel and channel order, is taken into account for the generation of the predicted next video frame, thus improving the naturalness of the predicted video frame". The Examiner argued that a skilled person, confronted with the problem of how to improve the naturalness of the predicted next video frame in view of Junhyuk et al. (D1), would find Oord et al. (D2), and that D2 would provide the necessary technical details for solving the problem, and even hints as having already done so in the abstract: "achieve log-likelihood scores on natural images that are considerably better than the previous state of the art".

On entry into the EP regional phase, DeepMind responded to the Examiner's objections (see here). Particularly, DeepMind argued that the Examiner has formulated the technical problem to be solved using hindsight, and that a skilled person would have no incentive from Junhyuk et al. (D1) to seek out Oord et al. (D2). Particularly, DeepMind argued that D1 does not relate to natural images, but instead to computer-generated game environments. Thus, a skilled person would not seek out D2 given D1, and for a skilled reader of D1 'to identify that "naturalness" of predicted images is desirable in the context of D1 (if it is) would itself be inventive'.

WO2018048945 (Processing sequences using convolutional neural networks, see full file here) in the international phase was also found novel but obvious/non-inventive in view of Canadian patent application CA 2 810457 (also cited with respect to WO2018048934 above). The Examiner's reasoning can be seen here. The Examiner argues that the technical problem solved by DeepMind's claims in view of D1 is how to allow the CNN of D1 to perform state-dependent sequence-to-sequence processing. The Examiner argues that the generic purpose of "performing state-dependent sequence-to-sequence processing" is not considered a technical purpose (i.e. providing a technical effect), and thus cannot be considered as providing the technical basis for an inventive step. To achieve grant, DeepMind will therefore have to either convince the national patent offices that the currently claimed invention would have been non-obvious to a skilled person in view of CA 2 810457, or limit the scope of the claims . The deadline for WO2018048934 to enter the European regional phase is 6 April 2019.

Coding Kat

It is worth noting that even if the European patent office considers an application novel and inventive, third parties may file observations to the contrary before grant, or oppose a granted European patent (for 9 months after grant). It will be interesting to see whether DeepMind's applications receive any such opposition.

Given the apparent relevance of DeepMind's patent to ML research, it is also worth noting that many jurisdictions exclude academic research from patent infringement. The UK Patents Act, for example excludes as infringing acts, "acts done privately and for purposes which are not commercial" (UKPA, Section 60(5)(a)).

This Kat will watch the further progress of DeepMind’s growing portfolio of patent applications with interest.

By Rose Hughes