Can academic peer-review learn something from patent prosecution?

All patent attorneys used to be scientists. Many of us hold a PhD. Working closely with scientists may lead us to wonder what life would have been like had we chosen to stay in science. However, for this Kat at least, nostalgia for the lab is quickly quashed by any exposure to the peer-review process. Patent prosecution, with its to and fro of structured, fair and reasoned argument can be an immensely satisfying process. Peer-review, by contrast, seems almost purposely designed to frustrate and dampen the spirit of scientific endeavour. However, both peer-review and patent prosecution have similar ends, namely determining whether a patent or scientific manuscript makes a meaningful contribution to the art. Perhaps then, peer-review could learn something from patent prosecution?

Patent prosecution and peer review

Both peer-review and the patent prosecution, of course, have their problems. Peer-review is considered, to paraphrase, the worst system for reviewing academic papers except for all the others. However, the comparison between peer-review and patent prosecution puts many of the problems of peer-review into sharp relief, and perhaps sheds some light on why science struggles to retain those from unrepresented groups.

Patent prosecution is a codified system in which professional patent office examiners determine whether a patent should be granted. The examiner assesses the claims of the patent application for novelty, inventiveness, and utility, as defined by the patent law. If the examiner raises an objection, the applicant is given some number of months to respond to the objection, either by arguing their case or by amending the claims. The to and fro between the applicant and the examiner may go on for many iterations, before final acceptance or rejection.

Peer-review is similar to patent prosecution except that the place of the examiner is taken by experts in the field, who are presumed to be in the best position to assess the truth, originality and significance of the research. Manuscripts submitted to a journal by an author are sent by the journal editor to expert academics in the field to review. Like the patent examiner, the peer-reviewer passes judgement on the merits of the paper and recommends publication, potentially subject to revisions, or rejection. The author is then invited by the editor to respond to the reviewer's comments and to make the suggested revisions. The editor takes the final decision on whether the paper should be accepted or rejected based on the reviewer comments and the author's reply.

Before beginning the comparison between peer-review and patent prosecution, it is noted that academic purists may be scandalised by any attempt to equate the world of peer-review with the commercial world of patents. However, such critics may be reminded that published papers are the currency of science, on the basis of which grants are awarded, jobs offered, and promotions given. A published paper therefore has economic value to its authors, albeit a more indirect one than a patent. The process by which papers are accepted and rejected should thus be as arguably fair and robust as those by which a patent is granted.

Why anonymity in peer-review matters

Science can be stressful

In patent prosecution, the identity of the applicant is known, as is the identity of the professional examiner. By contrast, peer-review relies on anonymity, or a least a pretence of anonymity. Ideally the reviewer should thus not be influenced by either their prior knowledge of the author, or by the fact that the author will know their identity.

However, in practice, true anonymity in peer-review is rarely achieved. Many authors self-publish their papers on arXiv before submitting to a peer-review journal. Reviewers can thereby easily check the author's identity. Even without online pre-prints, it is usually clear to anyone who knows the field from whom a paper originates, given that an author will often build upon their own previous work. Reviewers will also be keen to bring the author's attention to the reviewer's own work for citation, but by doing so, will unmask their identity.

So why does it matter if the identity of the author is clear to the reviewer and vice versa? Peer-reviewers, like patent examiners, are supposed to provide objective comments on the scientific merits of the work. However, a reviewer's objective judgment may be clouded if they suspect the identity of the author. Reviewers are likely to look more favourably on papers originating from eminent figures in the field. Reviewers will also be unlikely to advise rejection of papers originating from high profile companies or university departments at which they may be interested in future employment. On the flip side, reviewers may be prejudiced against certain authors with whom they have professional (or heaven-forbid, personal) disagreements. The peer-review system therefore provides the potential for personal and/or professional biases to cloud a reviewer's assessment of a paper.

By contrast, in the patent system, the identities of the examiner and applicant are known to each other. Anonymity is not required because examiners are not potential competitors of the applicants, and do not work in the field. 

Peer-review operates on vague rules

The problems that may occur because of the lack of anonymity in the peer-review process are compounded by the lack of clear, codified criteria for assessing a paper for scientific merit. This lack of clarity contrasts with the patent system, in which novelty and inventiveness is assessed according to objective tests. 

Peer-review should assess a paper in a similar way to how a patent examiner assesses the claims of a patent application, i.e. does the research relate to something that is new, non-obvious and useful? Depending on the journal, peer-review should also assess the potential impact of the research, i.e. is it something that a lot of people in the field would be interested in. However, in contrast to patent prosecution, the assessment criteria for much of peer-review is left vague. There is no definition of what it means for research to be non-obvious or impactful. This is left to the discretion of each and every reviewer.

Worse still, peer-reviewers are not, unlike patent examiners, trained review experts. Peer-reviewers are encouraged by editors to produce long reviews. The result is that peer-reviewers will produce reams of nitpicking comments (for example, on writing style, figure format and terminology), whilst not addressing the critical question of whether the paper is making a important contribution to the art. The peer-review system is thus one in which reviewers may be influenced by personal and professional prejudices that are not curtailed by a codified system of assessment to which the reviewer must abide. Again, this contrasts with the patent system, in which the rules and criteria are well-established, relatively consistent and known to both the applicant and the examiner. 

Peer-review provides only a limited opportunity to reply

Like the patent system, the peer-review system permits authors to respond to a reviewer's objections. The time given for response varies from months to a few days. There is often a word limit for response (e.g. limited to only a page) and authors may not be permitted to submit a corrected manuscript. Furthermore, reviewers will often ignore the author's response, in which case the judgment of the original review stands. By contrast, patent prosecution not only permits but legally requires that an applicant be given the opportunity to respond to an examiner's objections (EPO Guidelines for Examination, B. Right to be Heard).

The patent system of professional examiners and codified rules thus removes a lot of the problems associated with peer-review. Patent objections are clearly defined by the law, as is the method and processes for redress. Compared to the peer-review system, patent applicants have relative clarity as to the criteria the application must satisfy, and the tests that should be used for assessment. Having professional examiners also largely removes ego and potential conflicts of interest from the equation.

The ugly side of peer-review

To be a scientist, you need a thick skin. Rejection and failure are part of the daily grind of science, whether it is the failure of experiments or the numerous rejections of manuscripts and grant proposals that must be endured before (hopefully) final success. The uncodified process of peer-review exacerbates this problem. The propensity of reviewers to be both excessively critical of inconsequential details of the paper whilst simultaneously vaguely casting doubt on the authors general capabilities as a scientist, can seriously dampen a scientist's self-esteem. It is therefore of little surprise that unrepresented groups, who may lack self-confidence, are more likely to leave science than those from other groups. The patent system has its own problems, but at least is a far more impersonal system than peer-review. 

Perhaps then, peer-review could learn something from patent prosecution. The patent system and peer-review have similar aims. The purpose of patent prosecution is to establish whether an author's work should be awarded the credibility of publication in the journal, effectively a green tick from the journal that work is new, non-obvious and useful. The purpose of patent prosecution is to establish whether an applicant should be awarded an IP right for a new, non-obvious and useful invention. The patent system demonstrates that it is possible to objectively test for concepts such as inventiveness and utility. Why then could the peer-review system not use similar tests? A more formalised template for analysis in peer-review would help to remove ego from the equation and minimise the time spent on extraneous matters.

Reading some example peer-reviews (courtesy of Mr Kat) renders this Kat eternally grateful for the patent profession. Thankfully, she can leave behind the peer-reviewers and return to her task of explaining politely but firmly why an examiner should grant a patent according to the established principles of patent law. If the academic peer-review process could only adopt a few of the practises of patent prosecution it could perhaps become a similarly fair and rational process.
Can academic peer-review learn something from patent prosecution? Can academic peer-review learn something from patent prosecution? Reviewed by Rose Hughes on Wednesday, September 16, 2020 Rating: 5

9 comments:

  1. Neither peer review nor patent examination is without flaws. As a patent attorney working closely with inventors in universities and small companies, it seems to me that some patent examiners could also benefit from a more realistic view of how the research process, and science more generally, works. It can be frustrating to face a patent examiner who has decided that they are more expert than the scientists actively working in the field of the invention. This attitude can present itself e.g. by putting absurd and unrealistic interpretations on the prior art that leave the experts baffled, insisting that terms are unclear when they are commonly used in the academic literature, and so on. The examination procedure then becomes a dialogue of the deaf with the examiner unwilling to listen to the opinion of the scientists and the scientists left bewildered or outraged by what they perceive as an inexplicable attitude or wilful misunderstanding. This happens in a minority of cases but a large enough minority to be noteworthy. In that aspect the examination procedure is perhaps worse than peer review, where for all its faults the reviewers are generally operating from the same baseline of knowledge and shared understanding of terminology as the authors.

    So by all means let us have a dialogue between the patent examination process and the peer review process. But we shouldn't assume that the lessons to be learned flow only in one direction.

    ReplyDelete
  2. What an interesting article. Thank you, Rose.

    Like you, I find the debate with an Examining Division or TBA deeply satisfying. By contrast, I have found the peer review process somewhat frustrating. Mind you, my only experience of it (as writer) is with a piece I wrote on an aspect of patent law, for a peer-reviewed IPR journal. The reviewer was positive about my piece but then added a comment which gave me doubts whether the reviewer had grasped accurately the content of my text. But as I had no idea as to the identity of the reviewer there was no opportunity to debate the issue with them.

    Naively, I had supposed that the peer review process is not "double blind". I had supposed that the reviewer is routinely given the name of the writer. You write that "ideally" the process is double blind but (at least in the case of legal journals) is it not normal for the reviewer to be given (by the publisher) the identity of the writer?

    And then there are the patent law blogs. The ones here in Europe do not attract lively comment threads. Different in the USA. On Patently-O or Watchdog, my comments on patent law in Europe can attract lively criticism, both from commenters posting under a synonym and under their real name. Those comments are sometimes quite revealing.

    I'm (more or less) retired. I wish more patent attorneys, younger ones, would join in the comment threads. But I guess they are too busy. A shame, i think. Do they not read the blogs at all? Or do they read them but then feel they have nothing to add? Or do they want to add something but feel less than adequately qualified to do so?

    But whatever the answer is, I'm optimistic that this high quality blog is read by many professionals within the "interested circles" and therefore generally a GOOD THING.

    ReplyDelete
    Replies
    1. A lack of time is certainly a factor, but my suspicion is that the perceived benefit is too low in light of the perceived risk. You could get in a lot of trouble with an employer for saying the wrong thing in a blog comment, so whatever you say better be worth it. Mostly there's very little benefit - better to keep your clever ideas for clients and your complaints for trusted confidants.

      Delete
  3. Another important difference between patent prosecution and peer review is the possibility to appeal. A more or less independent review of both the arguments of the applicant/writer and the examiner/reviewer has led in the patent world to harmonization and it is debatable whether the quality of the patent prosecution process is because of the (harmonization by) codification or because of the (harmonization by) the possibility to appeal.
    Further, I agree with MaxDrei that providing comments to blogs or other online communications should be encouraged. Indeed, in the US based blogs this is more common (but it may also become dominated by a few persons and/or become nasty, such as with the patently-O blog), although there are several US blogs (such as PatentDocs) that also receive hardly any comment. Why this is, is questionable. Are these blogs not read anymore (perhaps the younger attorneys are too busy with fee earning?) or are they being read, but are people to afraid of giving their opinion (as anonymous of 12:24 argues)? If the first is the case, the employers should realize that information gathering for their attorneys is essential to keep well aware of the ins-and-outs of the profession. If the latter is the case, employers should realize that if they want to have attorneys that are able to use arguments in their patent practice, argumenting in a polemic on a blog might be a good way to practice.

    ReplyDelete
    Replies
    1. Well perhaps it is just me. I only ever post anonymously as I don't see how the comments would benefit me professionally, but I can see how they might come back to haunt me! I've also never wanted to have a pseudonym as I don't trust it won't be traceable.

      So perhaps there are just a lot of silent readers? I expect IPKat's page view figures would have the answer there.

      Anyway this is all a off topic really.

      Delete
    2. Millipede, your comment prompts me to observe that the writer of this particular blog post, Rose Hughes is (if I understand it right) in house at Astra Zeneca. How about that then for an example of an enlightened employer? Here in this blog, Hughes writes first class items on EPO case law. She deserves high quality comment threads, I think. And her employer three cheers for giving her the freedom to contribute here.

      Delete
  4. All patent examiners used to be scientists too. Even us ex-examiners were scientists once. Some years ago I started writing an article with a patent attorney friend and colleague which we called “Publish or Perish. (Or Patent?)” It went very much along the likes of this Kat's blog. (We really must finish it one day)

    It was further intended to show that academics don’t need to perish but can publish and patent, so long as they do it in the right order.

    There are numerous examples of academics having published first, and then filed a patent application to find their earlier publication comes back to haunt them.

    Patent, then publish is the way to go - and both can be possible

    Even so there is a need to be careful.

    Many academics are not known for modesty. They will write “our observations suggested that…” “our calculations predicted that” etc….

    If they publish like this, even after filing the patent, they are saying, “it was obvious to try”, “it was obvious that we would succeed”.

    In other words the invention is obvious; not inventive, and a 3rd party could bring a revocation action and succeed!

    (Thanks to Dr Eugene Sweeney for that cautionary tale)

    ReplyDelete
    Replies
    1. Rose Hughes's employer is Astra Zeneca, currently busy working with the boffins at Oxford University to bring a C19 vaccine to market. Surely, Stanfield, all involved are on board with the idea that one tells the Patent Office before one tells anybody else. Keep an eye also on the CRISPR patent battles just developing, and how much disclosure there was during the American grace period before the respective patent applications were filed. I suspect there wasn't much.

      I suspect nearly all "academics" these days are well-aware of the financial consequences of not filing patent applications early. How about the perovskite story, patented by Isis Innovation (ie Oxford University)? Surprisingly big performance improvement, out of the blue, in the field of solar panel electricity generation. How much are those patents going to be worth?

      Delete

All comments must be moderated by a member of the IPKat team before they appear on the blog. Comments will not be allowed if the contravene the IPKat policy that readers' comments should not be obscene or defamatory; they should not consist of ad hominem attacks on members of the blog team or other comment-posters and they should make a constructive contribution to the discussion of the post on which they purport to comment.

It is also the IPKat policy that comments should not be made completely anonymously, and users should use a consistent name or pseudonym (which should not itself be defamatory or obscene, or that of another real person), either in the "identity" field, or at the beginning of the comment. Current practice is to, however, allow a limited number of comments that contravene this policy, provided that the comment has a high degree of relevance and the comment chain does not become too difficult to follow.

Learn more here: http://ipkitten.blogspot.com/p/want-to-complain.html

Powered by Blogger.