Last week the European Qualifying Exams (EQEs) for patent attorneys were held online for the first time. The EPO has now released a statement, declaring the digital EQE a "success". The considerable achievement of getting the eEQE up and running in time for the exams this year should be recognised. However, the EPO's positive summary is at odds with the reported experience of many candidates during the examination week, and has been criticised by some for not being sensitive to candidates' concerns. The EPO's message on the first eEQE also came before the window for candidate feedback had closed.
"Smooth" is a relative term
From the EPO's perspective the new format ran "very smoothly". However, there were reports of software issues throughout the week, which are not mentioned in the EPO statement (as summarised over on the Salted Patent blog). It seems the LockDown Browser and WiseFlow were both clunky and temperamental, particularly at the stage of submitting your answer. To this Kat, it seems that the system is very much a work-in-progress as opposed to a final product.
 |
| EQE candidate at the end of examination week |
The EPO acknowledges the issues with Paper D1.1. This paper was initially only available to candidates in German, which was not much use to most English and French speaking candidates (IPKat: A chaotic start to the eEQEs). According to the EPO statement, the issue with Paper D1.1 "was solved quickly within a few minutes". Unfortunately, gaining access to the missing papers required candidates to refresh the page, contrary to the previous instructions given to candidates that they should avoid refreshing the browser during the exam (User Guide, 6.4.2). Many candidates therefore waited until they received instructions from their invigilator to refresh the browser. However, the invigilators were clearly overloaded by requests, and it was another 30 minutes before some candidates reported gaining access to the correct paper. There have also been reports of some candidates, not knowing what else to do, closing the system, attempting to log back in, and failing to do so (and thus being prevented from taking the exam).
On the day after Paper D, the Examination Board released a 3-line statement that it is "aware of a disruption affecting paper D1.1 and guarantees that no candidate will be disadvantaged as a result during the marking process". Part of the challenge will be addressing the spectrum of different problems experienced by candidates: Some candidates were able to access their paper straightaway, some within a few minutes, some within half an hour and some not at all (e.g. if they made the mistake of logging out of the system). Extra time was awarded at the end of the paper, but some candidates were then locked out of the system prematurely.
Another complication is of course the ongoing stress and anxiety that the worst affected candidates would have experienced. Following Paper D1.1, there were 2 more papers to sit that day. The deadline for reporting issues with the exams was also the end of the same day. For some candidates, the evening would therefore have been spent preparing and sending this report instead of preparing for the next day's exam.
For candidates who submitted comments to the Examination Secretariat, the EPO has issued a notice that these will be forwarded to the Examination Board (with no individual acknowledgement of receipt).
Lessons to be learnt
The EPO should be thanked for putting in a system that at least enabled candidates to take the examinations this year. The EQEs were cancelled at short notice last year, just as the true scale of the coronavirus pandemic was becoming apparent (IPKat). A digital EQE was certainly more preferable than further delay.
A digital EQE also has its advantages. At the very least, the environmental benefits of not forcing candidates to print a suitcase worth of case law, guidelines and legal texts can not be ignored. A digital EQE also avoids the stress, time and cost associated with travelling to the exam venue (the chill of Walsall football stadium is probably not going to be missed by many), not to mention the peculiar oddity of ensuring your exam scissors do not have "blades longer than 6 cm (as measured from the fulcrum)".
However, in this Kat's opinion it is also important for the EPO to acknowledge the issues with the eEQE this year, if they are to avoid the same issues in the future. Comments from candidates (see e.g. DeltaPatents) reveal a great deal of dissatisfaction with the system and how it was run. The EPO's statement does not acknowledge or address these concerns, and downplays the impact of the particular issue in Paper D1.1. It is also apparent that the examination papers (particularly Papers B and C) were even more challenging than usual this year, suggesting that they had not undergone the same scrutiny as in previous years, and included a number of errors (e.g. mis-numbered questions). This Kat will be monitoring developments and looks forward to seeing how the EPO will address these issues.
Reviewed by Rose Hughes
on
Sunday, March 14, 2021
Rating:
Follow the IPKat on LinkedIn 
![[Guest post] Thou shalt not discriminate the Lego block over any other kind of product – GRUR speaks out on C-211/24](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWJYQUJNvYydrZcjwSHPuxd14xX3_48aBusHKBogY-Wg1BPSJSABbFFjmGiUV3aqG2npP9xYaIl1vtWK5AsqGOQPgRD2TWFLhyyc4xGm1fHDD-tt9kHcrSSgAyju2P9wSKz0VW6h7D5MD4Hxd9vREyxKYzlMRNXxxtxc3DiNsXppfVaxgSxAMV/s72-c/21742619125_7c9e53cd39_o.png)
![[Guest post] Comparison of wine and gin: the EUIPO’s unquestionable Similarity Tool](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNfTEBXiPVS0XjZi4mNE1OtXLxXiEp1PUgcUK7MMMXlfyTkO5SMXv-VCMXXNdS_F1XdMMxYca1VbBTGvvDeTVpo5x-5YwR7atQJmK3x32T8LACpf4308kht3nEtaxPwEPcGGrB0s5xjGktmQPOiVwPVLRO5JbW19qupc7xUtiVutvaWHow0OpBkA/s72-c/Picture%201.jpg)
![[Guest post] Muddying the waters of copyright – a rapid reaction to today’s IPEC decision which sees the sinking of the WaterRower’s claims for copyright protection](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7chFANC7ppmZl4_nlR6_hZEcg6cOhVEJNVXqki4p82BoVlcLFotuOXzHWYD5lKLiiK_kRoWeXmJjjaV2DpoKsAWq8U3LFioO5W_841a9gcfDIFjvZWcGqfnr0jWJNtyXptIYR_8bxCssaDkJ9KMokuSqy1g65Il7p2hPwo4MfRCEGoUPQJesvbg/s72-w400-c-h189/Screenshot%202024-11-11%20at%2018.20.45.png)
Posts
It is the opposite of "smooth". Chaotic and confusion comes to mind. For paper D - candidates had no access to EN or FR version for 30 minutes. They started to randomly add times 10 minutes towards to the end of D1, shorten breaks for other parts. The language issued appeared again in the 2nd part.
ReplyDeleteFor paper A, it was full of contradictory and confusing statements.
Paper B was in my view not doable in 3.5 hours. The formatting of claims were horrendous. It took me 30 minutes to copy claims over to the text editor and correct formatting. We then had 3 independent claims, 3rd party observations and a list of errors from missing figure labels to NOT indicating amendments in the client's amended letter.
Paper C was also a horrendous exam online. The split created a host of issues and the paper became very unbalanced. Further, the claim formatting was also a major issue. It seems that the committee did not sufficiently consider the split and adapted the paper appropriately.
NO where near smooth. They are on another planet. Every candidate I spoken to had issues with the eEQE. Candidates are NOT against online exams but not with this current system. It was far too complex for no reason.
ReplyDeleteIts actually quite insulting to suggest that the system was running smoothly. They didn't even wait for candidates to provide feedback via the EPO feedback system before declaring major success. Candidates feedback and opinions are ignored or brushed aside as always.
ReplyDeleteIts clear the papers this year were not adapted at all for the e-EQE online format, particularly papers B and C. Paper A was not well written either. It does led me to believe that checks were not sufficiently carried out with the papers this year.
ReplyDeleteThe EPO has shot themselves in the foot. I'm glad IPKAT have provided an article on this because the EPO is trying to make it sound like everything was fine to everyone else in the profession.
ReplyDeleteThe truth is that the wiseflow software is not suitable for e-EQEs. You can't highlight, compare documents on the same page, can't copy without losing formatting, not allowed to open more than 9 tabs or the software will crash, can't use common functionality on keyboards such as ALT+TAB. Allowing parts to be printed but not others?
We had 3 mocks (all very late) and as far as I can tell, not many of these were fixed.
I want like to know what the EPO think is "smooth" about this year's format.
I was hugely affected by the language-gate in D1. It carried through to the next day in paper A as I was so anxious and worried that night. The EPO didn't release a statement until the day after to acknowledge their blunder.
ReplyDeleteA very embarrassing statement by the EPO.
ReplyDeleteThere are some positives doing it online but I'm afraid to say that my experience has been largely negative with the current eEQE system. They need to keep the system simple but the EPO has managed to make difficult year for candidates into a horrendous week.
PR gone wrong.
ReplyDeleteThanks to the EPO PR department as this will be a classic textbook study for future candidates taking PR lessons.
It was not just Paper D. I did not have access to Paper B in English for several minutes.
ReplyDeleteWhen you have experienced tutors struggling with the papers this year i.e. papers B and C, it appears to me that these papers were not adapted properly for online.
ReplyDeleteThere are also numerous IT issues like wiseflow crashing and printing issues as well as the the language problem in D1. Its far from "smooth" and I would say the majority of candidates would consider it a "rough" experience.
A disappointing statement by the EPO which clearly does not resonate with candidates and even tutors' experience of the system during the EQE week. They should take down that ridiculous statement.
ReplyDeleteCan't help but laugh that they think the language issue was only a few minutes. It took 30 minutes for many candidates to resolve - that's 30% of time available in D1.
ReplyDeleteThey didn't even mention about those candidates who got booted off the system randomly despite being promised the extra 30 minutes. Where was that in the statement?
With this press release, I fear the EPO will disregard many of the concerns candidates have legitimately raised about the software
ReplyDeleteAbsolutely disagree that the e-EQE format ran smoothly. What were they thinking releasing a statement like this when it is not the truth at all.
ReplyDeleteIt is a wholly inappropriate statement to make. My firm and myself will shortly be submitting a formal complaint to the Exam Sect about the lack of adaption of the exam papers as well as the wiseflow system in general.
ReplyDeleteI think the examination process was well organized and the EQE board did their best as usually. Every year all involved witness a comparable amount of anger and bold statements.
ReplyDeleteThere is no doubt that they did the best they could. It was not an easy task. However, stating that the eEQE was a success is simply not true and quite frankly an insensitive and disrespectful statement towards this year's candidates. The software was not suitable for online patent exams. Simple functions like strikethrough, which I am pretty sure a patent attorney uses daily, were not available, not to mention that highlighting in the pdfs was also not possible and neither was annotating. The side-by-side view was far from perfect and it was very time consuming to navigate from page 1 to page 12 as one could not just scroll down as in a "normal" pdf, but had to scroll down to the bottom of the page and then click to get to the next page (and repeat for every single page). The formatting in the editor was completely lost if one wanted to copy a passage from one section in the editor to another. All the issues reported during Mock 1, 2, and 3 clearly showed that the system was not ready, but nothing was done to fix it. One can only hope that a different software will be chosen for next year's EQE and also that the exam papers will be properly tested under exam conditions using the software before deciding to use them as exam papers.
DeleteI'm not sure saying the e-EQE running smoothly this year is accurate at all. The DI situation has never happened before and papers A, B and C seemed like they have been through proper checks. There were lots of errors and mistakes in the papers.
DeleteI don't think it is an easy task but equally, they had a whole year to sort this. The PEB exams were held in less time and more successfully. It is simply down to the online format being much simpler. Why create an over-complicated system where both parties (EPO and candidates) will only suffer.
papers A, B and C seemed like they haven't been through proper checks. There were lots of errors and mistakes in the papers
DeleteIt is absurd to suggest that things were smooth this year. Perhaps they need to talk to candidates and invigilators before issuing such ridiculous statement.
ReplyDeleteI agree with others. The system was clearly not ready to go online. Papers were not sufficiently adapted to the new eEQE format - that is clear this year. What is disappointing is that the EPO had 1 year to sort it out. The week was far from ideal for many many candidates so it cannot be labelled as a success at all. I didn't expect the EPO to release such a false statement.
ReplyDeleteI'm grateful we can do the exams online. I'm not grateful that the EPO has decided brushed aside candidates painful experience, concerns and feedback with this press release. I don't think they will take anything into account this year judging by this press release.
ReplyDeleteI sat all four papers for the first time, having been unable to last year due to the late cancellation.
ReplyDeleteI am thankful (I think- stress has been unnecessarily high) for being able to sit the exams without another year's delay.
However, the system and adaptations were wholly unsuitable. Limiting myself to technical problems outside my control, I had four technical problems related to the software in spread over the four exams. I, along with other candidates, took the considerable time to attend the three mocks and try to digest the huge amounts of material and contradictory statements in the lead up to the exams. Thus, the technical problems I experienced cannot be considered my fault.
My experience of, on average, one technical problem per exam is not at all unusual, based on slacking with several fellow candidates.
This is all before one comments upon the inadequacies of the system when actually working and the dodgy adaptation of the papers.
The self-congratulatory statement from the EPO is a final kick in the teeth and yet another example of the poor communications from the EPO throughout the set up and conducting of EQE 2021.
Nobody is saying that they do not want e-EQE online. Indeed, I too didn't expect everything to go well but the start of the week really was a bad start. Papers B and C were then clearly not fit for the current e-EQE online format. I would throw in paper A into that mix too. I didn't expect any press release to happen by the EPO but to release a statement to suggests things went well does NOT reflect what actually happened during the week.
ReplyDeleteBecause they seem to have started from the quoting that all candidates will cheat (i say this based on the tinge and content of communications and the first webinar)
ReplyDeleteI'm thankful that we have been told to email the exam sect, who will then forward onto the exam committee, but will anonymity be respected?
ReplyDeleteThe last thing I want is to write something which will be used against me.
At least set up a review. I don't believe many people in the EPO thought nothing was wrong with the new eEQE format. There were positives but there are also clearly more things that can be done to improve.
ReplyDeleteOf course candidates are very grateful that the EQE went ahead, but "doing your best" should still guarantee a certain minimum quality. It is a professional qualification, critical for future careers.
ReplyDeleteI am sure the D1-part 1 issues will be taken into account.
But no-one who can state with a straight face that Wiseflow, LD Browser and Zendesk was a good solution. It was cobbled together in the last weeks (Zendesk invigilators were not allowed to speak because it would be registered as "second voice" in the room by the AI, and candidates had to check every 20 mins whether they were still logged in to Zendesk due to the automatic timeout). The info on the website was updated just days before the exam.
If the individual struggles of candidates are also taken into account in the marking (also "doing their best"), then it does not matter what the official press releases say. But flexibility should work both ways.
I think its clear the committee will need to be flexible and apply much consideration this year. Whether they do this or not remains to be seen. As you've mentioned already Ned, there are far too many factors that could influence the exams this year. Further, the papers do not really seem to fit well with the current online format.
DeleteActually, the D1-1 was "just one" and the most apparent of the drawbacks. I believe that the structure of this year's be and even more the split paper C, have been the worst issue ever. I completely failed paper C after so much studying due to lack of time, which did not allow me going properly through the documents. Further, I must admit that the online exam is extremely stressful (you feel drawbacks just around the corner) and fatiguing.
ReplyDeleteAgreed about paper C and also paper B and to a certain extent, Paper A as well.
DeleteFor B and C, the claim formatting was so difficult to work with - it introduces errors and mistakes as the formatting was absolutely unbearable at times.
My opinion - they need to scrap wiseflow system for future years. Its unworkable for these exams.
ReplyDeleteWould we have been April 1st, I would have taken EPO’s publication on the “success” of the e-EQE as a nice April’s fool joke, but this publication is adding insult to injury.
ReplyDeleteBy no ways the e-EQE in the form it was proposed this year can be a true equivalent of a paper EQE. Unless the EPO proposes a system in which offers the same possibilities as any standard text processing program, any kind of e-EQE will not be equivalent to a paper EQE.
The decision to cancel the EQE was taken in March 20, the decision to go over to an e-EQE in July 20. This was part of the strategic orientation of the present head of the EPO. Everything has to be digitised to the maximum. An e-EQE is as such a good thing, but not the e-EQE which was offered at the beginning of March.
Why did it take till shortly before Christmas to offer a first opportunity to the candidates to test the system? There were then two further test possibilities before the exam, the last one just a week before the actual exam. This is not acceptable.
What did the people in charge of the e-EQE think when they sent out at the beginning of February a user’s notice of 90 pages? This should have been done in September 20 at the latest. It simply shows the amateurism of those involved. Using a text processor in which the most basic key combinations ends the candidate to lock the system or even to exclude himself is not acceptable.
That not all possible candidates participated in the mock is not necessarily to blame them. Lots of candidates hoped to use company computers, but this was not possible as very few IT managers were prepared to give EQE candidates administrator’s rights. Some needed to get equipment like larger displays or even cameras.
The papers used were the papers foreseen for 2020. The suggestion was made to use the 2020 papers as mocks. In view of the considerable amount of costs needed to draft a paper this was a no go. Paper C had to be split, but the split seemed quite arbitrary and not thought through.
Normally each paper is benchmarked before it is let on the candidates. In view of turn of events, one can have strong doubts that this was done with the present e-EQE.
Everything was rushed in the two months before the exam. And trying to sell this as a great success shows how far some people at the EPO have lost touch with reality.
The previous comments were clear enough to show what reality was. The gap between reality and the fiction described by the EPO is awesome.
I would add that according the EPO, the quality of the products it delivers has steadily increased since 2010. Why should it be different with the e-EQE?
For those who are seasoned observers of the EPO, it is no surprise that their management would decide to issue a self-congratulatory statement that others might find difficult, if not impossible, to reconcile with their understanding of events.
ReplyDeleteThe difference on this occasion is that there are many first-hand witnesses that can testify to the fact that, all things considered, the EPO's statement is complete BS. With this in mind, it is an illuminating exercise to consider how much weight to afford to statements from the EPO in circumstances where it is much more difficult to obtain independent evidence to either prove or disprove the veracity of the EPO's reporting.
It was clear that the papers were not adapted or adequately checked. There were alot of mistakes in the papers and plenty of confusing/contradictory statements across papers A, B and C. That's not usually the case at all.
ReplyDelete