Trainee anguish as GP exams flounder

Introduction by Croakey: The Royal Australian College of General Practitioners had to apologise to trainees this week after its annual barrier exams had to be cancelled due to technical issues with the online proctoring system.

The College and Board said they were “deeply sorry for the significant distress and disruption” of cancelling the Applied Knowledge Test and Key Feature Problem exam and offered candidates full refunds.

“The KFP and AKT are the culmination of months of hard work, determination and sacrifice and we know what unfolded was devastating for you, your loved ones and colleagues who supported you during your exam preparations,” the RACGP said.

The Australian Medical Association said it was “the last thing that GP registrars needed” in a year of deep uncertainty and criticised the RACGP for failing to heed alarm bells during practice exams:

The AMA extends its support to the aspiring General Practitioners who have put their lives on hold, taken leave from work, and now no longer know when their training will finish.

While technology failures can happen at any time, we understand that similar failures occurred during a mock exam two weeks ago.

Medical exam administrators across all Colleges should heed this warning and adopt robust contingency plans should future technology failures occur and ensure more clear communication with trainees.”

In this piece for Croakey, Dr Sama Balasubramanian from peak trainee body GPRA explains how high-stakes these exams are, and that for some, their mishandling has been the final straw.

Sama Balasubramanian writes:

The abrupt cancellation of the Royal Australian College of General Practitioners (RACGP) Fellowship examinations, scheduled to take place last weekend, has galvanised a growing feeling of disenfranchisement amongst GP trainees.

Despite the numerous problems encountered in running trial exams of the RACGP’s Key Feature Problem (KFP) and Applied Knowledge Test (AKT), the College assured trainees that these exams were ready to go.

These examinations were, incidentally, the first of their type to be delivered by the RACGP online, as the College sought to enable its trainees to meet their assessment requirements during the COVID-19 pandemic.

However, 45 minutes into the KFP exam, the server of the external provider assisting the College to run the exam, crashed. Over 1,000 trainees sat in anguish, staring at error messages. Many could not even commence the exam.

In fear of being disqualified, many trainees did not touch their phones or access the RACGP website for hours. They sat there, anxious, bewildered, and frustrated. There was minimal communication from RACGP to explain to trainees what had transpired.

One trainee said: “I was panicking and in tears. I thought I had done something stupid and it was all my fault, and I wouldn’t be able to sit the exam.”

Phone calls to the RACGP exam support hotline rang out; many messages to the online provider seeking support were met with bizarre, poorly-worded sentences from overseas proctors (“soory sever isshu”, “wate a lonch botto” and “kingg pokas on the camera”).

Trainees who managed to receive a response were often given a generic message with little clarity.

Hours later, candidates were informed that the KFP exam would be cancelled, and, later again that evening, they were informed that the AKT, which was to run the following day, would also be cancelled.

Aside from a few social media messages and a brief statement on their website, the RACGP did not meaningfully communicate with trainees until Monday afternoon. There was no initial apology or urgent plans for reparations. When trainees needed assurance the most, there was silence.

All weekend, GPRA fielded calls and emails from disappointed, frustrated, and despairing trainees.

One trainee said: “I don’t know if I have the courage to continue studying or trying again for these exams. It has made me question whether being a GP is worth it.”

Another trainee said: “I took two weeks of annual leave for this exam…this is just too hard.”

The stories of despair and anguish are numerous. We need motivated, supported GP trainees to give Australians the primary healthcare system they deserve; but many of these trainees are now at their wits’ end.

Preparing for the exam

One can only begin to understand the trauma experienced by the exam cancellations when they understand the stakes involved.

Having completed the RACGP exams myself in 2017, I can say emphatically that these assessments are one of the most stressful undertakings I have experienced.

Firstly, trainees must pay almost $5,000 up front to enrol into the exam. This is on top of annual membership fees they pay to the RACGP.

Additional costs are often incurred for exam preparation courses and materials. Many take annual leave to study and sit for the exam as there are no provisions or payment for study leave for GP trainees in community practice. It is not uncommon to hear of trainees spending well over $10,000 in preparation.

Secondly, a typical trainee will spend between six to twelve months of intensive study to prepare. For many full-time trainees, this means studying before and after a full day of work and on weekends. Study time is often at the expense of time with family, friends, and leisure. With a limit of a maximum of six attempts, across three years, to pass these exams the pressure can gnaw at the psychological wellbeing of trainees.

Lastly, exam preparation must occur while the trainee continues to pass other training requirements, as well as manage the normal stresses of day-to-day work in general practice—and it has been an extremely difficult year for GPs and GP trainees, who are bearing an extra burden of uncertainty and psychological distress brought on by lockdowns and the COVID-19 pandemic.

Many of the exam candidates were expecting relief on Saturday, pleased to have done their best at these exams; instead they were faced with uncertainty, despair and anger—feelings which were largely unacknowledged by the RACGP until days after the exams.

General practice: a specialty in flux

These issues have added to the growing feeling that the RACGP does not care about its GPs in training.

While the disenfranchisement of a cohort of trainees is not healthy for the profession, it is understandable in light of the mounting issues faced by GPs in training — suboptimal employment and training conditions, uncertainty around the delivery of the GP training program — and an unwillingness of key stakeholders and government to listen to GP trainees, and work together to create positive change.

For many years GPRA has sought to achieve better employment and training conditions for GP trainees. This includes basic entitlements such as paid parental leave and portable annual leave within the health system — a privilege enjoyed by many of our hospital-based counterparts, and other working Australians.

Additionally, registrars in general practice often earn less than as a junior doctor working in the hospital^, despite often having a higher level of training, independence, and responsibility.

With all these challenges, there is no surprise that there is a rapidly decreasing number of applicants to the major Government-funded pipeline that supplies our future primary healthcare doctors, the Australian General Practice Training (AGPT) program.

In 2019, just over 1,300 of the 1,500 places in the AGPT program were filled. A continuing shortfall of GP trainees will undoubtedly impact on the future primary healthcare workforce and potentially have detrimental effects on the future health of Australians.

These effects will likely be more painfully felt by our rural and remote communities, who tend to be most impacted due to the historical burden of difficulty recruiting into these settings.

Australians deserve high quality healthcare no matter where they live, and that is the kind of healthcare that GP trainees deliver.

Not all GP trainees are able to access the AGPT program, including large numbers of International Medical Graduates (IMGs), who often undertake GP training with limited supervision, a lack of organisational oversight, scant access to education and support, and in rural and remote settings with little social supports. For this cohort, their visas are often tied to the passing of their GP exams.

Unfortunately, exam cancellations are the icing on the cake of the issues GP trainees have been endured during their training experience, and it has left a bitter taste in their mouths.

Some trainees have told GPRA that these exam cancellations were the last straw — they are withdrawing from the GP training program. They have had enough. For some, this is due to stress, for others, this is due to the financial pressures.

Trainees deserve much better

Medical College examinations are high-stakes activities which have enduring ramifications for candidates. They also provide assurances to the Australian community that their doctors have achieved the expected high standard.

This process, however, entails a responsibility that the Medical Colleges, as training bodies tasked by our communities in delivering these specialist doctors, keep up with their commitments.

As such, more needs to be done by the RACGP to make reparations, beyond the already-announced refunds and free resits.

Trainees deserve to study for exams in the knowledge that these assessments will run smoothly and successfully, without the worry that the online exam system will crash mid-way through an answer.

Technical problems in the delivery of paperless and online examinations are not new to the sector.

The failures in the 2018 online delivery of examinations by the Royal Australasian College of Physicians (RACP) was a harbinger to Medical Colleges that proper contingencies are essential when using these exam delivery platforms.

Similar problems were seen in delivery of online exams by the Australian College of Rural and Remote Medicine (ACRRM) this year, another reminder of the risks.

The RACGP’s promise to undertake an external investigation of the incident is encouraging.  There must also be a firm commitment that this investigation will be open and transparent, with findings publicly released.

Lessons must be learned, and this disaster must not be repeated.

If the RACGP is truly committed to an independent review it must seek input from GPRA, the only independent GP trainee organisation, and not just rely on consultations with its own internal faculties. Anything less will be viewed as obfuscation and sweeping the issue under the carpet.

What the RACGP does going forward to restore confidence amongst, and show respect for, its future Fellow GPs is of utmost importance.

As for change within the broader general practice training sector, this is a matter of all stakeholders leaving their silos, putting aside their differences in Colleges, politics, and allegiances, and coming to the table to create bipartisan change.

GP trainees, the patients they care for and our communities they serve deserve no less.

Sama Balasubramanian is President of GPRA, the national peak body for General Practice trainees, medical students and prevocational doctors interested in general practice