Medical Specialty Certification Exams: The Real Scandal

Note to our readers:

This article was recently published on The American Thinker website, where it can be found here.  We’ve added some additional comments at the end of the article based upon reaction to the original, and some additional thoughts and news we’ve had since publication.

——————————————————————————————————————————–

Earlier this year, CNN broke the story that generations of dermatologists have been “cheating” on their specialty board certification exams.  Like their colleagues in radiology and most other medical specialties, large numbers of dermatology trainees have been studying for the boards by using massive collections of historical exam questions that were memorized and shared immediately after each year’s test.  Having access to these questions is useful because many specialty boards re-use questions from previous tests.  Until this year about 20% of the questions used by the American Board of Dermatology (ABD) were recycled from previous years, compared with about 50% for the written exam in radiology.

The widespread use of these so-called “airplane notes” or “recalls” has been common knowledge in the medical world for decades, but the flurry of recent press reports has caused consternation in the powerful board certification industry.  Stung by the publicity, the American Board of Medical Specialties issued a press release condemning the practice and calling it an unethical violation of copyright laws.  Apparently caught by surprise, the executive director of the ABD admitted that she’d “never seen airplane notes, but I’ve heard about it”, and subsequently cancelled a scheduled on-camera interview on the topic.  Meanwhile, the American Board of Radiology hurriedly announced that its 2012 certification examination will be composed of entirely new questions for the first time in ten years.  But while the lay press often frames the issue as one of “public safety”, the real scandal surrounding specialty board certification is quite different.

Modern specialty boards are holdovers from a time when there was little standardization or regulation of medical training.  The American Board of Dermatology was founded in 1932, a time when various programs to train dermatologists ran for either one, two or three years.  This era ended for good in 1981 when the Accreditation Council for Graduate Medical Education was established to regulate and accredit residency training programs for all medical specialties.  Every physician in America is now superbly trained over a period ranging from three to seven years depending upon their specialty.  As quality and standardization improved, medical boards began making test questions increasingly esoteric in order to preserve a normal statistical distribution of scores.  Most boards also began grading “on the curve”; failing a percentage of examinees with the lowest test scores regardless of whether the content they missed was critical to the practice of safe, high-quality medicine.  “Recall” and “airplane notes” developed in direct response to these “fail somebody” policies.  As one radiologist said to CNN: “”If they had a test where you could study relevant radiology knowledge and they tested on it, that would be fine.  Part of the problem is the test and the questions that they ask.  Because some of the questions are so obscure, that unless you know that they like to ask questions about that topic, you’re not going to study it because some of them are completely irrelevant to the modern practice of radiology.”

None of this would matter except that medical boards have managed to turn their monopoly on physician certification into a multi-million dollar gauntlet of tests and more tests that must be passed in order for physicians to stay on insurance plans and hospitals’ staffs.  Taking the initial dermatology board certification exam comes with a $2,500 fee, while regularly required “maintenance of certification” tests cost nearly $1,000 each.  And therein lies the true scandal in the saga of board certification and “recall” questions: it’s not that the tests are allowing incompetent physicians to care for patients, but that they are keeping many highly qualified doctors from entering practice and earning a living.

The math behind this injustice is relatively simple.  The nearby figure shows the score distributions of two hypothetical board certification exams like the one given by the ABD, each composed of 300 questions.  The left-hand curve shows a normal distribution of scores if none of the examinees have access to questions from previous tests that are re-used on this one.  The right-hand curve shows the change in scores that will result if 90% of the test takers already know the answers to 60 out of the 300 questions as a result of studying “airplane notes.  Let us further assume that the specialty board in question decides that it will flunk out the bottom 10^th percentile of test takers as a matter of “ensuring the quality” of its certification procedures.

Although the curves look similar, the use of recall notes produces a big difference in the specific score that will be used to fail test takers.  In turn, this translates into a major change in who will be allowed to pass the test.  Over half of doctors who did not use “airplane notes” and who would normally have been safely above the 10^th percentile threshold will now flunk as a direct result of their inability to answer all of the re-used questions correctly.  These well-trained and competent physicians will be wrongfully kept from medical practice while they prepare for next year’s $2,500 test.  If they’re smart they’ll find a set of “airplane notes” to study from rather than risk failure yet again.  It’s no wonder that training program directors have been known to encourage their young doctors to seek out every review course, old test question and study tool they can find.

In the modern world of American healthcare, none of this multi-million dollar hoop-jumping is necessary or even appropriate.  Every accredited specialty training program director already knows exactly what their residents know because they work with them each and every day.  By the time physicians completes their specialty training they’ll have been asked tens of thousands of questions by a host of attending physicians in almost every conceivable setting, from the surgery suite to the lecture hall.  Training programs have the ability to hold back or terminate questionable residents at any time.

Gilding the medical lily with endless layers of deliberately esoteric tests and certifications does nothing to improve patient safety or well-being; it simply prevents significant numbers of competent physicians from providing much-needed healthcare services.  It’s time for the medical establishment and the general public to decide that high levels of excellent training are far more important than all the testing in the world.

——————————-

In an effort to be perfectly constructive, we would like to suggest two alternatives to the current mess. The first is that one might have a board certification test that encompasses the truly important and mainstream information and examples of work-ups and treatment decisions that everyone in a given specialty ought to know. A pass rate of 100% should be the goal. What’s the point of flunking out ANY clinicians who know this basic and critical information? Make the old exams public for everyone to study. The entire point of the exercise needs to be ensuring basic awareness and good decision-making, not putting million of dollars into the coffers of self-selected specialty boards. A second alternative is asking residency training programs to do their own self-assessments of knowledge, decision-making and basic skills, and hold back or fail anyone who fails to meet their standards. This can again be based upon a national consensus of training program directors that documents those elements of training that are needed to produce good clinicians. Again, the goal should be a pass rate of 100%.

One has to wonder if there is not a class-action that should be taken against the medical specialty boards who recycle questions, that should be filed on behalf of all those who may have taken these exams and failed – quite possibly through no fault of their own – over the past 15 years.

One comment about the American Thinker article that struck us as being particularly interesting in light of the impending shortage of physicians is this one by someone using the pen name “Sud”:

After this article was posted on The American Thinker, we were contacted by the folks at the organization Change Board Recertification.  This organization has done some extraordinary work documenting the tens or hundreds of millions of dollars of “non-profit” profits that are flowing into specialty board coffers each year.  RTH readers may well find this organization, and the specialty board tax returns that it has made available on-line, worth a second and third look.