Dr. Rich has done a marvelous job of hosting Medical Blog Grand Rounds this week at The Covert Rationing Blog. Find the whole scoop and nothing but the scoop here.
Dr. Rich has done a marvelous job of hosting Medical Blog Grand Rounds this week at The Covert Rationing Blog. Find the whole scoop and nothing but the scoop here.
In previous post, we looked at the evidence, or rather the lack of it, behind the use of “pay-for-performance” (P4P) programs involving healthcare information technologies. Now let’s turn to a topic that is more traditionally associated with the P4P concept: using money in a direct attempt to influence the way in which physicians practice medicine on a daily basis. The important news in this case comes from an enormous study that examined the records of nearly half a million patients with hypertension (i.e., high blood pressure) in Great Britain between 2000 and 2007, however the results were just released in January of 2011.
In this study, Britain’s National Health System offered to pay physicians up to 25% of their annual salary based on the proportion of their patients who achieved certain “quality of care” indicators. There were 136 indicators involved to choose from, five of which involved management of high blood pressure. These included the frequency of blood pressure measurement and the proportion of patients whose blood pressure was controlled, both of which were incentivized with the offer of payment. The authors also looked at the rates of new treatment for hypertension and the use of intensive therapy, neither of which was part of the P4P payment program. One thing that was particularly nice about the way this study was done is that the investigators actually bothered to look at the all of the measures in the period before the P4P program was implemented as well as several years afterwards. This gave a detailed “before and after” picture of what actually went on over a prolonged period of time. Once nice thing about a study this large that is carried out over such a long period of time is that don’t have to fret about sample size, whether the statisticians did their job correctly or many of the other hazards that plague less Herculean efforts. So let’s head straight to the results as shown in the next three figures from the paper.
In this first figure, we see the affect of P4P on the percentage of patients with controlled blood pressure and monitored blood pressure before and after the incentive was put into place as denoted by the vertical grey bar. As you can see, the percentage of patients with controlled blood pressure fell significantly during the whole five-year period, while the percentage of patients who had their blood pressure measured each month rose significantly. However these trends were established well before the incentives were put into place. The affect of P4P? Exactly nothing. Zilch. Zero. Nada.
This second figure shows the impact of P4P on whether physicians began drug therapy, and the degree to which drug combinations were used in an attempt to control blood pressure. Again, the long-term trends were not affected ion the least by P4P financial incentives.
This third figure shows us the impact of this particular P4P program on hypertension-related adverse outcomes such as heart attacks and strokes. Once again there was absolutely no impact on the trends that existed before the program was put into place. Notice that, if you have simply done a simple “before” and “after” study that did not collect data at close intervals, one might have concluded that the effects of P4P were statistically significant.
After noting that there was no evidence that P4P had any impact whatsoever, the authors summarized the results this way:
“These findings may have several explanations. Firstly, given the observed improvements in quality of care indicators for hypertension in the years before pay for performance, such as more frequent monitoring of blood pressure and increasingly more aggressive treatment, doctors may have already been implementing the appropriate changes in practice to achieve the pay for performance standards. Although the financial incentives in the policy were considerable, it is possible that the pay for performance targets for hypertension were set too low and therefore doctors did not need to change behaviour significantly to attain them. A smaller study of the United Kingdom’s pay for performance initiative, which evaluated the impact of this intervention on calculated clinical quality scores for selected conditions (but not controlling for secular trends), found that the policy led to short term, modest improvements in the quality of care for two conditions: asthma and diabetes. Once the targets were reached, however, improvements in quality slowed.”
This is certainly a cautious approach. Rather than conclude that P4P doesn’t have any useful affect, maybe it’s just that we didn’t set the bar high enough? After all, a previous study showed that, as soon as you imposed a P4P program things suddenly got a lot better. Hmmm. Let’s take a closer look at that other study.
In this case, we have the before-and-after points comparison design that was fortunately avoided in the recent hypertension study as a result of collecting and measuring data continuously. The investigators looked at quality indicators for three different diseases that were included in the P4P program: coronary artery disease, diabetes and asthma. The P4P incentives were implemented in 2004.
Well gosh. From here it looks as if the incentives didn’t do a darned thing for the trend in the quality of management of coronary artery disease, but worked pretty well for diabetes and asthma. At the same time the incentives did not appear to do much for a patient’s ability to actually get in to see a physician. Access to a particular doctor actually declined after the incentives were introduced. Perhaps they were too busy filling out paperwork?
But the more interesting story is told by looking at what happened to particular quality indicators that either were or were not included in the incentive program as of 2004.
The interesting thing here is that the scores for pretty much all quality indicators took a big jump between 2003 and 2005, whether or not they were associated with any sort of incentives. It just so happened that all of the quality indicators that were incentivized started out higher than the non-incentivized indicators, but of course it’s the trend that really matters if we’re looking to rationalize the benefits of P4P. So when you come right down to it, one would have to conclude that, as in the case of the first study we examined, P4P had no beneficial impact whatever. Whatever happened was pretty much going to happen anyway.
We’re already covered a lot of important findings, but let’s look at one more recent study from 2009. This one looked at the unintended consequences of P4P on physicians in England and in California. P4P programs were implemented in both places, but with one key difference: the physicians in England were able to exclude patients (or report them as exceptions) if they refused treatment or were non-compliant. The investigators then interviewed 20 primary care physicians in both locations to see if any adverse consequences occurred as a result of their participation in their respective programs. They found that three major themes emerged from their analysis: “changes in the nature of the office visit, threats to the physician-patient relationship, and threats to professional autonomy.” We should let the researcher’s finding speak for themselves on each count. First, with regard to the changes in the nature of the office visit.
“Compared with California physicians, English physicians faced a much larger number of targets (80 vs 12 clinical targets in the statewide program at the time the interviews were conducted), and the program in England relied exclusively on data captured from electronic medical records. Pop-up boxes on the computer screen highlighted any areas of activity required to meet targets, prompting clinicians to take action or enter data during the office visit.
‘You look at the screen and the screen’s completely obscured by the list of yellow boxes, and it’s always trying to balance up the mood the patient’s in and getting the boxes ticked, especially with people that don’t come in that often. You know, they come in and tell you, you know, that “Oh, my son’s died last week,” and you go, “Yeah, yeah, whatever. Do you smoke?” or “Yeah, watch, watch your weight” and stuff…
One of the things that happens is the patient comes in, the boxes pop up, and you get straight into doing all that stuff… and they’re out of the room…. And I just think there is just more chance to, you know, miss [something significant], and that’s such an important bit, isn’t it…?’”
Gosh, that sounds like a great patient and provider experience, doesn’t it? Especially if you know that the P4P incentive isn’t going to result in any significant change in either the quality of healthcare delivered or the clinical outcome. California physicians largely did not share this same experience for two reasons. The first is that most of them didn’t even know what the P4P quality measures were, or the targets they were supposed to achieve. The second is that most of them did not have the “blessing” of electronic medical records to dictate their every action.
Next, the affect of P4P on the ongoing physician-patient relationship. The greatest affect in this case was predictably on the physicians in California because they could not report or exclude non-compliant patients from their P4P reporting statistics:
“Although the absence of electronic records and computerized prompts meant that targets were seen as less disruptive of the flow of office visits among US physicians, adverse effects on physician-patient relationships were nevertheless identified, especially among physicians affiliated with organization A, the organization with the largest financial rewards. Physicians affiliated with this group expressed resentment about patients who refused to comply with their advice. In extreme cases patient noncompliance led to physicians telling patients they would be dis-enrolled unless they changed their behavior.
‘I tell them to leave. I told someone, you’re killing my pay for performance. You are the one that keeps being my outlier. Go join another medical group…’
The inability to exclude patients who refuse treatment or testing (unlike the UK system) appeared to increase pressure to cajole and persuade patients to secure their compliance. Other strategies reported by physicians included accusing patients of damaging their physician’s rating or lying to patients about the financial consequences of their refusing to comply.
Some physicians also reported bypassing informed consent procedures to meet screening targets for Chlamydia trachomatis. In addition to considerations of ethics, choosing not to request informed consent raises questions about the potential damage to doctor-patient relationships when patients who are tested without their knowledge are subsequently found to have a positive test for C trachomatis.
‘Well, everybody who didn’t have one, we sent out a form with a letter for Chlamydia screening. And we got 5 people who actually came back and did it, out of I don’t know how many hundred. So now, anybody who comes in and is in that age, I just tell them to get a urine. And I just send it in. This is life: I just send it in. If we’re going to be rated on it by somebody, that’s fine. We do it.’”
The moral here seems to be that when you implement a coercive process, you end up with a coercive process. Once again, we as patients can take comfort in the idea that the people managing our healthcare system don’t seem to mind “destroying it in order to save it”. Heck, who needs a trusting and mutually supportive physician-patient relationship anyway?
Finally, the investigators reported a “perceived impact on autonomy”:
“Whereas in England all primary care physicians were allowed to vote before the introduction of the new incentives arrangements, in California proposed indicators were published on the Internet and public comment was invited. [In other words, there was little, if any, meaningful physician input with respect to this change in the way healthcare services were to be delivered. –Ed.] Despite efforts in both contexts to consult primary care physicians and to encourage commitment to the process, and despite the larger number of targets in England, English physicians were generally more supportive of and accepted the targets that formed part of the pay-for-performance program. Differences in attitudes appeared to be related to perceptions about the implications of the respective target regimes for clinical autonomy.
Most of the English physicians suggested that the targets were helpful and did not appear to view them as undermining their ability to act as autonomous professionals.
Although some Californian physicians were supportive of pay for performance, most expressed much less satisfaction with it than their English counterparts. The incentive program was perceived as something externally imposed and managed, which made physicians feel that their autonomy was being challenged or that they were not trusted to perform in the absence of incentive payments.
At the same time US physicians believed they were being held accountable for things beyond their control. Data on their performance were generally collected by third parties, and the situation was compounded by the physicians’ inability to exclude patients from performance data if patients refused treatment or the targets were inappropriate for other reasons…
‘Physicians are monitored more than anybody. Are attorneys monitored? No. Are dentists monitored? No, not as far as I know. Are chiropractors monitored? No. So, it seems to be that physicians have either rolled over and given over their rights, and maybe they’ll be pushed to a certain point where they will rise up and say, “No more.” I don’t know…’
The system was viewed by many as unfair and opaque because it failed to take account of variations in practice populations, comprised indicators that were not amenable to control by physicians, withheld money that was due to physicians, and added to workload.
‘You might get a little bonus of money that some doctors would’ve considered part of their rightful payment to begin with. So the whole notion of withholds leaves a very bad taste. If you say, “What do you think of withholds?” that’s like saying, “What do you think of hemlock?” Not something I’d want to take. The problem with paying for performance…is it smacks of a withhold…. There must be adequate accounting methodologies to account for the recalcitrant patient or the patient who refuses for whatever reason treatments and therapies, and that’s not in the current model. It’s a source of disgruntlement…’”
At this point one really should be asking oneself, is making our nation’s physicians angry, frustrated and depressed a good thing? Is it something that we really ought to be doing, especially if there is no evidence that we’re deriving any meaningful clinical benefit as a result of the exercise? Does it makes Americans healthier or safer, or does it make the world a better place? Most importantly, P4P programs have very tangible monetary costs because they add to the complexity of the healthcare system and incur substantial costs to implement and administer. Given the evidence we’ve seen, why would any rational human being – let alone the (presumably) gifted thought leaders and administrators that we’re paying to manage our healthcare system – think that this is a good idea?
Perhaps the most important lesson that we can learn from the P4P research to date is what it says about the quality of healthcare leadership. More on that in our next post – Part 3 in our “P4P adventure” series.
Dear RTH Reader,
Monday, February 7th, 2011 was the official release date of Overhauling America’s Healthcare Machine by Douglas Perednia, M.D. The publisher is FT Press, a division of Financial Times and Pearson PLC.
The book is available in either hard cover or an electronic version (Kindle or Adobe Digital Editions). It is (or will soon be) available at book sellers across the country. The hard cover price is $34.99.
As a reader of The Road To Hellth, you are invited to download a complete Kindle version of the book from Amazon.com. It will be available at no charge from February 7th through Saturday, February 12th only. Use the link http://amzn.to/OAHM-Free which will take you directly to the Amazon.com download page. Using this link the book should have a cost of $0.00. Just place your order and follow the instructions to download it for your PC, laptop, netbook, tablet, smartphone or electronic reader. A free Kindle app is available for nearly every type of computer, i-phone, i-Pad, etc.
Please feel free to share, Tweet or in any other way provide this link to your social network.
We would appreciate it greatly if you would return to Amazon.com and leave your comments and/or a personal review of the book. Your comments, thoughts, insights and reviews are also very welcome here at The Road to Hellth.
Thank you and we hope that you enjoy reading “Overhauling America’s Healthcare Machine – Stop the Bleeding and $ave Trillions”
- The RTH Webmaster
Most of our readers will have heard the story of the turtle and the scorpion:
Being a poor swimmer, a scorpion asks a turtle to carry him across the river on the turtle’s back.
“Are you out of your freakin’ mind?” the turtle says. “You’ll sting me the first chance you get. Everyone knows that that’s how scorpions are.”
“Oh come on”, replied the scorpion, “I wouldn’t do that. If I were to sting you we’d both drown. That would be foolish.”
Convinced by this logic, the turtle lets the scorpion climb on his back and proceeds to ferry him across the river. Just when they reached the middle, the scorpion stings the turtle and they both begin to sink.
“For crying out loud!”, wailed the turtle, “why the heck did you do that? Now we’re both going to drown.”
“”I’m sorry,” the scorpion said. “You see I couldn’t help it. It’s my nature.”
This story came to mind when reading a recent post by the inimitable Dr. Roy Poses, who typically writes about the role of money and corruption in healthcare on the Health Care Renewal blog. In his post, Dr. Poses calls our attention to the story of one Wendell Potter, a former head of public relations for Cigna – one of the largest private health insurers in the United States. In the midst of the negotiations that led to ObamaCare, Mr. Potter apparently had a change of heart and became a critic of the tactics used by private health insurers. In November of 2010 he published a book called Deadly Spin:An Insurance Company Insider Speaks Out on How Corporate PR Is Killing Health Care and Deceiving Americans.
I’ve not yet read Deadly Spin, but Dr. Poses’ post describes a number of typically offensive and despicable techniques that private health insurers typically use to manipulate public opinion and plead innocence while abusing both their customers and the public trust. These include making promises that they have no intention of keeping, using misinformation to discredit some of the factual assertions made in Michael Moore’s movie Sicko, and sending a “spy” to the funeral of a 17 year-old girl who had died because Cigna refused to pay for her liver transplant. However by far the most interesting aspect of the post is the completely integral role that the health insurance industry played in crafting the ObamaCare law and helping to sell it to the general public.
It seems that “large chunks” of the law were actually written by insurance company lobbyists behind the scenes, and they met with President Obama and members of administration many times while doing so. Insurance lobbyists were integral to including the mandate that individuals purchase private insurance policies or face a government-imposed fine. (That would be the mandate that a Federal judge in Florida recently decided was unconstitutional.) The notorious lobbying group America’s Health Insurance Plans (AHIP) also managed to kill the so-called “public option”, and launched a fake “grass roots” campaign to sow fear about various aspects of the bill that it did not like by using a front organization called Health Care America.
Of course, none of this will be news to readers of Dr. Rich’s Covert Rationing Blog, which has always contended that health insurers were largely behind much of the ObamaCare initiative all along. However it is sad that the cynical Dr. Rich had to be so right about it. It can only be to egg people like him on that one of the major justifications that President Obama and his Democratic colleagues now frequently uses to argue against repeal of the clearly mis-named “Patient Protection and Affordable Care Act” is that doing so would hand control of American healthcare back to the insurance companies. As they’ve done here:
“In one of the most animated speeches of two days of debate, Rep. George Miller, D-Calif., said repeal would return power to insurance companies. ‘Has anybody, any family in America, any single mother, any spouse, any child, any grandparent met a more bureaucratic system than the American health insurance system? There is no more bureaucratic system.’”
“WASHINGTON—Republican repeal of President Barack Obama’s health care overhaul law would take away personal freedoms and put power back in the hands of insurance companies, the administration’s top health official said Tuesday.”
The irony, in case anyone could miss it, is that insurers and their political allies have not only been in charge of much of the “reform” process all along, but still are. This is hardly surprising when we remind ourselves that the interests of the government and the interests of private insurers really aren’t that far apart. After all, the government itself is the biggest health insurer of them all. Anyone who doubts that the interests of private insurers and the interests of our various state and federal governments as insurers are in close alignment need only look to the words of Wendell Potter for some uncomfortable confirmation:
As Wendell Potter and Dr. Poses point out (intentionally or unintentionally), the rhetoric used by private insurance companies is virtually identical to that used by government regulators, the folks in Medicare and the Department of Health and Human Services and other officials. From Dr. Poses’ post:
“‘Potter shows then-Cigna chairman Edward Hanway’s leading role in “a multi-milllion-dollar public relations and advertising campaign” in the run-up to the 2008 presidential election, designed to ‘divert the public’s and the media’s attention’ away from the central fact of millions of uninsured and underinsured sick people, toward other problems that were harder to blame on insurers: aging Americans, extravagant doctors, expensive technology, and consumers demanding costly operations they don’t need.’
Note that the usual dogma one sees in the medical and health care research/ policy literature about the causes of US health care’s excess costs, declining access and poor quality are just as listed above: an aging population, high doctors’ fees and excessive utilization, technology that appears to be ever costlier (unlike technology everywhere but health care), and patients’ unreasonable demands.”
Alert readers will recognize some of this language from my last post on “pay-for-performance”, in which President Obama himself railed against doctors because they dared to remove tonsils in exchange for – God forbid – money. But on any given day of the week one is more than likely to find more than enough examples of what Mr. Potter is talking about coming from government and insurance company sources alike. And one addition worth adding is all of the time and effort that insurers and government representatives spend in re-directing the public’s attention toward “quality”, even though only 11% of Americans believe that improving quality and reducing errors is the most important problem we have in healthcare. Consider this exchange when HHS Secretary Kathleen Sebelius and former Senator Tom Daschle were specifically asked “how do we contain raising healthcare costs”:
“Kathleen Sebelius answers:
For too long, Americans have been at the mercy of insurance companies who, using faulty assumptions and loopholes, have gamed the system at the expense of middle class families—hiking up premiums to unfair and unsustainable levels and dropping health care coverage when people are most in need.
But with the passage of the Affordable Care Act, the balance of power is shifting back to the American people.
The new law contains the broadest package of health care cost-cutting measures in American history: investing in prevention; reimbursing providers based on the quality, not quantity, of care they provide; and setting new rules that hold insurers accountable…
The new law also encourages hospitals and providers to find new ways to provide higher quality care and avoid unnecessary hospital readmissions. Through Accountable Care Organizations that take responsibility for the cost and quality of the care received by patients, national pilot programs that bundle payments for the full course of a patient’s care and other innovative initiatives, the Affordable Care Act will help improve quality and reduce cost for patients…
Senator Thomas Daschle answers:
…Fourth, we need to stop unnecessary care, which may account for $700 billion of all care provided in the United States. Defensive, proprietary (is it a business or practice?) and market-driven medicine (those drug ads), along with fraud and abuse, account for much of the unnecessary care.
Finally, we must have far better management of chronic illness. There is too much inefficiency in the health care marketplace today. Accountable Care Organizations can contribute greatly to an improved system…”
Or take a look at this “U.S. Health Care Costs” background brief from the Kaiser Family Foundation. It reads as if it was taking dictation from Mr. Potter’s comments.
Clearly the political and appointed Leaders who are guiding America’s healthcare policies have far more in common with insurance companies executives than they do dividing them. They all think like insurers because they all are health insurers. The only difference is profit status (for-profit or non-profit), and the source of the premiums (tax dollars as opposed to premiums paid directly by businesses and individuals). This has some very important implications for you and your family. None of our policy makers are ever going to think like healthcare providers, not only because they’re not healthcare providers, but because they don’t have the same financial or social interests as your doctor. Basically your doctor is interested in two things: making a living and dealing with you and your health on an individual basis. Insurers and governments don’t deal with individuals, they deal with masses. It’s their nature.
And don’t even bother imagining that governments or insurers are ever going to think like patients. (This is in spite of all of those folksy stories they like to tell about Mrs. Smith who was denied insurance after she contracted leprosy while being treated for cancer caused by making apple pie.) For one thing patients consume resources – a great inconvenience when the need to balance the budget precludes building that new bridge to nowhere in the home district. And for another, they don’t have anything in common with you and your family when it comes to healthcare coverage. Their insurer can never drop them, no matter how sick they get. They can never be denied coverage, will never be uninsured for as long as they have their job, will always have access to the best facilities and probably pay little or nothing for their coverage. But most importantly, most of them have never been really sick. It’s extremely difficult for people who have always been well to relate to or understand the problems and frustrations faced by people who are sick and treated in the U.S. healthcare system. They’ve never done the paperwork. They’ve never dealt with having to keep track of prescriptions split between local and distant mail-order pharmacies. They’ve never been denied the referral or the pre-authorization. They don’t see all of the fancy and complicated healthcare information technology or ill-conceived “quality” initiatives as barriers to care. It’s their nature.
In a nutshell, this is why no insurance agency – public or private – can possibly be expected to work in the interest of patients and providers without being pushed, prodded and watched like a hawk at every turn. Of course they’ll make promises they have no intention of keeping. Of course they’ll try to stiff providers and deny coverage at every opportunity. Of course they’ll require mountains of paperwork and ask everyone to jump through hoops just in order to receive the most basic care – it’s just another sly way of rationing the benefits. Of course they’ll distract with farcical programs that claim to insist on “quality” while utilizing mandatory guidelines that fit only a fraction of the population. They can’t help it.
Yup. It’s their nature.
Like the walls of Jericho, the purported justifications for the ways in which government bureaucrats are micromanaging American healthcare just keep tumbling down. The latest victims are so-called “pay-for-performance” schemes and America’s virtual death march toward the mandatory deployment of fancy electronic medical record systems.
In case you hadn’t heard, pay-for-performance (“P4P” for short) is a strategy in which government regulators or insurers use money to motivate doctors to do whatever it is they wish them to do. While this is often couched in positive terms (i.e., “we will pay you extra to do what we want”) this particular management tool can be used as either a carrot or stick. For example, here in Portland, Oregon, Providence Health Plan currently rewards doctors with slightly higher insurance payments if they engaged in each of three activities: using electronic billing, using electronic medical records (EMRs, which are also called “electronic health records” or “EHRs”), and accepting Medicare and Medicaid patients into their practices. However the Federal government has turned this around by saying that Medicare providers who do not use EMRs by 2015 will have their Medicare payments cut by an additional 1% for each year of non-compliant behavior. Similar penalties will apply to e-prescribing beginning in 2012.
You might think that there is nothing wrong with using a few bucks to “encourage” desirable behavior and discourage wrong-doers, and in some cases you’d be right. After all, the government imposes fines for littering all of the time, and some insurance policies will give you discounts for not having accidents. But attempting to manipulate people like this is a funny thing. Sometimes it’s hard to discern what’s “right” and what’s “wrong”. For example if a pharmaceutical company offered to pay doctors to prescribe one of their drugs that clearly increased life expectancy for people who take it, it would be illegal. But if the government mandates that doctors buy and use healthcare information technology that has few or no documented benefits and might actually harm people, proponents of the technology (such as Medicare chief Dr. Don Berwick) might label that as “the future of health care itself” and “a win-win, right across the board”.
“Hey”, one might ask, “what’s up with that?”
Good question. America appears to have turned over its healthcare policy making to people who: (a) don’t know much about healthcare; (b) don’t know much about setting rational and effective policy; (c) govern hypocritically; or (d) practice some combination of these. This reality has been driven home by several recent scientific studies that everyone should know about. The first of these deal with electronic medical records. Others deal with the concept of pay-for-performance itself as a tool for managing doctors and other healthcare providers. We’ll discuss the latter in our next post.
Let’s start with the new research about EMRs. As you may know, evidence-based medicine is all the rage these days. President Obama is all for it. In a press briefing on July 22, 2009 he had this to say:
“Part of what we want to do is to make sure that those decisions are being made by doctors and medical experts based on evidence, based on what works…. Right now, doctors a lot of times are forced to make decisions based on the fee payment schedule that’s out there. … the doctor may look at the reimbursement system and say to himself, ‘You know what? I make a lot more money if I take this kid’s tonsils out … I’d rather have that doctor making those decisions based on whether you really need your kid’s tonsils out, or whether … something else would make a difference…. So part of what we want to do is to free doctors, patients, hospitals to make decisions based on what’s best for patient care.”
Meanwhile, President Obama’s chief of Medicare and Medicaid, Dr. Don Berwick, claimed that:
“Evidence-based medicine” is not just a catch phrase; it is a promise we want to make to our patients – to use all the care – and only the care – that can help them…
Through modern clinical epidemiology, technology assessment, and clinical research, we have developed powerful new tools to assemble, digest, and judge the evidence-base for clinical practice. Rational care plans can emerge, based firmly in scientific evidence, and drawing on research published in hundreds of journals that serve as the basis for the expert opinions and guidance of professional medical societies. Intermediaries have emerged – some call them “cybrarians” — to write and update the convenient, helpful, reliable “black books” of digested knowledge, much like those that we all kept at our side in training days..”
So what does the available evidence say about the safety and effectiveness of the EMRs being deployed under the “pay-for-performance” mandates of the Health Information Technology for Economic and Clinical Health Act (HITECH) Act?
Two landmark studies have been recently published on this topic. The first one by Romano and Stafford was published in the Archives of Internal Medicine. These two investigators used data from the 2005-2007 National Ambulatory Medical Care and National Hospital Ambulatory Medicine Surveys to examine the effect of EMR use on 20 different “quality of care” indicators. They compared the evidence for encounters that did not use EMRS, encounters that used EMRs, and encounters that used EMRs with so-called “clinical decision support” (CDS) capabilities that alert, remind or direct doctors to do certain things according to clinical guidelines of care. The data were compared for over one-quarter million patient encounters.
What they found was that the use of EMRs improved the performance of just a single quality measure – diet counseling in high risk adults.
“In the other 19 indicators, there was no statistically significant difference in quality based on the presence or absence of an EHR. These patterns were present both in bi-variate analysis and in multivariate logistic regression models. Non-statistically significant differences of greater than 5% favored EHR use in 3 instances and lack of EHR use in 2 instances. In the context of multiple comparisons across quality indicators, these findings suggest no association between EHR use and quality.
Clinical decision support also was not associated with higher-quality care. Among EHR visits, CDS was associated with better performance on only 1 of 20 ambulatory care quality indicators: avoidance of unnecessary electrocardiography during routine examinations… In the other 19 indicators, there were no significant differences in quality. Non-statistically significant differences of greater than 5% favored CDS in 1 instance and EHRs without CDS in 3 instances. Given the multiple comparisons, our findings indicate no association between CDS and quality.”
Dr. Scot Silverstein has written an excellent blog post that discusses a published response to the findings of this study.
Then within one week, a second study was published in the journal PLoS Medicine. This one was a systemic review of all reviews discussing the clinical impact of electronic medical records published between 1997 and 2010. The investigators looked at a total of 108 review publications and analyzed them to summarize what the scientific literature says that we know about the risks and benefits of EMRs. Lo and behold, they too found that the data supporting the use of EMRs, computerized physician order entry, e-prescribing and clinical decision support was weak or non-existent, and balanced by similar data demonstrating potential risks posed by use of these systems.
“Although a number of reviews purporting to assess the impact of EHRs were found, many of these in fact investigated auxiliary systems such as CDSS, CPOE, and ePrescribing. As a result, most of the impacts assessed were more relevant to these other systems. We found only anecdotal evidence of the fundamental expected benefits and risks relating to the organisational efficiency resulting from the storage and management facilities within the EHR and thus the potential for secondary uses… We did find, however, a small amount of secondary research relating to time efficiency for some health care professionals and administrators and data quality (in particular legibility, completeness, and comprehensiveness), which demonstrated weak evidence of benefit for both. Risks largely went ignored apart from anecdotal evidence of time-costs associated with recording of data due to both end-user skill and the inflexibility of structured data, increased costs of EHRs, and a decrease in patient-centeredness within the consultation.”
Given the enormous cost and potential risks and benefits to patients supposedly associated with these technologies, the authors rightly asked why more attention has not been paid to specifically proving the safety and effectiveness of these technologies.
“Our major finding from reviewing the literature is that empirical evidence for the beneficial impact of most eHealth technologies is often absent or, at best, only modest. While absence of evidence does not equate with evidence of ineffectiveness, reports of negative consequences indicate that evaluation of risks – anticipated or otherwise – is essential. Clinical informatics should be no less concerned with safety and efficacy than the pharmaceutical industry. Given this, there is a pressing need for further evaluations before substantial sums of money are committed to large-scale national deployments under the auspices of improving health care quality and/or safety.”
The nature of these risks should not be understated, especially in light of the vendor and government-driven initiatives that have characterized the deployment of these technologies for the past decade. With Presidents and their appointees pushing HIT, no one in a position of power or responsibility has wanted to look for, much less find defects in their magical thinking. Again, Dr. Silverstein has recently found that a whole truckload of troubling reports about HIT are now flowing into the FDA’s MAUDE (Manufacturer and User Facility Device Experience) database.
We’ll deal with using P4P as a tool for managing the clinical behavior of doctors and other healthcare providers in our next post, but what does what we’ve learned so far tell us about the legitimacy of pay-for-performance as a tool in the hands of America’s health policy leaders?
Take a second to re-read the comments that President Obama and Dr. Berwick made earlier with respect to the importance of valid scientific evidence when making decisions in healthcare. President Obama laments the fact that the existing RBRVS system may distort medical decision-making by giving providers a financial incentive to take actions that may not be evidence-based. He is correct in this, and (as we’ve written previously) the entire RBRVS system should be scrapped. Meanwhile Dr. Berwick reminds us that “Evidence-based medicine is not just a catch phrase; it is a promise we want to make to our patients – to use all the care – and only the care – that can help them.” Yet in the case of EMRs, e-prescribing and other healthcare information technologies we now have a situation in which the Federal government itself is creating and enforcing financial incentives to purchase and use technologies that have no good scientific evidence that supports their use. Still worse, research into the potential risks associated with their use has been ignored entirely. It’s really no different than having these same “evidence-based” government officials mandate payments for the use of an expensive, complex and potentially disruptive but inadequately studied set of surgical procedures, while completely ignoring any discussion of their potential risks. Tell us again, how this is any different from the RBRVS system that President Obama deplores? And where were Dr. Berwicks’ “cybrarians” while all of this is going on? Do our healthcare policy leaders believe in evidence or not when it comes to making public policy that will affect the lives of hundreds of millions of Americans?
If we’re serious about maximizing quality and minimizing cost in healthcare, here’s a P4P program that’s long overdue. Let’s pass a law that deletes a substantial fraction of salary compensation and/or time in office for any government or insurance company official who makes healthcare policy decisions that are not evidence-based.
This approach is guaranteed to result in far fewer regulatory decisions and P4P programs in healthcare. And with all of the salary givebacks, it might even balance the Federal budget.