In  previous post, we looked at the evidence, or rather the lack of it, behind the use of “pay-for-performance” (P4P) programs involving healthcare information technologies.  Now let’s turn to a topic that is more traditionally associated with the P4P concept: using money in a direct attempt to influence the way in which physicians practice medicine on a daily basis.  The important news in this case comes from an enormous study that examined the records of nearly half a million patients with hypertension (i.e., high blood pressure) in Great Britain between 2000 and 2007, however the results were just released in January of 2011.

In this study, Britain’s National Health System offered to pay physicians up to 25% of their annual salary based on the proportion of their patients who achieved certain “quality of care” indicators.  There were 136 indicators involved to choose from, five of which involved management of high blood pressure.  These included the frequency of blood pressure measurement and the proportion of patients whose blood pressure was controlled, both of which were incentivized with the offer of payment.  The authors also looked at the rates of new treatment for hypertension and the use of intensive therapy, neither of which was part of the P4P payment program.  One thing that was particularly nice about the way this study was done is that the investigators actually bothered to look at the all of the measures in the period before the P4P program was implemented as well as several years afterwards.  This gave a detailed “before and after” picture of what actually went on over a prolonged period of time.  Once nice thing about a study this large that is carried out over such a long period of time is that don’t have to fret about sample size, whether the statisticians did their job correctly or many of the other hazards that plague less Herculean efforts.  So let’s head straight to the results as shown in the next three figures from the paper.

Effect of pay-for-performance and blood pressure control and monitoring in United Kingdom.

In this first figure, we see the affect of P4P on the percentage of patients with controlled blood pressure and monitored blood pressure before and after the incentive was put into place as denoted by the vertical grey bar.  As you can see, the percentage of patients with controlled blood pressure fell significantly during the whole five-year period, while the percentage of patients who had their blood pressure measured each month rose significantly.  However these trends were established well before the incentives were put into place.  The affect of P4P?  Exactly nothing.  Zilch.  Zero.  Nada.

Effect of P4P on intensity of treatment for hypertension.

This second figure shows the impact of P4P on whether physicians began drug therapy, and the degree to which drug combinations were used in an attempt to control blood pressure.  Again, the long-term trends were not affected ion the least by P4P financial incentives.

Effect of P4P on hypertension-related adverse outcomes (heart attack, stroke, kidney failure, heart failure).

This third figure shows us the impact of this particular P4P program on hypertension-related adverse outcomes such as heart attacks and strokes.  Once again there was absolutely no impact on the trends that existed before the program was put into place.  Notice that, if you have simply done a simple “before” and “after” study that did not collect data at close intervals, one might have concluded that the effects of P4P were statistically significant.

After noting that there was no evidence that P4P had any impact whatsoever, the authors summarized the results this way:

“These findings may have several explanations. Firstly, given the observed improvements in quality of care indicators for hypertension in the years before pay for performance, such as more frequent monitoring of blood pressure and increasingly more aggressive treatment, doctors may have already been implementing the appropriate changes in practice to achieve the pay for performance standards. Although the financial incentives in the policy were considerable, it is possible that the pay for performance targets for hypertension were set too low and therefore doctors did not need to change behaviour significantly to attain them. A smaller study of the United Kingdom’s pay for performance initiative, which evaluated the impact of this intervention on calculated clinical quality scores for selected conditions (but not controlling for secular trends), found that the policy led to short term, modest improvements in the quality of care for two conditions: asthma and diabetes. Once the targets were reached, however, improvements in quality slowed.”

This is certainly a cautious approach.  Rather than conclude that P4P doesn’t have any useful affect, maybe it’s just that we didn’t set the bar high enough?  After all, a previous study showed that, as soon as you imposed a P4P program things suddenly got a lot better.  Hmmm.  Let’s take a closer look at that other study.

In this case, we have the before-and-after points comparison design that was fortunately avoided in the recent hypertension study as a result of collecting and measuring data continuously.  The investigators looked at quality indicators for three different diseases that were included in the P4P program: coronary artery disease, diabetes and asthma.  The P4P incentives were implemented in 2004.

Well gosh.  From here it looks as if the incentives didn’t do a darned thing for the trend in the quality of management of coronary artery disease, but worked pretty well for diabetes and asthma.  At the same time the incentives did not appear to do much for a patient’s ability to actually get in to see a physician.  Access to a particular doctor actually declined after the incentives were introduced.  Perhaps they were too busy filling out paperwork?

But the more interesting story is told by looking at what happened to particular quality indicators that either were or were not included in the incentive program as of 2004.

The interesting thing here is that the scores for pretty much all quality indicators took a big jump between 2003 and 2005, whether or not they were associated with any sort of incentives.  It just so happened that all of the quality indicators that were incentivized started out higher than the non-incentivized indicators, but of course it’s the trend that really matters if we’re looking to rationalize the benefits of P4P.  So when you come right down to it, one would have to conclude that, as in the case of the first study we examined, P4P had no beneficial impact whatever.  Whatever happened was pretty much going to happen anyway.

We’re already covered a lot of important findings, but let’s look at one more recent study from 2009.  This one looked at the unintended consequences of P4P on physicians in England and in California.  P4P programs were implemented in both places, but with one key difference: the physicians in England were able to exclude patients (or report them as exceptions) if they refused treatment or were non-compliant.  The investigators then interviewed 20 primary care physicians in both locations to see if any adverse consequences occurred as a result of their participation in their respective programs.  They found that three major themes emerged from their analysis: “changes in the nature of the office visit, threats to the physician-patient relationship, and threats to professional autonomy.”  We should let the researcher’s finding speak for themselves on each count.  First, with regard to the changes in the nature of the office visit.

“Compared with California physicians, English physicians faced a much larger number of targets (80 vs 12 clinical targets in the statewide program at the time the interviews were conducted), and the program in England relied exclusively on data captured from electronic medical records. Pop-up boxes on the computer screen highlighted any areas of activity required to meet targets, prompting clinicians to take action or enter data during the office visit.



‘You look at the screen and the screen’s completely obscured by the list of yellow boxes, and it’s always trying to balance up the mood the patient’s in and getting the boxes ticked, especially with people that don’t come in that often. You know, they come in and tell you, you know, that “Oh, my son’s died last week,” and you go, “Yeah, yeah, whatever. Do you smoke?” or “Yeah, watch, watch your weight” and stuff…



One of the things that happens is the patient comes in, the boxes pop up, and you get straight into doing all that stuff… and they’re out of the room…. And I just think there is just more chance to, you know, miss [something significant], and that’s such an important bit, isn’t it…?’”

Gosh, that sounds like a great patient and provider experience, doesn’t it?   Especially if you know that the P4P incentive isn’t going to result in any significant change in either the quality of healthcare delivered or the clinical outcome.  California physicians largely did not share this same experience for two reasons.  The first is that most of them didn’t even know what the P4P quality measures were, or the targets they were supposed to achieve.  The second is that most of them did not have the “blessing” of electronic medical records to dictate their every action.

Next, the affect of P4P on the ongoing physician-patient relationship.  The greatest affect in this case was predictably on the physicians in California because they could not report or exclude non-compliant patients from their P4P reporting statistics:



“Although the absence of electronic records and computerized prompts meant that targets were seen as less disruptive of the flow of office visits among US physicians, adverse effects on physician-patient relationships were nevertheless identified, especially among physicians affiliated with organization A, the organization with the largest financial rewards. Physicians affiliated with this group expressed resentment about patients who refused to comply with their advice. In extreme cases patient noncompliance led to physicians telling patients they would be dis-enrolled unless they changed their behavior.



‘I tell them to leave. I told someone, you’re killing my pay for performance. You are the one that keeps being my outlier. Go join another medical group…’



The inability to exclude patients who refuse treatment or testing (unlike the UK system) appeared to increase pressure to cajole and persuade patients to secure their compliance. Other strategies reported by physicians included accusing patients of damaging their physician’s rating or lying to patients about the financial consequences of their refusing to comply.



Some physicians also reported bypassing informed consent procedures to meet screening targets for Chlamydia trachomatis. In addition to considerations of ethics, choosing not to request informed consent raises questions about the potential damage to doctor-patient relationships when patients who are tested without their knowledge are subsequently found to have a positive test for C trachomatis.



‘Well, everybody who didn’t have one, we sent out a form with a letter for Chlamydia screening. And we got 5 people who actually came back and did it, out of I don’t know how many hundred. So now, anybody who comes in and is in that age, I just tell them to get a urine. And I just send it in. This is life: I just send it in. If we’re going to be rated on it by somebody, that’s fine. We do it.’”

The moral here seems to be that when you implement a coercive process, you end up with a coercive process.  Once again, we as patients can take comfort in the idea that the people managing our healthcare system don’t seem to mind “destroying it in order to save it”.  Heck, who needs a trusting and mutually supportive physician-patient relationship anyway?

Finally, the investigators reported a “perceived impact on autonomy”:

“Whereas in England all primary care physicians were allowed to vote before the introduction of the new incentives arrangements, in California proposed indicators were published on the Internet and public comment was invited. [In other words, there was little, if any, meaningful physician input with respect to this change in the way healthcare services were to be delivered. –Ed.] Despite efforts in both contexts to consult primary care physicians and to encourage commitment to the process, and despite the larger number of targets in England, English physicians were generally more supportive of and accepted the targets that formed part of the pay-for-performance program. Differences in attitudes appeared to be related to perceptions about the implications of the respective target regimes for clinical autonomy.



Most of the English physicians suggested that the targets were helpful and did not appear to view them as undermining their ability to act as autonomous professionals.



Although some Californian physicians were supportive of pay for performance, most expressed much less satisfaction with it than their English counterparts. The incentive program was perceived as something externally imposed and managed, which made physicians feel that their autonomy was being challenged or that they were not trusted to perform in the absence of incentive payments.



At the same time US physicians believed they were being held accountable for things beyond their control. Data on their performance were generally collected by third parties, and the situation was compounded by the physicians’ inability to exclude patients from performance data if patients refused treatment or the targets were inappropriate for other reasons…



‘Physicians are monitored more than anybody. Are attorneys monitored? No. Are dentists monitored? No, not as far as I know. Are chiropractors monitored? No. So, it seems to be that physicians have either rolled over and given over their rights, and maybe they’ll be pushed to a certain point where they will rise up and say, “No more.” I don’t know…’



The system was viewed by many as unfair and opaque because it failed to take account of variations in practice populations, comprised indicators that were not amenable to control by physicians, withheld money that was due to physicians, and added to workload.



‘You might get a little bonus of money that some doctors would’ve considered part of their rightful payment to begin with. So the whole notion of withholds leaves a very bad taste. If you say, “What do you think of withholds?” that’s like saying, “What do you think of hemlock?” Not something I’d want to take. The problem with paying for performance…is it smacks of a withhold…. There must be adequate accounting methodologies to account for the recalcitrant patient or the patient who refuses for whatever reason treatments and therapies, and that’s not in the current model. It’s a source of disgruntlement…’”

At this point one really should be asking oneself, is making our nation’s physicians angry, frustrated and depressed a good thing?  Is it something that we really ought to be doing, especially if there is no evidence that we’re deriving any meaningful clinical benefit as a result of the exercise?  Does it makes Americans healthier or safer, or does it make the world a better place?  Most importantly, P4P programs have very tangible monetary costs because they add to the complexity of the healthcare system and incur substantial costs to implement and administer.  Given the evidence we’ve seen, why would any rational human being – let alone the (presumably) gifted thought leaders and administrators that we’re paying to manage our healthcare system – think that this is a good idea?

Perhaps the most important lesson that we can learn from the P4P research to date is what it says about the quality of healthcare leadership.  More on that in our next post – Part 3 in our “P4P adventure” series.