Expected Value of Perfect information

Simply basing decisions on expected cost-effectiveness or, equivalently, net health or monetary benefit will ignore the question of whether the current evidence is a sufficient basis for adopting or reimbursing a health technology. It would fail to address the question of whether further research is needed to support such a decision in the future. The value of evidence or the health costs of uncertainty can be illustrated using a simple example as shown in Table 1. Each row represents a realization of uncertainty, that is, the net health benefit (commonly measured in quality-adjusted life years, or QALYs) that results when all the parameters that determine expected costs and effects each take one of their many possible values. These realizations may be generated by probabilistic sensitivity analysis, which commonly randomly samples (Monte Carlo simulation) from each of the distributions assigned to parameters. Therefore, each row can be thought of as representing one of the ways things could turn out given our current uncertainty. The expected net benefit for Treatments A and B is the average over all these possibilities (in this example, the range of potential values is simplified to only five possibilities).

On the basis of current evidence, we would conclude that Treatment B was cost-effective, and on average we expect to gain an additional 1 QALY per patient treated compared with Treatment A. However, this decision is uncertain, and Treatment B is not always the best choice (only 3 times out of 5), so the probability that B is cost effective is .6. For some realizations (2 out of 5), Treatment A would have been the better choice. Therefore, a decision to adopt B based on current evidence is associated with an error probability of .4. This is substantially greater than the traditional benchmarks of statistical significance, such as .05. But whether or not this level of uncertainty “matters” depends on the consequences, that is, what improvement in net benefit (or avoidance of harm) could have been achieved if this uncertainty had been resolved.

The decision maker is faced with three choices:

(1) adopt Technology B based on current evidence,

(2) adopt the technology now but conduct further

For example, if uncertainty could be completely resolved, that is, through complete evidence or perfect information about effect and cost, then we would know the true value of net health benefit before choosing between A and B. Therefore, with perfect information, we should be able to adopt whichever technology provided the maximum net benefit for each realization of uncertainty (the fifth column in Table 1). Of course, we can't know in advance which of these values will be realized, but on average (over the fifth column) we would achieve 13.6 rather than 13 QALYs—a gain of .6 QALYs. It should be clear that the cost of uncertainty or the value of evidence is just as “real” as access to a cost-effective treatment, as both are measured in terms of improved health outcomes for patients. In principle, evidence can be just as, or even more important than, access to a cost-effective technology. In this case, the expected value of perfect information is .6 QALYs, which is more than half the value of the technology itself, that is, 1 QALY gained by adopting B.

...