As economies develop, public health systems evolve, and technologies emerge, public health interventions become increasingly complex and options become limited by cost-effectiveness. As such, the use of economic evaluation has become increasingly important for decision-making in developing countries (1). In China, the desires to move towards a data-driven healthcare system and to promote use of health technology assessments (HTA) have highlighted the importance of economic evaluation (2).
In public health, evaluation may broadly include both economic and program evaluation. Economic evaluation is defined as “the comparative analysis of alternative courses of action in terms of their costs and consequences” (3). Program evaluation is “a systematic way to improve and account for program actions involving methods that are useful, feasible, ethical, and accurate” (4). The two methodologies may be used for the same interventions but answer distinctly different questions. They are often intertwined as economic evaluations may rely heavily on the findings from program evaluations.
HTA was defined by the US Office of Technology Assessment in 1994 as “the structured analysis of a healthcare technology, a set of related technologies, or a technology related issue that is performed for the purpose of providing input to a policy decision” (5). Economic evaluation is a component of HTA, while other components focus on effect size, equity, and ethical concerns of the technology. Economic evaluations of medical technologies often require conducting new HTAs to generate estimated effect sizes and other parameters if sufficient information from existing studies is not available.
Prevention effectiveness studies are applications of economic evaluation in assessing the health and economic impact of public health policies, programs, and practices by determining their effectiveness, safety, and cost (6). Because prevention effectiveness studies are often attempting to project the impact of policy or programs in hypothetical scenarios, these studies face additional challenges. These studies usually involve estimation or simulation of counterfactual impacts, which may not be observed in real life, subjecting them to data and methodological criticism.
Methods of economic evaluation include cost-effectiveness analysis (CEA), cost utility analysis (CUA), cost benefit analysis (CBA), and cost analysis. Each of the first three methods requires a careful cost analysis and an assessment of both beneficial and adverse health effects (7). All three methods measure costs in the same way but methods to value health and other benefits differ. CEA examines the cost per unit of health outcome for all options. CUA is a specific form of CEA in which the health outcomes are converted to measures of health-related quality of life such as the quality-adjusted life year (QALY). CBA differs from CEA and CUA in that it includes all the impacts of programs or policies—health and non-health impacts—and monetizes their value to assess whether the benefits exceed the costs. CBA is particularly useful for analyses that include important and substantial non-health benefits. Cost analysis is the process of estimating the cost of a program or policy or the financial cost of the health outcomes (8). Results from cost analyses are useful for decision making on their own or can be included in CEA, CUA, and CBA.
Key points to consider during an economic evaluation include: audience, problem or question to be analyzed, treatment or intervention, perspective, time frame and analytic horizon, analytic method, marginal or incremental analysis, costs, health outcomes, discount rate, sensitivity analyses, summary measure, and distributional concerns (7). Perspective is of particular importance for economic evaluation in public health, which usually takes a societal or government perspective. A recent guideline recommended economic evaluation studies to take two perspectives including societal perspective (9). Further materials outlining the components and details of economic evaluation are available elsewhere (7).
Economic evaluation has played an important part in public health decision-making for almost 80 years. US CDC has used economic evaluation in making key program and policy decisions, most notably beginning in early immunization policy. In 1995, US CDC created the CDC Steven M. Teutsch Prevention Effectiveness Fellowship, a two-year postdoctoral training program in health economics to build agency expertise in economic evaluation. The fellowship has trained more than 150 individuals since its inception and has developed a cadre of economists at US CDC with expertise in economic evaluation and policy analysis (10). Applications of economic evaluation at US CDC include topics ranging from folic acid fortification to food-borne disease surveillance (11–12). However, economic evaluation research within US CDC does not represent the full spectrum of the field as there is a plethora of critical theoretical and applied research on economic evaluation outside of the agency. Systematic reviews elsewhere provide more comprehensive lists of economic evaluation studies within and outside of US CDC (1–2).
China is also utilizing economic evaluation for public health decision-making. Earlier economic evaluations of public health programs in China include those on the use of Japanese encephalitis vaccine (13), annual influenza vaccination among children aged 6 months to 14 years (14), and a nationwide hepatitis B catch-up vaccination among children and adolescents (15). Many economic evaluations in China have employed the modeling of complex diseases and interventions such as those for HIV/AIDS, universal newborn hepatitis B vaccination, and diabetes prevention (16). Some of the economic evaluations have resulted from collaboration between public health communities in the US and China (13-14).
Use of economic evaluation as part of HTA in China has steadily increased over the last decade. China’s commitment to evidence-based decision-making in health and medicine has spurred interest and investment in HTA (2). To illustrate this point, a study of the Tufts Medical Center Cost-Effectiveness Analysis Registry (CEA Registry) found an increasing trend in the number of CEA studies with a geographic focus on Mainland China, concentrating on oncology, infectious and parasitic diseases, and endocrine, metabolic, and nutritional diseases (2). More broadly, a bibliometric analysis of full economic evaluations between January 1, 2012 and May 3, 2014 found China-based studies accounted 30% of all economic evaluation in upper-middle-income countries with similar major diseases areas of focus (1). CEA is more popular than CUA in upper-middle-income countries than it is in high-income countries (1), possibly due to the difficulty of developing population-specific QALY weights (17).
There are common misconceptions about the value of economic evaluation for public health decision-making. Some public health practitioners are not convinced of the need for economic evaluations because their value may be limited for straightforward situations. However, they can be critical for complex decision-making. For example, funding allocations between different disease areas or between public health and other functions often require systematic and comprehensive evaluation. Even for a specific health condition, there may be a multitude of options to be assessed. As high impact policy and program opportunities have become exhausted (eg., water fluoridation and urban sanitation), public health practitioners are now dealing with increasingly complex interventions involving emerging technologies and intricate social dynamics. Systematic, evidence-based evaluations of the impacts, cost, and benefits are critical for informed decision-making.
Another misconception is that economic evaluation serves as post hoc justification and has limited impact. However, economic evaluations have provided unexpected critical insights and guided public health policies. For instance in 2009, US CDC used economic evaluation to conduct a regulatory impact analysis to assess the health and other impacts of removing HIV infection from the definition of communicable disease of public health significance, which would allow foreign persons with HIV infection to be admitted into the US (18). US CDC researchers estimated that even at an upper bound estimated annual healthcare system cost of US $173 million, the benefits of the policy including family reunion, bringing in high-skill immigrants, and reducing stigma associated with HIV patients outweighed the cost (18). In 2010, regulations based on the US CDC analysis removed HIV status as automatic grounds for inadmissibility.
Another example, possibly illustrating the complexities of economic evaluations, is a recent CBA conducted by the Food and Drug Administration (FDA) of a proposed rule that would require graphic warning labels on cigarette packs (19). Disputes over the methodology used in the analysis has prevented the FDA from issuing a regulation that would have promoted tobacco cessation. After the economic community weighed in on the methodological controversy (19), the FDA issued a updated proposed rule and will change the recommended methodology for examining regulatory impact of the rule (20).
While economic evaluation becomes more widespread and methods continue to develop, both users and practitioners are working to standardize the field. Three recent notable efforts include the updated CEA guidelines published by the Second Panel on Cost-Effectiveness in Health and Medicine (9), the Gates Reference Case for CEAs of Health Projects funded by the Bill & Melinda Gates Foundation (21), and methods for valuing mortality risks in low- and middle-income countries published by researchers from the Harvard T.H. Chan School of Public Health (22).
As the importance of economic evaluation as a decision-making tool grows, China’s public health community should prioritize building its economic evaluation capacity. This will allow public health practitioners to identify the policies, interventions, and programs that offer the best value for the Chinese people and to contribute to its methodological development.
Economic Evaluation: Terminologies
Common Methods of Economic Evaluation
Use of Economic Evaluation in Public Health: US and China
Misconceptions about Economic Evaluation
Recent Developments and Conclusions