Concept: Composite Health Indices

Definition of Composite Health Indices

A composite index is a mathematical combination of several indicators or measures in order to form a single number. This single index can be used to describe an entire set of indicators, and allows for an examination of differences between places (e.g., Regional Health Authorities (RHAs) and Winnipeg Community Areas (CAs)) and across time (three-year periods). In other words a composite index rolls several related measures (indicators) into a single score that provides a summary of how the health system is performing in certain areas.

Composite indices that summarize the provision of healthcare services and/or the extent of some health behaviors might give a "weather report" of the implemented policies and further direct the efforts to improve the health and healthcare of Manitobans. Essentially, the construction of composite indices involves subjective judgment on the selection of indicators, the choice of aggregation model, and on the weights of the indicators in the construction of the composite measure. When developing any composite measure or index of health or healthcare, it is critical to know the data needs and gaps, the implications of the gaps in the data, the methods used, and the assumptions underlying the methods.

Information for this concept comes directly from the MCHP deliverable titled Composite Measures/Indices of Health and Health System Performance by Metge et al. (2009).

Pros and Cons of Using Composite Indices:

The following list of pros and cons of using composite indices are identified in the Composite Measures/Indices of Health and Health System Performance deliverable and are adapted from a document titled Tools for composite indicators building. Nardo M., Saisana M., Saltelli A. and Tarantola S., 2005, European Commission, EUR 21682 EN, Institute for the Protection and Security of the Citizen, JRC Ispra, Italy. As of June 17, 2013, this document is available at: http://ipsc.jrc.ec.europa.eu/fileadmin/repository/eas/composite-indicators/Documents/EUR_21682_EN_Tools_for_Composite_Indicator_Building.pdf .

Pros of Using Composite Indices:

• Summarize complex or multi-dimensional issues which help to support decision makers;
  
• Are easier to interpret than trying to find a trend in many separate indicators;
  
• Facilitate the task of ranking regions on complex issues (e.g., benchmarking exercise);
  
• Assess regions over time on complex issues;
  
• Reduce the size of a set of indicators;
  
• Place issues of region performance and progress at the centre of policy considerations; and
  
• Facilitate communication with ordinary citizens and promote accountability

Cons of Using Composite Indices:

• May send misleading policy messages if they are poorly constructed or misinterpreted;
  
• May invite the drawing of simplistic policy conclusions if not used in combination with the indicators;
  
• May lend themselves to inappropriate use if the various stages (e.g., selection of indicators, choice of model) are not transparent and based on sound statistical or conceptual principles;
  
• Selection of indicators and weights could be the target of political challenge;
  
• May disguise serious failing in some dimensions of the phenonmenon and, thus, increase the difficulty in identifying the proper remedial action; and
  
• May lead to wrong policies if the dimensions of performance that are difficult to measure are ignored

How MCHP Built Composite Indices

We started by looking at existing research on developing and using composite health indices. We also consulted with experts in the area and drew upon our previous experience using individual indicators. Based on this background we attempted to construct a series of composite indices using administrative data housed at MCHP. This includes data from the province's Health Ministry and self-reported data on Manitobans' health behaviors gathered through the Canadian Community Health Survey (CCHS).

We were able to create composite indices in four areas: illness prevention and screening; healthy living; surgical wait times; and overall health status. We could not, however, develop indices for quality of primary care, quality of drug prescribing, or a summary of the extent of chronic disease in the population.

Composite indices were deemed to "work" when we found strong correlations between their various indicators. For example, if rates of mammography and Pap tests rose or fell together in a similar pattern across RHAs, this would suggest an underlying related factor between these two indicators. Where this was the case, a single score could be used to describe an RHA's performance across these indicators. For most of the composite indices, we looked at two three-year time periods: April 1, 2000 to March 31, 2003, and April 1, 2003 to March 31, 2006.

Limitations

We created a Prevention and Screening Index, to show how well health regions are doing in screening for cancer and preventing infectious diseases (see table 3.1 and figure 3.9 in Composite Measures deliverable). It pulls together rates of mammography and Pap tests in women, flu shots for older adults, and vaccinations among 2-year-old children.

At first glance, (see figure 3.9 - Prevention and Screening Composite Index Scores, by RHA) the resulting composite index appears to show that prevention and screening improved slightly over the two time periods. In particular, rates increased somewhat in northern RHAs (NOR-MAN and Burntwood) and in some of the least healthy areas of Winnipeg (Downtown and Point Douglas). If you were to look more closely at the indicators upon which the composite is based however, you would see that the overall increase was in fact the result of an increase in only one indicator - the percentage of older adults receiving a flu shot. Rates of mammography and Pap tests stayed the same, and childhood immunizations rates actually fell significantly.

We created two indices of healthy living to determine the prevalence of behaviors thought to lead to premature death from preventable diseases such as cardiovascular, respiratory, cancer and diabetes. The Health-Promoting Behaviors Composite Index includes self-reported data on healthy eating, changes to improve health, and physical activity; the Health-Risk Behaviors Composite Index is a summary of self-reported rates of smoking and binge drinking (see figure 4.6 - Healthy Living Composite Indices Scores, by RHA).

While you would assume that areas with high rates of healthy eating and fitness would have low rates of tobacco and alcohol use, this pattern was in fact true only in Winnipeg's healthiest CAs. Risky behaviors had a far greater impact than healthful actions on overall health status. Because this index consists of only two indicators - smoking and binge drinking - it would be easier to simply measure those rates than to build an entire composite index using these and other indicators.

Another potential limitation to these indices is that they are based on survey data from Statistics Canada where the same questions may not be included in future surveys.

Composite indices, by their very nature, can provide us with only a high-level indication of our health system's performance or the health of our population. In the same way that the "service engine" light on your vehicle's dashboard requires a detailed diagnosis by a mechanic to determine specifically where the problem lies, composite indices require planners to go "under the hood" to figure out what is or is not working. Our findings suggest that the effort required to develop, validate, and update composite indices is disproportionate to the value of the information they yield.

Validation

We created a Surgical Wait Times Composite Index to get a sense of how long residents in one RHA or Winnipeg CA wait for surgery compared to residents in other regions. This index pooled wait times for six common elective procedures:


• surgical removal of the gallbladder
  
• hernia repair
  
• removal of breast lesions
  
• stripping/ligation of varicose veins
  
• carpal tunnel release
  
• tonsillectomy

The Index scores increased over time for all RHAs, indicating that Manitoba residents were waiting longer for these procedures at the end of the study period (see figure 5.10 - Surgical Wait Times Composite Index Scores, by RHA). As well, wait times appear to have been shortest in areas with the highest rates of premature mortality (a surrogate for overall healthiness of a region's population), which suggests those people most in need of surgery waited the least amount of time to be operated on.

Although we succeeded in building a handful of valid, reliable composite indices, we came away from the project questioning the value or utility of such tools for health planners and decision makers. While the notion of rolling up large amounts of detailed information about health status and system performance into single markers or measures is appealing, such index scores lack real world application and sufficient "actionability" to improve the health of Manitobans.