Concept: Sensitivity Testing - Issues and Guidelines

1. Defining the Event

How do we separate the numerator into 'yes' and 'no' groups?

When using physician claims data, an 'event' can be defined by:


• Requiring only one claim in total or over a specified period of time
  
• Requiring several claims in total or over a specified period of time.

A. One potential problem with relying solely on physician claims is that tests/procedures may be missed if the physicians do not submit a claim.

Example: Pap tests - Roos, Traverse and Turner (1999) noted that the number of tests performed might be underestimated because physicians may not always bill or because women may receive the test from a non-billing clinic. One algorithm used to estimate the true amount of testing was to include physician claims as well as lab component claims not within 14 days after a physician claim.

Result: the number of pap tests increased by <= 2%.

* This algorithm would still miss women who receive cervical cancer screening through both a non-billing clinic and a non-billing laboratory.

B. Distinguishing between screening and diagnostic tests can be difficult for some tests.

Example: Mammography - Before the Manitoba Breast Screening Program was established in 1996, mammography was entered twice in the claims database, once as a physician claim and once as laboratory bill. Double counting was prevented by the presence of the encrypted PHIN on each claim. Physician claims and laboratory bills show very high levels of agreement. With the program, mammography is entered once with a special code. The limited numbers of mammograms done in hospitals are billed by the physicians involved and are recorded in available administrative data.

However, the claims do not explicitly distinguish between screening and diagnostic mammograms. Roos et al. (1999) compared women with 1 mammogram in 1, 2, or 3 years to women with one mammogram in two years.

Result:

Table A: Population Coverage



Area of Residence	% Women with One or More Mammograms (1994-1996)
Fiscal 1995, 1996 Rural (23,343) Urban (71,560)	46.9 51.1

Table B: Estimation of Screening Mammography



Area of Residence	% Women with Only One Mammogram in Two Years (1994-1996)
Fiscal 1995, 1996 Rural (23,343) Urban (71,560)	35.7 38.6

Using this conservative algorithm Table B estimates that approximately 76% of all women receiving mammograms (Table A) had them for screening purposes. This proportion is slightly lower than other Canadian research that indicated that approximately 85% of women who had mammograms as having screening mammograms.

2. Defining the Numerator and Denominator

Who is included? Who is excluded?

A. Defining the Numerator : Adding or excluding certain subgroups in the study population may affect the outcomes. Relevant criteria may include age, residence, coverage, or certain clinical characteristics.

Example: The above tables do not include women with treaty status because the amount of underreporting for this test is unknown. Using claims data, overall rates for these women were considerably lower than those among non-aboriginals.

Result:

Area of Residence	%Treaty Status Women (50-69 yrs) With At Least One Mammogram (1994-1996)
Rural (n = 1,821) Urban (n = 489)	16.4 27.4

Thus, including women with treaty status in the overall analyses would have underestimated the proportion of all women in the general population having mammograms.

B. Defining the Denominator : Typically the only available information are demographics including age, residence, and gender. Thus, for the study by Roos et al. (1999), the denominator for the mammography analyses was based on the total number of women ages 50-69 in Manitoba rather than on the total Manitoba population (which includes men and other ineligible women).

Specific clinical groups cannot be included or excluded as this type of information is not available from administrative data.

C. Migration : Longitudinal studies and cross-sectional studies for any period longer than a day must address the issue of migration of individuals into the province after the study's earliest service date, or out of the province before the study's latest service date. Numerator and denominator must both be corrected.

Example: Screening mammography (Roos, Traverse and Turner, 1999). For the numerator, excluded screened women not in the province for two full years before their index date; minimal effects. The denominator was affected a bit more as all relevant women who migrated into or out of province were eliminated.

Result:

Without correction, percentage screened dropped slightly from 33.8 to 33.4 among non-native women for fiscal years 1994 and 1995. Differences across income quintiles were changed only minimally.