Max Rady College of Medicine

Concept: Cancer Diagnoses Validity / Agreement In Administrative Health Data

 Printer friendly

Concept Description

Last Updated: 2016-02-02

Introduction

    This concept contains a description of a method that investigates the diagnostic validity / level of agreement of cancer diagnoses codes that are recorded in different administrative health data sets, and the findings of this investigation. The information in this concept comes from the MCHP deliverable, Cancer Data Linkage in Manitoba: Expanding the Infrastructure for Research, by Lix et al. (2016).

    The investigation examined the validity (accuracy and timing) of cancer diagnoses in administrative health data sets using the Cancer Registry data as the reference data. Knowledge of this type of information is important for investigators who may not have access to the Cancer Registry data and want to develop cancer cohorts for their research.

Methodology

    As one aspect of data quality, Lix et al. (2016) examined the diagnostic validity / level of agreement between specific cancer diagnosis codes recorded in the Manitoba Cancer Registry and Treatment data and the corresponding information in the Hospital Abstracts data and the Medical Services (Physician Claims) data. The following five cancer sites were investigated: bladder, breast, colorectal, lung, and prostate.

    Different International Classification of Diseases (ICD) coding systems are used to record diagnoses information in each of these data sets, including: ICD-O-3 in the Cancer Registry data; ICD-9-CM and ICD-10-CA in the Hospital Abstracts data, and ICD-9-CM in the Medical Services (Physician Claims) data. The following table identifies each of the five cancer sites investigated and the corresponding ICD codes used to identify each type of cancer.

    Cancer Site
    ICD-0-3
    ICD-9-CM
    ICD-10-CA
    Bladder C670–C679 188 C67
    Breast C500–C509 174 C50
    Colorectal (colon, rectum and rectosigmoid) C180–C189, C199, C209, C260 153, 1540, 1541 C18, C19, C20
    Lung and bronchus C340–C349 162 C34
    Prostate C619 185 C61

    This analysis was undertaken for the period from April 1, 1997 to March 31, 2011. Individuals were identified as cancer cases in administrative health data if they had at least one hospital abstract or one physician claim with the relevant cancer diagnosis within one month, six months, and one year (before or after) the cancer diagnosis date in the Manitoba Cancer Registry.

Measuring the Diagnostic Validity / Level of Agreement

    The following statistics were used to estimate the diagnostic validity for each cancer site:


    The Wilson score interval method was used to calculate the 95% confidence interval for these statistics.

    The kappa (κ) statistic was used to measure the level of agreement between the data sources. The interpretation of kappa (κ) adopted in this analysis was:

    • κ < 0.20 is poor agreement,
    • 0.20 <= κ <= 0.39 is fair agreement,
    • 0.40 <= κ <= 0.59 is moderate agreement,
    • 0.60 <= κ <= 0.79 is good agreement, and
    • κ >= 0.80 is very good agreement (Altman, 1990).

Findings

    The results of this investigation are reported below in Table 1. "When we used a one–month observation window, that is, when we considered the period from one month before to one month after the cancer diagnosis to estimate diagnostic validity, sensitivity estimates ranged from 0.69 for prostate cancer to 0.89 for lung cancer. Specificity attained its upper bound of 1.00 for all cancers. PPV values were below 0.99 for bladder cancer only. Kappa estimates ranged from 0.77 (good agreement) for prostate cancer to 0.92 (very good agreement) for lung cancer.

    Diagnostic validity improved when a six–month observation window was used, that is, when we considered the period from six months before to six months after the cancer diagnosis to estimate diagnostic validity. For prostate cancer, sensitivity increased to 0.95. Sensitivity was lowest for bladder cancer (0.92). PPV was also lowest for bladder cancer (0.84), and was highest for breast cancer (0.98). Kappa estimates showed very good agreement for all cancers, and ranged from 0.87 for bladder cancer to 0.97 for breast cancer.

    There was almost no change in the estimates when a 12–month observation window was used, that is, when we considered the period from 365 days before to 365 days after the cancer diagnosis. Furthermore, in a sensitivity analysis we found that the validity estimates did not vary by cancer stage (data not shown)."
    Table 1: Validation Measures for Cancer Diagnoses in Administrative Data
    By time period, cancer site and agreement measure (95% Confidence Interval)

    Time Period: 1 month before and after cancer diagnosis

    Cancer Site
    Sensitivity
    Specificity
    Positive Predictive
    Value (PPV)
    Negative Predictive
    Value (NPV)
    Kappa
    Bladder 0.72 (0.69, 0.74) 1.00 (1.00, 1.00)
    0.89 (0.87, 0.91)
    0.99 (0.99, 0.99)
    0.79 (0.77, 0.81)
    Breast 0.87 (0.86, 0.87) 1.00 (1.00, 1.00)
    0.99 (0.99, 0.99)
    0.96 (0.96, 0.96)
    0.90 (0.90, 0.91)
    Colorectal 0.86 (0.86, 0.87) 1.00 (1.00, 1.00)
    0.99 (0.99, 0.99)
    0.96 (0.96, 0.96)
    0.90 (0.89, 0.91)
    Lung 0.89 (0.89, 0.90) 1.00 (0.99, 1.00)
    0.99 (0.98, 0.99)
    0.96 (0.96, 0.97)
    0.92 (0.91, 0.92)
    Prostate 0.69 (0.68, 0.70) 1.00 (1.00, 1.00)
    0.99 (0.98, 0.99)
    0.92 (0.92, 0.92)
    0.77 (0.77, 0.78)

    Time Period: 6 months before and after cancer diagnosis

    Cancer Site
    Sensitivity
    Specificity
    Positive Predictive
    Value (PPV)
    Negative Predictive
    Value (NPV)
    Kappa
    Bladder 0.92 (0.90, 0.93) 0.99 (0.99, 0.99)
    0.84 (0.82, 0.86)
    1.00 (1.00, 1.00)
    0.87 (0.86, 0.89)
    Breast 0.98 (0.98, 0.98) 0.99 (0.99, 0.99)
    0.98 (0.97, 0.98)
    0.99 (0.99, 0.99)
    0.97 (0.97, 0.97)
    Colorectal 0.96 (0.95, 0.96) 0.99 (0.99, 0.99)
    0.97 (0.97, 0.98)
    0.99 (0.99, 0.99)
    0.96 (0.95, 0.96)
    Lung 0.96 (0.96, 0.97) 0.99 (0.99, 0.99)
    0.97 (0.97, 0.97)
    0.99 (0.99, 0.99)
    0.96 (0.95, 0.96)
    Prostate 0.95 (0.95, 0.96) 0.99 (0.99, 0.99)
    0.97 (0.97, 0.98)
    0.99 (0.99, 0.99)
    0.95 (0.95, 0.95)

    Time Period: 1 year before and after cancer diagnosis

    Cancer Site
    Sensitivity
    Specificity
    Positive Predictive
    Value (PPV)
    Negative Predictive
    Value (NPV)
    Kappa
    Bladder 0.93 (0.92, 0.95) 0.99 (0.99, 0.99)
    0.81 (0.79, 0.83)
    1.00 (1.00, 1.00)
    0.86 (0.85, 0.87)
    Breast 0.98 (0.98, 0.99) 0.99 (0.99, 0.99)
    0.97 (0.96, 0.97)
    0.99 (0.99, 1.00)
    0.97 (0.96, 0.97)
    Colorectal 0.96 (0.96, 0.97) 0.99 (0.99, 0.99)
    0.96 (0.96, 0.97)
    0.99 (0.99, 0.99)
    0.95 (0.95, 0.95)
    Lung 0.97 (0.97, 0.97) 0.99 (0.98, 0.99)
    0.96 (0.96, 0.96)
    0.99 (0.99, 0.99)
    0.95 (0.95, 0.96)
    Prostate 0.97 (0.96, 0.97) 0.99 (0.99, 0.99)
    0.96 (0.96, 0.97)
    0.99 (0.99, 0.99)
    0.95 (0.95, 0.96)

    This study demonstrates that investigators who do not have access to the Manitoba Cancer Registry can accurately ascertain cancer cases from hospital abstract records and medical services / physician billing claims for the specific cancers investigated in this research.

NOTE:

Related concepts 

Related terms 

References 

  • Altman DG. Practical Statistics for Medical Research. 1st edition. London, UK: Chapman & Hall; 1990.(View)
  • Lix L, Smith M, Pitz M, Ahmed R, Quon H, Griffith J, Turner D, Hong S, Prior H, Banerjee A, Koseva I, Kulbaba C. Cancer Data Linkage in Manitoba: Expanding the Infrastructure for Research. Winnipeg, MB: Manitoba Centre for Health Policy, 2016. [Report] [Summary] (View)

Keywords 

  • Validation


Request information in an accessible format

If you require access to our resources in a different format, please contact us:

We strive to provide accommodations upon request in a reasonable timeframe.

Contact us

Manitoba Centre for Health Policy
Community Health Sciences, Max Rady College of Medicine,
Rady Faculty of Health Sciences,
Room 408-727 McDermot Ave.
University of Manitoba
Winnipeg, MB R3E 3P5 Canada

204-789-3819