Introduction

This concept summarizes the process for validating the definition of chronic kidney disease (CKD) in children (aged 0-17) by comparing case definitions from administrative data to a "gold standard" developed from laboratory data. This validation process was carried out in the Care of Manitobans Living with Chronic Kidney Disease deliverable by Chartier et al. (2015).

The validation process involved comparing administrative case definitions for CKD in children to the "gold standard" developed from laboratory data, and then measuring the validity / level of agreement using several different statistical methods. The concept also provides a summary of the validation results and provides direct links to additional information reported in the deliverable.

Methodology

The methodology used to validate a CKD diagnosis in children in Chartier et al. (2015) is described below. This includes the data sources used and how the CKD case definitions were developed from the data. It also identifies the measures of validity and agreement that were used, and provides links to additional relevant information available in the deliverable.

Data Sources and Case Definitions

1. Laboratory Data - The "Gold Standard"

In this research, the Diagnostic Services Manitoba - now named Shared Health Diagnostic Services (SHDS) - Chemistry data from 2006/07 to 2011/12 was used to develop the CKD validation cohort (the "gold standard"), using the technical case definition for identifying CKD from laboratory data. This definition identifies the lab tests (serum creatinine to estimate glomerular filtration rate and urine tests for protein to assess for proteinuria) and abnormal levels that indicate a CKD diagnosis. Two abnormal laboratory tests were required 90+ days apart to meet criteria for CKD. For more detailed information, please see the technical definition of CKD using laboratory data in the Chronic Kidney Disease (CKD) concept.

The following table summarizes how the CKD validation cohort was selected from the laboratory data and the numbers related to this selection. The selection process resulted in 918 children in the validation cohort.



Inclusion / Exclusion Criteria	Number to Exclude	Remaining In Cohort
Children in Manitoba (aged 0-17)	.	291,781
- No laboratory data	273,682	18,099
- No abnormal laboratory tests	12,957	5,142
- Only 1 abnormal laboratory test	3,928	1214
- Abnormal tests occurred within 90 days	279	935
- Children receiving dialysis or kidney transplant	17	918
Final Cohort **	.	918


NOTES: The exclusion criteria are based on the available laboratory data.

** The final cohort includes children meeting the Chronic Kidney Disease case definition based on laboratory data.

For more descriptive information on the validation cohort of children with CKD, please see Appendix 5: Validation of Administrative Data Definition of Chronic Kidney Disease in Children in the deliverable.

2. Administrative Data

The administrative data sources used to develop case definitions of CKD included: hospital abstracts data, medical services data and prescription information from the Drug Program Information Network (DPIN) data.

The algorithms developed consisted of different combinations of the number of relevant hospital abstracts, medical services (physician claims) and prescriptions (medication records) over a one-, two- and three-year time period. In total, 18 different algorithms were investigated. Table 3.3: Number of Children with Chronic Kidney Disease Identified with Administrative Data Case Definitions and Laboratory Data provides a list of the algorithms investigated and the resulting number of children with CKD found in the administrative data, the laboratory data, and in both the administrative and laboratory data for each algorithm.

Measures of Validity and Level of Agreement

Statistical measures of validity and level of agreement were then calculated for each algorithm. The measures included:


• sensitivity,
  
• specificity,
  
• positive predictive value (PPV),
  
• negative predictive value (NPV),
  
• the Kappa (κ) statistic, and
  
• area under the receiver operating characteristic (ROC) curve.

Table 3.4: Validity of Administrative Data Case Definitions of Chronic Kidney Disease Compared to Laboratory Data provides information on the results of the statistical measures of validation and level of agreement used in this research.

Results

As reported in Table 3.4, all algorithms had very low sensitivity (0.221–0.416) but very good specificity (0.994–0.999). The positive predictive value was similarly low (0.180–0.342), whereas the negative predictive value was high (0.998). The receiver operating characteristic (ROC) curves indicate poor to fair accuracy (0.610 – 0.705). Generally speaking, the sensitivity improved slightly with increasing years of data (three years vs. one or two years). Algorithm #15 had the highest sensitivity (0.416), and included one or more hospital records, one or more physician claims and one or more prescriptions over three years.

The algorithm selected as the administrative data case definition for this research was #16: one or more hospital records, two or more physician claims and one or more DPIN prescription over three years. To see the detailed case definition for identifying CKD in children that is used in this research, please see the technical definition of CKD using administrative data in the Chronic Kidney Disease (CKD) concept. This case definition successfully identified 37.6% (sensitivity) of children who had CKD, and 99.6% (specificity) of those who did not have CKD, as confirmed by laboratory testing.

The low sensitivity is generally unacceptable for population-based surveillance for CKD and suggests that using administrative data only to estimate the prevalence of CKD will underreport the true rates. This reinforces the importance of using laboratory data when investigating the prevalence of CKD.