Introduction

High school completion / graduation is frequently used as an outcome measure for a child's level of educational attainment. Attainment of Grade 12 by a certain age, such as 17 years old, provides a similar measure. This concept describes the general method used by MCHP to measure high school completion / graduation. This method identifies the data sources required, the considerations for developing a cohort of students to follow, and other considerations for measuring high school completion / graduation and progress, including: the time required to complete / graduate from high school, the criteria used to identify high school completion / graduation, and other indicators of high school progress. The concept also identifies and describes the general methods used in MCHP research to measure High School completion / graduation and other MCHP research that measures education attainment levels. Information related to existing SAS code for calculating high school completion / graduation and general cautions / limitations with the data are included at the end of this concept.

This concept was developed because over time, not all schools or school districts in Manitoba consistently record a graduation status indicator in their data to identify students who have completed Grade 12 and we believe our methods improve the completeness of measuring high school completion / graduation in the administrative data in our Repository. ( Brownell et al., 2006 ).

General Method of Measuring High School Completion / Graduation

Different methods have been used in measuring high school completion / graduation in research using the MCHP Data Repository. This section describes the general methods used at MCHP to measure high school completion / graduation in Manitoba.

Data Sources

The main data source used to investigate high school completion / graduation is the Enrollment, Marks and Assessment Data from the department of Education. This data contains student records that identify the classes students are enrolled in and the marks achieved by each student as they progress through high school. A secondary data source, the Manitoba Health Insurance Registry, is used to identify students who remain residents of Manitoba over time to ensure that all relevant enrollment and marks data are available for determining their level of achievement.

Developing a Cohort

Different cohorts have been used in MCHP research related to high school completion / graduation. In Oreopoulos et al. (2008), birth cohorts covering several years were developed and then followed for 17 years to identify those students that had reached grade 12 by age 17. In most MCHP research, a cohort is developed that includes all students who are enrolled in grade 9 in a given year and remain on the Manitoba Health Insurance Registry for the duration of their high school education. These students are then followed forward in time in the enrollment and marks data to track their progress and level of achievement.

Different exclusions have been applied when developing the cohorts, depending on the research project. For example, in Brownell et al. (2004), students who appeared to be in Grade 9 but were younger than 11 years or older than 16.5 years as of December of the year they were first enrolled in Grade 9 were excluded. This was done to remove any errors (the very young students) and to remove any "adult learners" who might be in the data and enrolled as being in grade 9, because we know that we do not capture all credits for adult learners nor is their graduation requirement the same as a regular high school student. In some MCHP research, no age restrictions are applied.

Another consideration is students moving away from the province during or after Grade 9, or students who move into the province after Grade 9. In these situations, enrollment and marks data for these students will be incomplete and therefore these students are excluded from analyses. In addition, because enrollment and marks data are often incomplete for schools operated by First Nations Communities, students enrolled in First Nations schools are also excluded. Additional considerations and exclusions specific to individual research projects are identified and discussed later in this concept.

Time to Complete / Graduate from High School

The amount of time required to complete / graduate from high school is an important consideration in calculating high school completion / graduation rates. The approach taken at MCHP is to investigate graduation rates at different times - either after a 4, 5 or 6-year period after entering grade 9. The rationale for this is because while most students will complete high school within the expected four year period, a number of students will take longer to achieve this. In order to get a clearer picture of the number of students that actually complete / graduate from high school, and how long this is taking, MCHP often uses a six year time frame.

SAS® Macros

To assist in this type of analysis, MCHP has developed a set of four High School Completion SAS® macros that calculate graduation rates over different time periods. These macros are available (internal access only) in the MCHP SAS Macro Library. The four macros are:


• _grad_0yr_followup - 0 year follow up (on time / expected graduation)
  
• _grad_1yr_followup - 1 year follow up post expected graduation year
  
• _grad_2yr_followup - 2 year follow up post expected graduation year
  
• _grad_3yr_followup - 3 year follow up post expected graduation year

High School Completion / Graduation Criteria

Three criteria have been used to identify high school completion / graduation. These criteria are applied in sequential order that avoids duplicate counting, and combined, give a measure of the total number of students who completed / graduated from high school. The criteria include:


1. the "Year End Status" variable (or Graduation Flag ) on the student's high school marks data. If this variable indicates that the student graduated then the student would be identified as a 4, 5, or 6 year graduate, depending on the year the indicators appeared in the student record.
   
   NOTE: As mentioned in the Introduction section of this concept, the "Year End Status" variable is not consistently recorded in the earliest years of Education data in our Repository. However, as noted in a recent project, the "Year End Status" variable is more consistently completed starting with 2009 academic year (2009-2010 school year) data. Due to the earlier inconsistency, two additional criteria were developed to help identify high school completion / graduation.
   
   
2. achieving a required total number of high school credits.
   
   NOTE: The minimum number of credits is between 28 and 30, depending on the year of high school completion / graduation. In April 2007, the provincial Department of Education mandated two new Physical Education/Health Education (PE/HE) courses at grades 11 and 12 as recommended in the Healthy Kids, Healthy Futures Task Force Report. This change brings Manitoba Senior Years graduation requirements to 30 credits for those students graduating in 2009/2010 and beyond. Changes to the graduation credit requirements for the Senior Years have been phased in over the course of three school years:
   
   
   • academic year <= 2007 (the 2007/2008 school year) - Minimum Credit Requirements: 28
     
   • academic year = 2008 (the 2008/2009 school year) - Minimum credit Requirements: 29
     
   • academic year >= 2009 (the 2009/2010 school year) - Minimum Credit Requirements: 30
     
     (Manitoba Education Web Site - Graduation Requirements: http://www.edu.gov.mb.ca/k12/policy/grad_require.html - accessed October 5, 2010).
   
   
3. earning four or more grade 12 course credits, regardless of how many credits in total the student had earned during high school.
   
   NOTE: In Brownell et al. (2015), this was increased to six or more grade 12 credits. In subsequent work in April 2023, a sensitivity analysis of the 4+ or 6+ grade 12 credits and the graduation flag was performed to see how it affected high school graduation rates. Using the 6+ credits instead of the 4+ credits resulted in a ~2% drop in the overall graduation rate. The conclusion was that although criteria three does not make a huge difference to overall graduation rates, probably because the "Year End Status" (graduation flag) variable is now better recorded in the Education data, it is recommended to keep using the 4+ grade 12 course credits rather than the 6+ grade 12 course credits in the high school completion definition.

Other Indicators of High School Progress

For those students that do not complete / graduate high school within the expected amount of time (e.g.: within four or five years), other indicators were developed to measure a student's progress. ( Brownell et al., 2004; Brownell et al., 2008 ). These indicators include:


• Near graduate: A high school student who:
  
  • enrolled in Grade 12 at least once in the 4th or 5th year after entering Grade 9,
    
  • did not have a graduation flag,
    
  • had between 26 and 27.5 credits, and
    
  • did not enroll in a 5th and/or 6th year of high school
  
  
• Continuing Student: A high school student who:
  
  • was still enrolled in school, but had not made it to Grade 12 by the 5th year after Grade 9, and
    
  • did not have a graduation flag or 28+ credits. The credits earned were quite low for these students (e.g. 14 out of the required 28 credits)
  
  
• Withdrawn From School: A high school student who:
  
  • was not enrolled in school for any 2 consecutive years at any time after entering Grade 9. It is possible that some of these students transferred to a First Nations school, but we do not have their enrollment data to identify these cases.
    
    NOTE: Due to data limitations, our best guess is that between 2-3 % of all school aged children in Manitoba are in a First Nations school.
  
  
• Progressing: A high school student who is still enrolled and attending high school.

MCHP Research Measuring High School Completion / Graduation

Most MCHP research uses the general method described above in measuring high school completion / graduation. This section describes the specific methods used in MCHP research projects to define and measure high school completion / graduation.

1. Brownell et al. (2004)

In the How Do Educational Outcomes Vary With Socioeconomic Status? Key Findings from the Manitoba Child Health Atlas report by Brownell et al. (2004), they investigated high school performance and high school completion for students in four different levels of socioeconomic status (SES) - low, low-mid, middle and high. The study reported on a 1997 cohort of grade 9 students and measured the high school completion / graduation rate at the end of five years. This was measured by the presence of a graduation flag in the student record or at least 28 credits earned during high school after following these grade 9 students after five years. Figure 4 - High School Completion Rates in this report illustrate the outcomes (graduation, near graduate, continuing student and withdrawn from school) by SES category.

2. Brownell et al. (2008), Martens et al. (2010), and Brownell et al. (2015)

In this research, high school completion / graduation was measured using three different criteria from the enrollment and marks data set:

1. the presence of a graduation flag in the "Year End Status" field; OR
   
2. earning the minimum number of credits (at least 28 credits); OR
   
3. if the student earned four or more grade 12 course credits during high school. Note: In Brownell et al. (2015), six or more grade 12 course credits were used.

In the Manitoba Child Health Atlas Update deliverable by Brownell et al. (2008), they investigated high school completion rates by following two separate cohorts of grade 9 students. This research introduced several slight variations in the general method of measuring high school completion/graduation from previous research. First, the two cohorts were followed for six years (rather than five years) to determine what percentage of the students completed high school in that time. The first cohort involved students enrolled in grade 9 in 1997/98 and were followed until the 2002 academic year (2002/03 school year). The second cohort involved students enrolled in grade 9 in the 2000 academic year (2000/01 school year) and were followed until the 2005 academic year (2005/06 school year). Students in First Nations schools were excluded because enrollment data for students in First Nations schools is often incomplete.

The graduation rates calculated in this study are presented in three figures:

• Figure 8.12 - High School Completion Rates by RHA illustrate the percentage of students completing high school within six years of enrolling in grade 9 for the two different cohorts used in this study, and any significant changes over time.
  
• Figure 8.13 - High School Completion Rates by Winnipeg Community Areas (CAs) illustrate the percentage of students completing high school within six years of enrolling in grade 9 for the two different cohorts used in this study, and any significant changes over time.
  
• Figure 8.14 - High School Completion Rates by Income Quintiles illustrate the percentage of students completing high school within six years of enrolling in grade 9 for the two different cohorts used in this study, and any significant change over time.

In the Health Inequities in Manitoba: Is the Socioeconomic Gap in Health Widening or Narrowing Over Time? deliverable by Martens et al. (2010), they used the same methods employed by Brownell et al. (2008) for investigating high school completion. Of note is that this report also presents and discusses high school completion rates that include / exclude students from First Nations schools. For more information, please read the section titled High School Completion (Graduation) in this report.

In The Educational Outcomes of Children in Care in Manitoba deliverable by Brownell et al. (2015), they investigated high school completion rates by Children in Care (CIC) / Ever in Care category, using the same criteria described above, except for the third criteria, they counted any students who had completed six or more grade 12 credits as graduates. For more information, see the section titled High School Completion in the on-line Deliverable.

3. Martens et al. (2010), Chartier et al. (2012) and Smith et al. (2013)

In the Profile of Metis Health Status and Healthcare Utilization in Manitoba: A Population-Based Study by Martens et al. (2010), Health and Healthcare Utilization of Francophones in Manitoba by Chartier et al. (2012), and Social Housing in Manitoba by Smith et al. (2013), High School completion rates within six years of grade 9 were measured using two criteria of high school completion from the enrollment and marks data:


1. the presence of a graduation flag in the Year End Status field; OR
   
2. earning the minimum number of credits (either 28 or more credits in 2007/08 or in previous years, 29 or more credits in 2008/09, or 30 or more credits in 2009/10 and beyond).

For more information from Martens et al. (2010), please read the section titled High School Completion Rates within Six Years of Grade 9, in this report.

For more information from Chartier et al. (2012), please read the section titled High School Completion, in this report.

For more information from Smith et al. (2013), please read the section titled High School Completion, in this report.

4. Brownell et al. (2012)

In the How Are Manitoba's Children Doing? deliverable by Brownell et al. (2012), they investigate and report on different methods of measuring high school completion. Using different methods can produce different results, as illustrated by the findings presented in this research. Three different methods were developed by MCHP and investigated in the deliverable. One method employs a cross-sectional approach similar to the provincial Department of Education for measuring high school completion / graduation, and two other methods employ the cohort approach. These methods are described below. The high school completion rates obtained using these approaches are compared with the graduation rates reported by the provincial Department of Education.

1. Cross-Sectional Approach


The provincial Department of Education uses a cross-sectional, or "proxy cohort" method for reporting graduation rates, calculated by taking the number of students graduating from high school in a given year and dividing that by the number of students enrolled in grade 9 four years previously. Information on Manitoba's High School Graduation Rates from June 2002 to June 2011 published by Manitoba Education is available at: http://www.edu.gov.mb.ca/k12/docs/reports/grad_rate/grad_rate.pdf . A more detailed description of the cross-sectional method used by Manitoba Education is available in Appendix 8: High School Completion Calculations of the MCHP report.

For this project, MCHP also developed a cross-sectional approach using available administrative data to measure high school completion rates. This method was similar, but not identical to the Manitoba Education method. The main difference is explained by the inconsistent completion of the "Year End Status" in the marks data over time and access to the paper records of graduates by the Department of Education that were not available to MCHP in the administrative data. For more information, please read the discussion available in Appendix 8: High School Completion Calculations of the MCHP report.

2. Cohort Approach


In this and previous research related to high school completion at MCHP, a cohort approach is used that follows a group of students in grade 9 for a number of years, and measures high school completion at the individual level. In this report, high school completion rates are reported for two different cohorts - the first cohort followed for four years after entering grade 9 and the second cohort followed for six years after entering grade 9.

For each approach, the following methods were consistently applied:



• the denominator consisted of the number of students enrolled in grade 9. The enrollment codes used to identify students enrolled in grade 9 are '100', '101', '105', '110', '120', '130', '135', '136', '140', '190', '210', '310', '320', '340', '390'.
  
• the numerator consisted of the number of these grade 9 students who graduated. Three criteria are used to identify graduates:
  
  1. the presence of a graduation flag in the "Year End Status" field - the codes used to identify graduates included: '0100', '0101', or '0102' - NOTE: See the document titled Year End Status Codes Used for Manitoba Children - 2012 in the Links section below (internal access only) for more information on the Year End Status codes used in this research; OR
     
  2. earning the minimum number of credits (either 28 or more credits from 2000-2007, 29 or more credits in 2008, or 30 or more credits in 2009); OR
     
  3. earning four or more grade 12 course credits.
• the following exclusions were applied at grade 9 and at the last school attended:
  
  • students enrolled at Home Schools,
    
  • students enrolled at Non-Funded Independent Schools,
    
  • students enrolled at First Nations Schools, or
    
  • students enrolled at Adult Learning Centres
• other exclusions included records with duplicate SCRPHINs, invalid SCRPHINs, invalid or missing postal codes, or missing values for Regional Health Authorities (RHAs) and Winnipeg Community areas (WCAs).
  
• other considerations in this research included:
  
  • no age restrictions,
    
  • Public Trustee postal codes are not excluded, and are put into the Not Found (NF) category for income quintile analysis, and
    
  • Churchill is combined with Burntwood for the RHA analysis.

Comparison of Methods

The methods used in this research are discussed in detail in Appendix 8: High School Completion Calculations of the MCHP report, including a list of some advantages and disadvantages for both approaches.

Appendix Table A8.1 - High School Completion Rates of Students Enrolled in Grade 9 in 1997/98-2006/07, Measured Using Cross-Sectional and Cohort Methods presents the percent of Grade 9 students completing high school using the information reported by the provincial Department of Education and the findings from the three methods employed by MCHP in this research.

5. Chartier et al. (2016)

In The Mental Health of Manitoba's Children deliverable by Chartier et al. (2016), they investigate and report on high school completion by mental disorders, developmental disorders and suicidal behaviours. For more information, see the section titled High School Completion in the deliverable.

Other MCHP Research Measuring Education Attainment Levels

Other MCHP research has investigated Grade 12 attainment levels using different methods than the cohort approach described above.

1. Oreopoulos et al. (2008)

In the publication Short, medium, and long term consequences of poor infant health: An analysis using siblings and twins by Oreopoulos et al. (2008), they estimate the level of educational attainment by analyzing Manitoba enrollment records to determine whether a student has attained Grade 12 by age 17. Not attaining Grade 12 by this age could indicate that the student entered school late for age, has been held back in a grade at least once, or has dropped out. Oreopoulos et al. (2008) use this variable as their main measure of education attainment because the birth sample is matched to enrollment records only until age 17 (for all of the seven birth cohorts used in this study). This indicator is a proxy measure for being at risk of ending up with a low level of education attainment.

2. Brownell et al. (2010) and Heaman et al. (2012)

In addition to the Education data, the MCHP Data Repository contains alternative education data sources that provide measures of high school achievement levels. Two of these sources are:


• BabiesFirst Screening Form - contains an indicator of the parental education levels, for both the mother and father of either less than grade 12, or grade 12 and above.
  
  
• Families First Screen / Screening Form - contains an indicator of the parental education levels, for both the mother and father of either less than grade 12, or grade 12 and above.

These alternative data sources have been used in MCHP research to measure level of high school achievement. In Brownell et al. (2010), they compared high school completion / graduation levels retrieved from the Education data to information collected on the Families First Screening about education achievement. They found over 85% agreement between the two data sources.

In Heaman et al. (2012), they developed an indicator called Women Giving Birth With Less Than a Grade 12 Education using the education achievement information available on the Families First Screening Form.

SAS Code

• A link to an example of the SAS code which a programmer can use to read the appropriate Education data sets to determine if someone has completed high school is available below.

  CAUTION: For programmers and researchers, please be sure to read the comments accompanying this code as they stress the need to give some thought to what you want and to whom you apply the code. For example, if you apply this code to a cohort that does NOT continuously reside in Manitoba throughout all their years in high school then this code will likely identify some individuals as non-graduates because we will not have complete high school data for the group. High School Completion SAS code (internal access only).

• In January 2013, a SAS macro was developed that processes a grade 9 cohort file using the general method described above. It produces output for the cohort following them for 4, 5, and 6 years. This macro is available in the MCHP SAS® Macro library.
  
  
• SAS® formats for various Education related data elements can be found in the MCHP SAS format libraries.

Cautions / Limitations

• Although using multiple data sources may provide a more complete picture of high school completion / graduation levels, it is suggested that the data sources be compared to determine the level of agreement and that a sequential approach be used to avoid double counting.
  
• Education data involving First Nations schools is incomplete. In addition, the Families First Screening data does NOT include information for women living in First Nations communities.
  
• In Brownell et al. (2012), students enrolled at a home school, First Nations Schools (including those administered by Frontier School Division under an education agreement), non–funded independent schools, and Adult Learning Centres were excluded prior to calculating high school completion rates.