Introduction

This concept describes the methods reported in MCHP research to measure the performance of grade 12 students on the Language Arts Standards Test, and how the performance relates to socioeconomic status. The concept focuses on the two methods used to measure test performance outcomes related to socioeconomic status (SES) in different research projects.

By comparing results based on the methods for measuring student performance on the Language Arts Standards Test, a significantly different story unfolds.

Methods for Measuring Grade 12 Language Arts Test Performance

Two different methods have been used at MCHP to measure the performance of students on the Grade 12 Standards Tests / Achievement Tests in Language Arts.


1. Those Who Wrote the Test - the first method looks at the students who wrote the test and measures the pass/fail rate of this group.
   
   
2. Population-Based - the second method looks at the eligible population who should be writing the test, based on their age and expected level in school, and measures different outcomes for this group. The outcome measures using this method include those who:
   
   • pass
     
   • fail
     
   • are withdrawn from school / not enrolled
     
   • are in grade 11 or lower
     
   • are in grade 12 but no test result, or
     
   • dropped/absent/exempt/incomplete

For more information on this population-based perspective, please read the Grade 12 Provincial Exam Performance glossary term.

Measuring Socioeconomic Status (SES)

At MCHP, socioeconomic status (SES) has been measured using different methods, such as Income Quintiles or Socio-Economic Factor Index (SEFI). In MCHP research related to measuring grade 12 Language Arts test performance, different methods of measuring SES have been used. The specific methods used in the related research projects will be identified in the discussion below.

Comparison of Methods Measuring Test Performance

The following two studies have investigated and compared the two methods of measuring grade 12 Language Arts test performance and socioeconomic status (SES). These studies include:

1. Brownell et al. (2004)

In the How Do Educational Outcomes Vary With Socioeconomic Status? Key Findings from the Manitoba Child Health Atlas 2004 deliverable by Brownell et al. (2004), they investigated and compared the two methods of measuring Language Arts test performance. For an SES measure, they developed a 4-category scale based on socioeconomic characteristics of neighbourhoods, including unemployment rates, lone-parent households, high school completion, and female workforce participation. The characteristics are similar to the characteristics of the Socioeconomic Factor Index - Version 2 (SEFI-2), but differ slightly by including "female workforce participation" instead of "average household income". The four categories of SES in this research included: Low, Low-Mid, Middle and High.

The results of the comparison of test performance measures are presented in Figure 3 - Grade 12 (S4) Performance, by Winnipeg SES Group, Language Arts Standards Test, 2001/02.

From the figure, one can see that the bar chart on the left hand side of the image reflects what is seen when the performance measures only the students who have taken the tests. The numbers presented on the left however do not tell the whole story from a population perspective - they just report results for those who are in school, in grade 12 and writing the tests. The bar chart on the right focuses on who should be writing the test, and clearly shows a different picture based on the method of measuring performance. This picture is one that is available to Manitobans because of MCHP's ability to work with different datasets and look at things from a population perspective.

For more information, please read the section titled High School Performance and Completion in this deliverable.

2. Roos et al. (2010)

In the publication, Enhancing policymakers' understanding of disparities: Relevant data from an information-rich environment by Roos et al. (2010), they investigated and compared the two methods of measuring test performance. The SES measure is based on income quartiles calculated from median income reported by the 2001 Canada Census Data for enumeration areas. Four categories were developed from this: low, low-mid, middle and high, and they added a fifth category, "on income assistance" (IA), based on information from the Employment and Income Assistance (EIA) data. This includes any children whose families received IA at some point during their high school years.

The result of this comparison of test performance measures are presented in an updated, on-line version of the graph presented in the 2010 publication: 'Getting the "True" Story: Grade 12 Language Arts Test Performance by Socio-Economic Status (SES)'.

For additional information, please click on the link to the publication abstract available below.

Measuring Test Performance from a Population Perspective

The following two studies investigate and measure the grade 12 Language Arts test performance using the method that looks at the entire population. These studies include:

1. Brownell et al. (2008)

In the Manitoba Child Health Atlas Update deliverable by Brownell et al. (2008), they investigate grade 12 Language Arts test performance by income quintile, reporting pass rates for both urban and rural areas for two different cohorts (born in 1984 and born in 1988).

The on-time pass rates for these groups are presented in Figure 8.11: On-time Pass Rates for the Grade 12 Standard LA Test by Income Quintile

For additional information, please see the material available in Chapter 8: Education of the on-line Data Extras section for this report.

2. Brownell et al. (2012)

In the How are Manitoba's Children Doing? deliverable by Brownell et al. (2012), they investigate grade 12 Language Arts test performance by income quintile, reporting pass rates (crude percentage) for both urban and rural areas over a number of years (from 2001/02 to 2009/10).

The pass rates for these groups are presented in two different graphs:


• Figure 5.53: Grade 12 Language Arts Standards Test by Rural Income Quintile, 2001/02-2009/10
  
• Figure 5.54: Grade 12 Language Arts Standards Test by Urban Income Quintile, 2001/02-2009/10

For more information, please read the section titled Trends by Socioeconomic Status available in the deliverable.