Concept: Grade 9 Achievement Index

Introduction

The purpose of developing performance indicators for grade 9 students is to be able to measure how these students are doing in relation to the rest of their peers. The Grade 9 Achievement Index is one of three indices developed by MCHP to measure educational achievement. The other two indices, the Language Arts Achievement Index and the Mathematical Achievement Index, focusing on grade 12 achievements, are described in the Educational Indices concept.

NOTE: The Grade 9 Achievement Index has also been referred to as the Grade 9 Performance Index.

Data Sources

The data for developing the Grade 9 Achievement Index comes from the Manitoba Health Insurance Registry data (e.g.: demographic information on age and gender) that MCHP receives from the department of Manitoba Health, and from the Enrollment, Marks and Assessments data (e.g.: high school (grade 9-12) class enrollment and course grades / marks) that MCHP receives from the department of Manitoba Education and Training.

Methodology

A performance index was developed based on possible average marks in all classes and the number of credits earned during the grade 9 school year. The Grade 9 Index does not differentiate between required and elective courses, but currently there are five compulsory courses for grade 9 students in Manitoba: English Language Arts, Mathematics, Physical Education/Health Education, Science and Social Studies. See the Manitoba Education and Training Web Site at: https://www.edu.gov.mb.ca/k12/mychild/grade9.html for more information on the grade 9 curriculum in Manitoba.

A number of approaches were tested and reviewed. The grade 9 index was originally developed based on the characteristics of grade 9 credits completed regardless of the age of completion. The updated definition takes into consideration all courses (S1-S4 level / grades 9-12) taken by an individual in only their expected year of grade 9 attendance. Each individual was assigned to a category based on their grade 9 achievement, taking into consideration the number of credits passed and average grade.

Those who did not have any courses in their expected year of grade 9 attendance were assigned to an alternate category of either "Lower grade in expected G9 Year or Home School", "No G9 - NE (Not Enrolled), SS (Special Secondary), SE (Special Elementary) or Band School (schools specifically for Aboriginal children)" or "No MEY Record up to G9".

A scaled logit score was created based on the rank categories each individual was assigned to. Individuals who had less than seven credits completed in their expected G9 year were put into lower rank categories than those with similar marks and seven or more credits completed. The seven credit definition was used as the cut-off because of higher correlation with the grade 12 index; five and eight credit cut-offs were also explored.

The categories for each level of the achievement / performance index are defined as follows:


• 01 - 90-100 & 7+ Credits
  
• 02 - 85-90 & 7+ Credits
  
• 03 - 80-85 & 7+ Credits
  
• 04 - 75-80 & 7+ Credits
  
• 05 - 70-75 & 7+ Credits
  
• 06 - 65-70 & 7+ Credits
  
• 07 - 60-65 & 7+ Credits
  
• 08 - 80-100 & <7 Credits
  
• 09 - 50-60 & 7+ Credits
  
• 10 - 70-80 & <7 Credits
  
• 11 - 60-70 & <7 Credits
  
• 12 - 50-60 & <7 Credits
  
• 13 - No G9 - Lower Grade in Expected G9 Year, or Home School
  
• 14 - 0-50 Average
  
• 15 - No G9 - NE, SS, SE or Band School
  
• 16 - No MEY Record up to G9

The updated definition resulted in more individuals assigned to the lower categories than the initial definition, based on the number of grade 9 credits, no matter when the courses were taken. Upon investigation, this result was due to more individuals previously defined as taking G9 courses that are now defined as having no records or attending a lower grade in their expected G9 year.

The graphs in the linked document Grade 12 and Grade 9 Index Comparisons.pdf show the difference using the updated methodology versus the "old" methodology. The graphs show that those who completed 5 or 7 credits in grade 9 and get high marks in those courses, end up doing almost as well on their Grade 12 Language Arts (LA) Index scores as those who completed 8 or more credits and also achieved high marks. This is reflected in the green spikes on the graphs.

Validation of the Index

Finally, there are always multiple ways to construct an index. An important justification for using the Grade 9 Achievement Index incorporating 7 or 7.5 credits is that such a choice generates a smooth relationship with the combination of credits and grades down to the 60-65 grade category. This provides face validity and facilitates presenting and explaining the index to others. The reliability/validity checks support such a decision. For more information about validation of this index, please download and read Appendix 3: Index Validation from the What is Most Important publication on the Social Science Research Network web site, available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1978965 .

Other Considerations

Only the highest mark in a course is taken if there are duplicate courses for an individual. The following is a breakdown of the "Letter" grades an individual could have for their final mark.


• S - Standing - this indicates the credit can be challenged by the student and applies to certain courses only. Also used for courses that can't be given a numeric values grade.
  
• M - Medical exemption for compulsory physical education
  
• P - Pass - assigned a final mark of 55%
  
• CO - Complete - assigned a final mark of 65%
  
• IN - Incomplete - assigned a final mark of 0

Those with an "S" are excluded because a numeric grade could not be given for the course. Those with "M" are also excluded because they were exempted from the course. Individuals with a "letter" grade represent a very small portion of the total records in the education data set and are more present in earlier years of data. The following is a percentage estimate of some of the "letter" grades in all of the education data: "S" = 0.15%; "M" = 0.0005%; and "P" = 0.0011%

Cautions / Limitations

The following issues / limitations relate to this data:


• there are very few individuals who are "ahead of grade" - taking grade 10-12 courses in their expected grade 9 year. Because of the small numbers these cases are suppressed in the final reporting of the grade 9 index scores.
  
• individuals who had less than seven credits completed in their expected grade 9 year were put into a lower category than those with similar marks and seven or more credits completed.
  
• the seven credit definition was used as the cut-off because of its higher correlation with the grade 12 performance index. The use of five and eight credit cut-offs were also explored, but seven was chosen for the purposes of this study.