[Previous] [Contents] [Next]

Appendix 1: Calculating the overall rating out of 10

The overall rating out of 10 (Cote globale sur 10) is intended to answer the question: "In general, how is the school doing, academically?" In order to answer this question a number of aggregations of a variety of data sets, many with dissimilar distributions, must be accomplished. Further, since the overall rating out of 10 is a key indicator of improvement over time, the method of its derivation must take into account that even the annual values within a given data set may not share statistical characteristics. For example, the mean and standard deviation of the distribution of average examination marks across schools in Language of Instruction studies may vary between English and French and within either subject from year to year. Thus the need for aggregation of dissimilar data and for comparability of data with data sets year-over-year dictated the use of standardized data for the calculation of the overall rating out of 10.

The following is a simplified description of the procedure used to convert each year's raw indicator data provided by the Ministry of Education into the overall rating out of 10 contained in the detailed tables.

1. Results in the English and French versions of History of Québec and Canada (Secondary IV) were aggregated to produce a weighted average examination mark, fail rate, and rate of grade inflation rate by the school without standardizing since we have no reason to believe that the French and English versions of the same examination are dissimilar. The two versions of Physical Science (Secondary IV) were aggregated in the same way. Note that the calculation of weighted average results use enrolment proportions as the weighting factor.
2. All the results were then standardized by solving the equation Z = (X - m)/s, where X is the individual school's mean result; m is the mean of the all-schools distribution of results and s is the standard deviation of the same all-schools distribution.
3. Since the French as a Second Language (Secondary V) and English as a Second Language (Secondary V) courses each have several distinct parts that are examined, for each course the results for these components were aggregated to produce weighted average indicator results for the course. The weighted average results for each of the two courses were then re-standardized.
4. All the aggregated standardized results as well as the two results for Language of Instruction (these two distinct data sets did not need to be aggregated prior to the calculation of the overall results) were then aggregated to produce overall average weighted average examination mark, failure rate, grade inflation by schools, and indicators of gender gap in the Language of Instruction and Physical Science. These weighted average overall results were again re-standardized.
5. The five overall results were then combined to produce a weighted average summary standardized score for the school. The weightings used in these calculations were Examination marks (50%), Fail rate (30%), grade inflation by schools (10%), and combined gender-gap indicators (10%).
6. This summary standardized score was standardized.

This standardized score was converted into an overall rating between zero and 10 as follows:

7. The maximum and minimum standardized scores were set at 2.0 and -3.29 respectively. Scores equal to, or greater than, 2.0 will receive the maximum overall rating of 10. This cut-off was chosen because the occasional, although infrequent, occurrence of scores above 2.0 (two standard deviations above the mean) allows the possibility that more than one school in a given year can be awarded a "10 out of 10." Scores equal to, or less than, -3.29 will receive the minimum overall rating of 0 (zero). Schools with scores below -3.29 are likely outliers--a statistical term used to denote members of a population that appear to have characteristics substantially different from the rest of the population. We therefore chose to set the minimum score so as to disregard such extreme differences.
8. The resulting standardized scores were converted into overall ratings according to the formula: , where OR is the resulting overall rating; is the average calculated according to the formula = (ORmin - 10 (Zmin / Zmax)) / (1 - (Zmin / Zmax)); ; and StanScore is the standardized score calculated in (5) above and adjusted as required for minimum and maximum values as noted in (7) above. As noted in (6) above, ORmin equals zero. As noted in (7) above, Zmin equals -3.29; and Zmax equals 2.0.
9. Finally, the derived overall rating is rounded to one decimal place to reflect the significant number of decimal places found in the original raw data.

Note that the overall rating out of 10, based as it is on standardized scores, is a relative rating. That is, in order for a school to show improvement in its overall rating, it must improve more than the average. If it improves, but at a rate less than the average, it will show a decline in its rating.

[Previous] [Contents] [Next]

info@fraserinstitute.ca You can contact us at the above email address for any comments or information requests. Please report any dead links or technical problems.

If you know someone who would be interested in this web page, please enter their email address below, and we will forward this URL to them: Email Address:

Last Modified: October 20, 2000.