The
Economic Freedom
Network

Report Card on Quebec's Secondary Schools

[Previous] [Contents] [Next]

Appendix 1: Calculating the overall rating out of 10

The Overall rating out of 10 is intended to answer the question, "In general, how is the school doing, academically?" In order to answer this question a number of aggregations of a variety of data sets, many with dissimilar distributions, must be accomplished. Further, since the Overall rating out of 10 is a key indicator of improvement over time, the method of its derivation must take into account that even the annual values within a given data set may not share statistical characteristics. For example, the mean and standard deviation of the distribution of average examination marks across schools in language of instruction studies may vary between English and French and within either subject from year to year. Thus, the need for aggregation of dissimilar data and for year-over-year comparability of data within data sets dictated the use of standardized data for the calculation of the Overall rating out of 10.

The following is a simplified description of the procedure used to convert each year's raw indicator data provided by the Ministry of Education into the Overall rating out of 10 contained in the detailed tables.

1. Results in the English and French versions of Secondary IV level History were aggregated to produce a weighted average examination mark, fail rate, and school level grade inflation rate without standardizing. We did not standardize prior to weight averaging because we have no reason to believe that the French and English versions of the same examination are dissimilar. The two versions of Secondary IV level Physical Science were aggregated in the same way. In both cases, student enrollment proportions were used as the weighting factor.

2. All the results were then standardized by solving the equation is the individual school's mean result; is the mean of the all-schools distribution of results and is the standard deviation of the same all-schools distribution.

3. Since the Secondary V level French as a second Language and Secondary V level English as second language courses each have several distinct components that are separately examined, for each course the results for these components were aggregated to produce weighted average indicator results for the course. The weighted average results for each of the two courses were then re-standardized.

4. All the aggregated standardized results as well as the two language of instruction results (these two distinct data sets did not need to be aggregated prior to the calculation of the overall results) were then aggregated to produce overall weighted average examination mark, fail rate, school level grade inflation, Language of instruction gender gap and Physical science gender gap indicators. These weighted average overall results were again re-standardized.

5. The six overall results (for years previous to 1999/2000, the Promotion rate indicator was not used to calculate the Overall rating out of 10(1)) were then combined to produce a weighted average summary standardized score for the school. The weightings used in these calculations were as follows: Examination marks – 40%, Fail rate – 20%, School level grade inflation – 10%, combined gender gap indicators – 10%, and Promotion rate – 20%. Where only one gender gap indicator could be calculated, it received a 10% weight. Where no gender gap indicator could be calculated, the weightings used were as follows: Examination marks – 50%, Fail rate – 20%, School level grade inflation – 10%, and Promotion rate – 20%.

6. This summary standardized score was standardized.

This standardized score was converted into an overall rating between zero and 10 as follows:

7. The maximum and minimum standardized scores were set at 2.0 and -3.29 respectively. Scores equal to, or greater than, 2.0 will receive the maximum overall rating of 10. This cut-off was chosen because the occasional, although infrequent, occurrence of scores above 2.0 (two standard deviations above the mean) allows the possibility that more than one school in a given year can be awarded a "10 out of 10." Scores equal to, or less than, -3.29 will receive the minimum overall rating of 0. Schools with scores below -3.29 are likely outliers – a statistical term used to denote members of a population that appear to have characteristics substantially different from the rest of the population. We therefore chose to set the minimum score so as to disregard such extreme differences.

8. The resulting standardized scores were converted into overall ratings according to the formula: OR = + (* StanScore), where OR is the resulting overall rating; is the average calculated according to the formula = (ORmin - 10 (Zmin / Zmax)) / (1 - (Zmin / Zmax)); = (10 - ) / Zmax; and StanScore is the standardized score calculated in (5) above and adjusted as required for minimum and maximum values as noted in (7) above. As noted in (6) above, ORmin equals zero. As noted in (7) above, Zmin equals -3.29; and Zmax equals 2.0.

9. Finally, the derived Overall rating is rounded to one place of the decimal to reflect the significant number of places of the decimal in the original raw data.

Note that the Overall rating out of 10, based as it is on standardized scores, is a relative rating. That is, in order for a school to show improvement in its overall rating, it must improve more than the average. If it improves, but at a rate less than the average, it will show a decline in its rating.

The weightings used in these calculations for the school years previous to 1999/2000 were: Examination marks--50%, Fail rate – 30%, School level grade inflation – 10%, and combined gender gap indicators – 10%.

[Previous] [Contents] [Next]

info@fraserinstitute.ca
You can contact us at the above email address for any comments or information requests. Please report any dead links or technical problems.

If you know someone who would be interested in this web page, please enter their email address below, and we will forward this URL to them:
Email Address:

Last Modified: October 20, 2000.