[Previous] [Contents] [Next]

A measure of academic effectiveness for secondary schools

The foundation of the Report Card is an overall rating of each school's academic performance. Building on student results data provided by the Ministry of Education, we rate each school on a scale from zero to 10.

How does the school perform on key academic indicators?

We base our overall rating of each school's academic performance on the students' results in four core academic courses. They are Secondary V level courses in the Language of Instruction and the Second Language and Secondary IV level courses in the History of Canada and Quebec and Physical Science. The results used as indicators are:

• Average uniform examination mark
• Percentage of courses failed
• School level grade inflation
• Difference between the results of male and female students on the previous measure.

We have selected this set of indicators because they provide systematic insight into a school's performance.¹² Because they are based on annually generated data, we can assess not only each school's performance in a year but also its improvement or deterioration over time.

Indicators of effective teaching

1 Average uniform examination mark

For each school, under the heading Résultats aux épreuves the table lists the average raw uniform examination mark achieved by its students in each of the four core courses at the June examination sitting in each school year. For the purposes of determining the trend, if any, over time and the overall rating out of 10 (I), the average marks for all four courses are combined to produce an overall average mark. Detailed explanations of the method used to calculate trends and the overall rating are included in the relevant sections below.

Examinations are designed to achieve a distribution of results reflecting the inevitable differences in students' mastery of the course work. Differences among students in interests, abilities, motivation, and work-habits will, of course, have some impact upon the final results. However, there are recognizable differences from school to school within a district in the average results on the uniform examinations. There is also variation within schools in the results obtained in different subject areas. Such differences in outcomes cannot be explained solely by the personal and family characteristics of the student body. It seems reasonable, therefore, to include these average uniform examination marks for each school as one indicator of effective teaching.

2 School level grade inflation

For each school, this indicator--noted in the tables as Surestimation par l'école--measures the extent to which the average "school" mark--the accumulation of all the results from tests, essays, quizzes and so on given in class--exceeds the average uniform examination mark obtained in the four core courses. Where a school's average examination mark is higher than the average school mark, the school is assigned a zero on this indicator.

Effective teaching includes regular testing of students' knowledge so that they may be aware of their progress. As a systematic policy, inflation of school-awarded grades will be counterproductive. Students who believe they are already successful when they are not will be less likely to invest the extra effort needed to master the course material. In the end, they will be poorer for not having achieved the level of understanding that they could have achieved through additional study.

The effectiveness of school-based assessments can be determined by a comparison to external assessments of the students. The same authority--the Ministry of Education--that designed the course administers the uniform examination. This examination will test the students' knowledge of the material contained in the course. If the marks assigned by the school reflect a level of achievement that the student subsequently achieves or exceeds on the uniform examination, then the school has not deceived the student into believing that learning has occurred when it has not. It seems reasonable, therefore, to use this indicator as a second measure of effective teaching.

Indicators of equitable teaching

Effective schools will ensure that all their students are assisted and encouraged to reach their potential regardless of any real or perceived disadvantages resulting from personal or family characteristics. At such schools, teachers will take into account the characteristics of their students when they develop and execute their lesson plans. In doing so, they will reduce the probability that systematic differences in achievement are experienced by sub-populations within the student body.

1 Percentage of uniform examinations failed

For each school, this indicator--noted in the tables as Échec--provides the rate of failure (as a percentage) in the four core courses. It was derived by dividing the sum, for each school, of all the core courses completed where a failing grade was awarded by the total number of such course completions by the students of that school.

In part, effective teaching can be measured by the ability of all the students to pass any uniform examination that is a requirement for successful completion of a course. Schools have the responsibility of preparing their students to pass these final examinations.

There is good reason to have confidence in this indicator as a measure of effective teaching. First, these courses are very important to students regardless of their post-secondary plans. In order to obtain a secondary-school diploma, students must successfully complete two of these courses (Language of Instruction, Secondary V level, and History of Québec and Canada, Secondary IV level). Anglophone students must also successfully complete French as a second language at the Secondary V level. Physical Science (Secondary IV level) is a prerequisite for a variety of CEGEP courses. Second, since each of these courses has prerequisite courses, their successful completion also reflects how well students have been prepared in the lower grades. Since successful completion of the courses is critical for all students and requires demonstrated success in previous courses, it seems reasonable to use the percentage of uniform examinations failed as an indicator of the effectiveness of the school in meeting the needs of all its students.

2 The Gender Gap indicator

In a recent study of gender differences in the academic results of British Columbia students, the authors found that "there appears to be no compelling evidence that girls and boys should, given effective teaching and counselling, experience differential rates of success." ¹³ However, the study revealed that in British Columbia,

[g]irls receive higher grades on school-based assessments in all subjects regardless of their relative performance on the provincial examinations. The general rule seems to be that if an assessment or award is made at the school level, girls achieve better results than boys do.¹⁴

The data from Quebec's Ministry of Education upon which this study is based provides evidence that there are similar systematic differences in the results of these groups. For example, the 1998/1999 results show that, on average, female students score about 4½ percentage points higher on the examinations in Language of Instruction (Secondary V) than male students do, while on the examinations of Physical Science (Secondary IV), male and female students have roughly the same results. However, on school marks, female students out-score their male classmates in both courses--by about 6½ percentage points in Language of Instruction and by about 2 percentage points in Physical Science.

The gender-gap indicators (écarte sexes) demonstrate the extra gain in their results that female students in most schools enjoy on school-based assessments. The indicators are calculated as follows:

1. Calculate the difference for female students between the raw school score and the raw uniform examination score on each of two courses--Physical Science (Secondary IV) and Language of Instruction (Secondary V).
2. Calculate the same difference for male students.
3. Calculated the difference between the result in (1) and (2).

When all six years of data are considered, female students were favoured more than 75% of the time in Language of Instruction courses and 65% of the time in the Physical Science course. Why are female students seemingly at an advantage over male students in the school-based assessments regardless of their relative examination results? Do these results reflect real differences in learning or a systematic bias in school-based assessment procedures? In either case, schools with a low gender gap are more successful than are others in helping students of both genders to reach their potential.

In general, how is the school doing academically?

Overall rating out of 10 (Cote globale sur 10)

While each of the indicators is important, it is almost always the case that any school does better on some indicators than on others. So, just as a teacher must make a decision about a student's overall performance, we need an overall indicator of school performance. Just as teachers combine test scores, homework, and class participation to rate a student, we have combined all the indicators to produce an overall school rating.

To derive this rating, the results for each of the indicators, for each of the six years, were first standardized. Standardization is a statistical procedure whereby sets of raw data with different characteristics are converted into sets of values with "standard" statistical properties. Standardized values can be combined and compared.

The standardized data were then combined as required to produce five standardized scores for each school, for each year:

1. Résultats aux épreuves This is the combined average uniform examination mark achieved by the students at the school in all the courses that were considered in this study and for which results data was received from the Ministry of Education.
2. Échec This is the combined rate of failure experienced by the students at the school in these same courses.
3. Surestimation par l'école This is the combined average number of percentage points by which the mark assigned to the students by the school exceeds the mark achieved by the same students on the uniform final examination in these same courses.
4. Écart sexes : langue maternelle This is the combined average number of percentage points by which the genders differ in the calculated difference between the school mark and the uniform examination mark in the Language of Instruction (Secondary V).
5. Écart sexes : sciences physiques This is the combined average number of percentage points by which the genders differ in the calculated difference between the school mark and the uniform examination mark in Physical Science (Secondary IV).

The five standardized scores were then weighted and combined to produce an overall standardized score. Finally, this score was converted into an overall rating out of 10. (Explanatory notes on the calculation of the overall rating out of 10 are contained in Appendix 1.) Noted in the tables as Cote globale sur 10, the overall rating out of 10 answers the question, "In general, how is the school doing, academically?"

It is from this overall rating out of 10 that the school's provincial rank and its rank within the administrative region are determined.

[Previous] [Contents] [Next]

info@fraserinstitute.ca You can contact us at the above email address for any comments or information requests. Please report any dead links or technical problems.

If you know someone who would be interested in this web page, please enter their email address below, and we will forward this URL to them: Email Address:

Last Modified: October 20, 2000.