The
Economic Freedom
Network

The Fraser Institute

BC Secondary Schools Report Card, 2000:
Key academic indicators of school performance

[Previous] [Contents] [Next]

The foundation of the Report Card is an overall rating of each school's academic performance. Building on data about student results provided by the Ministry of Education,⁶ we rate each school on a scale from zero to 10.

We base our overall rating of each school's academic performance on five indicators:

average provincial examination mark
percentage of provincial examinations failed
difference between the school mark and examination mark in provincially examinable courses
provincially examinable courses taken per student
graduation rate.

We have selected this set of indicators because they provide systematic insight into a school's performance. Because they are based on annually generated data, we can assess not only each school's performance in a year but also its improvement or deterioration over time.

To make the indicators as transparent as possible we have kept manipulation of the Ministry's data to the very minimum required. The process by which the five indicators are developed involves no significant editing of the raw data. Thus, parents, administrators, teachers, or other interested parties can replicate our measures with a minimum of effort.

Three indicators of effective teaching

1 Average provincial examination mark

This indicator (in the tables Average exam mark) is the average percentage achieved by a school's students on the uniform final examinations in all of the provincially examinable courses.⁷ For each school, the indicator is the average of the mean scores achieved by the school's students in each of the provincial examinations at all sittings during the year, weighted by the relative number of students who wrote the examination.

Examinations are designed to achieve a distribution of results reflecting the differences in students' mastery of the course work. Differences among students in interests, abilities, motivation, and work-habits will inevitably have some impact upon the final results. However, there are recognizable differences from school to school within a district in the average results on the provincial examinations. There is also variation within schools in the results obtained in different subject areas (see section below, Are there any academic strengths or weaknesses at the school? Course Results for specific courses). Such differences in outcomes cannot be explained solely by the characteristics of the student body. It seems reasonable, therefore, to include the average examination mark for each school as one indicator of effective teaching.

2 Percentage of provincial examinations failed

For each school, this indicator ( in the tables Percentage of exams failed) provides the rate of failure (as a percentage) in the provincial examinations. It was derived by dividing the sum, for each school, of all provincial examinations written where a failing grade was awarded by the total number of such examinations written by the students of that school.

In part, effective teaching can be measured by the ability of the students to pass any uniform examination that is a requirement for successful completion of a course. Schools have the responsibility of preparing their students to pass these final examinations.

There is good reason to have confidence in this indicator as a measure of effective teaching. A student need only successfully complete one provincially examinable course in order to graduate. Such a student's course of study may not include the prerequisites for all post-secondary educational options but it will be sufficient for graduation from secondary school. Thus, students enroll in the provincially examinable courses, in large measure, because they want to take them. Further, their success in grade 12 reflects to a certain extent how well students have been prepared in the lower grades. All of the 20 provincially examinable courses have prerequisite courses. Indeed, depending on the school, admission to the grade 12 course may require that the student have received a prescribed minimum grade in the prerequisite lower-level course. Since the decision to take provincially examinable courses is, for the most part, voluntary and requires demonstrated success in previous courses, it seems reasonable to use the percentage of examinations failed in these courses as an additional indicator of the effectiveness of the teaching in secondary schools.

3 Difference between school mark and examination mark

For each school, this indicator (in the tables School vs exam mark difference) gives the average of the absolute value of the difference between the average mark obtained on the provincial examinations and the average "school" mark--the accumulation of all the results from tests, essays, quizzes, and so on given in class--for all the provincially examinable courses.⁸

Effective teaching includes regular testing of students' knowledge so that they may be aware of their progress. For such assessment to be useful, it must accurately reflect the student's understanding of the course material. As a systematic policy, inflation of school-awarded grades will be counterproductive. Students who believe they are already successful when they are not will be less likely to invest the extra effort needed to master the course material. In the end, they will be poorer for not having achieved the level of understanding that they could have achieved through additional study. On the other hand, the systematic deflation of grades can work to the detriment of students in those situations where post-secondary admissions and scholarship awards are, in part, based on school assessments. Students may also lose interest in a subject when their actual understanding of the material is disparaged by inadequate recognition.

The effectiveness of school-based assessments can be determined by a comparison to external assessments of the students. The same authority--the Ministry of Education--that designed the course, administers the uniform provincial examination. This examination will test the students' knowledge of the material contained in the course. If the marks assigned by the school are a reasonably accurate reflection of students' understanding, they should be roughly the same as the mark gained on the provincial examination. Thus, if a school has accurately assessed a student as consistently working at a C+ level, the student's examination result will be at a similar level. If, however, on average a school is consistently granting marks substantially different than those achieved by its students on the final examinations, then the school is not providing an accurate indicator of the extent to which knowledge of the course material is being acquired.

Two indicators of practical, well-informed counselling

During the secondary school years, students must make a number of decisions of considerable significance about their education. They will, for instance, annually decide whether to embark on, or continue, the learning of a second language. Before grade 9, they are required to choose between different streams in Mathematics. In grade 12, they may face the choice of completing high school or abandoning it in favour of full-time work.

Will these young people make good decisions? It is unrealistic to presume that they can do so without advice. What practical, well-informed counselling can they call upon? While parents, in the main, are willing to help, many lack the information they need to be able to provide good advice. It falls, therefore, to the schools to shoulder some responsibility for advising students and their parents about educational choices.

The final two indicators used in the calculation of the Overall rating out of 10 assess the counsel given by the schools by measuring the quality of the decisions taken by the students about their education. Of course, wise students will seek guidance not only from the counsellors designated by the schools but also from teachers and administrators, parents, and other relatives. Where students have strong support from family and community, the school's responsibility for counselling may be lighter; where students do not have such strong support, the school's role may be more challenging. These indicators measure the school's success in using the tools at its disposal to help students make good decisions about their education.

There are two very important decisions that senior students must make. First, they must decide whether or not to take a number of academically challenging provincially examinable courses. Second, having made it through school to the end of September in grade 12, they must decide whether to stick it out, do the work, and graduate with their class.

A decision in the negative would be comfortable for a student, especially one who lacks the kind of support that we are trying to measure. Students can quite easily rationalize taking less rigorous courses in grade 12 on the basis that these courses more closely parallel their present interests. Likewise, there are all sorts of reasons that can be advanced for deferring graduation: "The few courses I need can be picked up later." "I'm going to fail anyway, so why try?" "There's a job that pays $15.82 an hour available right now, so I can't afford to stay in school." The list is conveniently long. The decisions to be measured have been chosen because students without well-informed counsel may well select the more comfortable yet perhaps less productive options.

1 Provincially examinable courses taken per student

This indicator (in the tables Exams taken per student) measures the average number of provincially examinable courses completed by the students at a school. It is derived by summing the participation rates for all the provincially examinable courses taken at that school. (The participation rate is the ratio, for a school, between the number of students writing the provincial examination in a particular subject and the number of students enrolled in grade 12.)

In their senior years, students have freedom to choose from a considerable variety of courses. Their choices will have an impact upon their literacy, numeracy, and analytical skills upon graduation. Their choices also affect the variety of post-secondary options open to them.

Provincially examinable courses offer study at the senior level in a variety of core disciplines: English, other languages, the sciences, Mathematics, and the humanities. Course offerings in each area include alternatives that reflect the post-secondary ambitions of different groups of students. So, far from being courses only for a university-bound elite, these courses teach skills and knowledge that will benefit students no matter what they plan to do after graduation. Further, it is the marks obtained in these courses that are commonly used by post-secondary institutions--institutes of technology and community colleges as well as universities--to assess the applicant's readiness for further study and for admission to programs with limited enrollment. Thus, for most students a decision to take advantage of these courses is a good one and a school that is successful in encouraging students to take these courses shows that it offers practical, well-informed counselling.

2 Graduation rate

This indicator compares the number of "potential" graduates enrolled in the school on September 30 with the number of students who actually graduate by the end of the same school year. Only those enrollees who are capable of graduating with their class within the current school year are included in the count of potential graduates.

Graduation from secondary school retains considerable value since it increases options for post-secondary education. Further, graduates from secondary school who decide to enter the work-force immediately will on average find more job opportunities than those who have not graduated.

By completing the 11 years of schooling in preparation for the final secondary school year, students have already demonstrated a reasonable ability to handle the basic courses offered by the school. Moreover, for the majority of students, the minimum requirements for graduation are not particularly onerous. The chance that students will not graduate solely because they are unable to meet the intellectual demands of the curriculum is, therefore, relatively small.

Nevertheless, the graduation rate varies quite widely from school to school throughout the province. While there are factors not related to education--emigration from the province, sickness, death, and the like--that can affect the data, there is no reason to expect these factors to influence particular schools systematically. Accordingly, we take variations in the graduation rate to be an indicator of the extent to which students are being well coached in their educational choices.

In general, how is the school doing academically?
The Overall rating out of 10

While each of the indicators is important, it is almost always the case that any school does better on some indicators than on others. So, just as a teacher must make a decision about a student's overall performance, we need an overall indicator of school performance ( in the tables Overall rating out of 10). Just as teachers combine test scores, homework, and class participation to rate a student, we have combined all the indicators to produce an overall school rating. The Overall rating of school performance answers the question, "In general, how is the school doing, academically?"

To derive this rating, the results for all the years were converted into a score out of 10 using the following procedure. For each indicator, the results for the base-year (1992/1993) were sorted from highest to lowest. They were then divided into 10 ranges and each range was assigned a score between 10 and 1. The range that included the top 10 percent of results was given a 10; the next range, a 9; and so on. The results from each subsequent year were then assigned the number score corresponding to the range of values established in the base-year into which each fell.

The number scores for the five indicators were then averaged to produce the annual Overall rating for each school. The decile range tables for each of the indicators are provided in Appendix 1.