Introduction

In March 1998, the Fraser Institute published A Secondary Schools Report Card for British Columbia (hereafter, Report Card 1998). For the first time, a variety of relevant, publicly available data were combined to produce an academic rating of the province's public and independent secondary schools. The rating was based on student results data provided by the BC Ministry of Education. For each school, there were five indicators of school performance: (1) the average provincial examination mark; (2) the percentage of provincial examinations failed; (3) the difference between the average examination mark and the average school mark; (4) the graduation rate; and (5) the number of provincial examinable courses taken per student. From these statistics, a rating for each of the five school years 1992/1993 through 1996/1997 was calculated.

In The 1999 Report Card on British Columbia's Secondary Schools, we include explicit indicators of the trends in each indicator of school performance during the past six years. These trends should be of particular interest to parents about to decide which high school their child is to attend as well as to taxpayers interested in how their money is being invested.

Report Card 1998 generated considerable discussion. Some felt that the measurement of something so complex as school performance was virtually impossible and would only lead to frustration, confusion, and antagonism among parents, teachers, and administrators. Others accepted the Report Card 1998's overall ratings as the only evidence they needed that public schools in the province were failing miserably. Many more felt--as we do--that a report card for British Columbia's secondary schools is both a much needed tool in the effort to improve our schools and a useful input for parents interested in selecting the education provider best suited to their children's needs.

We are aware that Report Card 1998 has already stimulated action at schools where previously no action was taking place. Meetings to discuss issues arising from the data in Report Card 1998 have involved parents, teachers, counsellors, and school officials. To the extent that these meetings result in improvements, we are much encouraged.

It is in this spirit that we are pleased to present The 1999 Report Card on British Columbia's Secondary Schools. We hope that all readers will use it, not as a battleground, but as the basis for constructive discussion and action leading to real improvement. We have closely considered the comments received from educators, parents, government officials, and other interested citizens. Some of their suggestions have been incorporated into this edition. Other improvements and additions are in development and will be incorporated into future editions. However, it is important as we move forward with changes and improvements that we do not lose sight of the purpose of Report Card 1998, which is as valid today as it was when the project began.

Why Should We Measure the
Performance of Schools?

Reason 1: improving performance

Remarkably, although it is responsible for the $4 billion spent each year educating students from kindergarten to grade 12, the British Columbia Ministry of Education makes no systematic effort to determine whether each school is effective in the discharge of its duties. Until the Report Card 1998 was published, there was no easily accessible database allowing school administrators, parents, or other stakeholders to compare one school's performance with that of others--public or independent--in the school district or in the province as a whole. Nor was there a convenient means by which to compare a school's present and past performance.

What good will such measurement do? It will determine whether our schools are doing their job satisfactorily. School communities can then use these measurements as a baseline to develop an annual plan for improving the school where it is shown to be weak.

Reason 2: consumer awareness

Parents and students in many parts of the province have some ability to choose among education providers. They may choose among public schools in the neighbourhood, "magnet" schools (i.e., schools, such as the Langley Fine Arts School, that have a specific focus), private schools, and home schooling. In order to make an informed choice, parents and students need indicators of school performance. As American President Clinton stated recently, "In too many communities, it's easier to get information on the quality of the local restaurants than on the quality of the local schools." Until Report Card 1998 was published, that was the case here in British Columbia as well.

How Can We Improve the Report Card?

We have received comments on Report Card 1998 from a variety of sources. We have carefully considered them and, where appropriate, we have made improvements in this edition. Other comments have encouraged us to begin development of new indicators to be included in future editions. We thank all those who communicated with us for their valuable contribution to this work.

Criticism 1: the report card covers only
academic results

One thread running through the critical comment was that a great deal of value produced by schools went unmeasured by Report Card 1998. Some argued that the report card should be expanded to include other aspects of school performance. We accept this criticism but, unfortunately, data on schools' performance in many of the suggested non-academic areas--sport and recreation, the fine arts, applied skills and career preparation, citizenship and leadership training--are not captured in a way that produces consistent, centrally available data. Indeed, the Report Card 1998 was conceived, in part, in reaction to the paucity of systematic measures of school performance available to parents. We, as all parents, are to a great extent captives of the data made available by the Ministry of Education.

During 1999, we shall investigate the possibility of adding to our database school performance measures that are only available at the district and school level. It may be possible to collect such information using surveys of school administrators and parents.

Some correspondents felt that by focusing only on academic courses we had in some way invalidated the resulting comparison of schools' performance. We categorically reject this criticism. A primary function of the school is to enable its students to acquire the skills and knowledge embodied in the academic curriculum. The importance of measuring performance against this primary objective is in no way diminished by our temporary inability to measure other objectives.

Criticism 2: the report card only deals with Grade-12 university preparation subjects

We see value in the report card well beyond a simple rating of each school's grade-12 academic program. Students do not begin their senior studies from ground zero. They are methodically prepared for more advanced studies throughout their elementary and junior-secondary years. The quality of this preparation will obviously have an impact on results in their senior year. When we measure school performance in terms of results in senior-level courses to some extent we are assessing the effectiveness of the secondary school's feeder system in adequately preparing its students.

In next year's report card we will measure each school's success in developing its students over the secondary school years with more accuracy. We will incorporate into the report card newly available school-performance data derived from certain Grade-10 results. By doing so, we hope to provide a measure of the value added by the school over time.

What of the criticism that the ratings are based on university preparation courses and that they ignore achievement by students and underrate effectiveness of schools in a wide range of academic and applied courses not included in the production of the ratings? A review of the courses concerned should put this issue to rest. The courses that form the basis for four of the five performance indicators require the student to write a uniform provincial examination as a condition of successful completion. While it is true that results from these provincially examinable courses are used by many post-secondary institutions in the evaluation of applicants, it is not true that they are of value only to students bound for university. Of the 19 such courses offered, seven are courses in a second language, three are English courses each designed with a different student objective in mind, two are similarly specific courses in Mathematics, three are basic science courses, one is an applied science course, and the remaining three are survey courses in the humanities. It is clear that the curriculum that includes these courses provides value to students regardless of their post-secondary ambitions.

Criticism 3: private school results should be segregated from those of the public schools

Is it right, fair, and productive to include public schools and independent schools in the same report? Would it not be better to have two leagues, one for the public schools, which, it is maintained, do not select their students in any way and another--a sort of Premier league--for the independent schools that are selective in their admission policies and therefore can create a student body of excellent, motivated, and supported students. Our answer to this suggestion is a simple "no." Regardless of the method by which scarce seats are assigned to numerous applicants, independent schools are a choice that will be considered by many parents. This fact alone warrants their inclusion in the general ratings and rankings lists. More importantly, an awareness of the success (or failure) of alternative education delivery systems provides useful information for the effort to improve all schools, public and private. There is simply no good reason--given the purpose of the report card--to hide such potentially valuable data.

Criticism 4: results in Communications 12 should be excluded

Communications 12 is a provincially examinable course designed for students who need further work on the development of their communications skills. Since it is deemed not to be as challenging as English 12 or Technical and Professional Communications 12, some of our correspondents felt that it should not be included in the four course-based indicators. They suggested that by encouraging more capable students to take Communications 12, schools could improve their rating. We would be surprised and dismayed if, simply in order to improve a school's ratings, counselling staff directed students into courses that were inappropriate to their education goals. All courses for which there is a provincial final examination require that students master a curriculum. Evidence that the school has enabled students to do this is evidence that the school is performing effectively. That is precisely what the report card is intended to measure. We have decided to continue our policy of including all the provincially examinable courses in the calculation of the indicators.

Criticism 5: school-based assessments should not be compared with examination marks

One indicator given in the report card compares the average mark achieved by students on the school-based assessments to the corresponding average examination mark. Inclusion of this indicator generated considerable controversy. It was suggested that since school marks in these subjects measure aspects of student learning that cannot be measured effectively on the final examination, there need not be any correlation between the average school mark and the average examination mark. If this were the case, we would expect a fairly normal distribution of the difference between the two assessments. Some schools would prepare the student for the examination better than others, while other schools would be more effective in teaching those components of the curriculum understanding of which was tested at the school level. Why then are average school marks consistently higher than average examination marks? In 1997/98, for instance, almost 78 percent of reported average school marks were higher than the corresponding average examination marks. We intend to devote considerable effort to the question of the relationship between examination marks and school marks but this indicator will remain in the report card.

Criticism 6: schools cannot be compared
without understanding the home situation
of their students

It has been suggested that the report card should take into account the effect of certain socio-economic characteristics of the student body at each school. For example, when The Province newspaper published the results of our first report card, it reported the average income of those living in each school's postal code area. If children from different home backgrounds are often not equally well prepared or equipped to succeed in school, it may be that the school performance rating does not tell the whole story about school effectiveness. Advantaged students at one school may have an easy time learning, not because they have better teachers and counsellors, but simply because they have enough to eat and a supportive home environment.

We have considered the suggestion carefully and have decided to add an indicator of the socio-economic characteristics of students' families at each school. We believe that educators can and should take into account the abilities, interests, and backgrounds of their students when they design their lesson plans and deliver the curriculum. By doing so, they may be able to overcome any disadvantages that their students have. Such an indicator enables us to identify schools that are successful in spite of adverse conditions faced by their students at home. Similarly, it also identifies schools where students with a relatively more positive home situation appear not to be reaching their presumed potential. Thus, by comparing school performance to the characteristics of students' families we may be able to develop another measure of the added value being delivered by the school.

Using enrollment data sorted by postal code provided by the Ministry of Education and census data provided by Statistics Canada, we established a profile of the student body's home characteristics for each of the schools. We then used multiple regression analysis to determine which, if any, of the home characteristics were associated with variations in average school performance on the average mark indicator. We identified one characteristic for which there was a strong association: if the average number of years of education obtained by the female parent or by the lone parent in a single-parent family was greater, the average mark was likely to be higher. We have decided to include this statistic, "Average Education of the Female/Lone Parent," for each school in The 1999 Report Card on British Columbia's Secondary Schools. It is the first step in the development of a new indicator of school performance. At present, this statistic is only available for the province's public schools.

A full explanation of the procedure by which the contextual measure is derived can be found in Appendix 2.

Criticism 7: statistically significant change
over time should be indicated

It was suggested that, in addition to providing the raw data for a period of several years, it would be helpful to identify any important school-level changes over the study period. We think this is an excellent idea and have therefore included an indication of the direction of any statistically significant change in each school's performance on each of the indicators.

Where the indicator is positive, credit is due the school. Where the indicator is negative, the school activity measured by the indicator should be closely scrutinized to determine the cause of the decline, its importance, and any remedial action that may be undertaken to reverse the trend.

One important initial finding brought to our attention by this change indicator is that the number of improvements (265) and declines (84) in school performance is dwarfed by the number of instances (961) in which no statistically significant change has been made. It is our hope that by recognizing change, we will encourage improvement.