Report Card on Ontario's Secondary Schools : 2001 Edition:
Introduction
Parents want better schools. Students want better schools. Teachers, counsellors, principals, superintendents, members of local school boards, and officials in the Ministry of Education want better schools. Taxpayers and employers want better schools. While there is agreement about the need for better schools, there is no agreement about how to improve our schools. One thing, however, is certain. Any effective plan to make our schools better will require that we regularly determine whether or not the school is meeting its objectives. We must measure each school's performance.
The Report Card on Ontario's Secondary Schools (hereafter, Report Card) collects a variety of relevant, objective indicators of school performance into one easily accessible public document so that all interested parties can analyze and compare the performance of individual schools. In this way, the Report Card encourages and assists those seeking to improve their schools.
The Report Card facilitates school improvement
How will the Report Card lead to better schools? Simply measuring results is certainly no guarantee of improvement. Regular measurement of performance is, however, a necessary component of any plan for improvement. Fifty years ago, Dr. Joseph Juran1 and others pointed out the role of measurement in building more effective organizations. Juran recommended the adoption of a "quality-spiral" approach to improvement. This approach is simple. The starting point in the spiral is the definition of the school's objectives— what is it supposed to do? Then, historical performance against these objectives is documented so that the school has a benchmark against which to compare improvements. Once the benchmark is established, the school then adopts a short-term goal for improvement; develops a plan designed to achieve that goal; executes the plan; measures the results; revises the goal or plan as required; executes the plan and, thereafter, continues the spiral of action, measurement, and planning toward improvement into the future. It is to the continuous improvement of all Ontario schools that the Report Card is dedicated.
The use of the measurement of results as the basis for improvement is widespread. In many jurisdictions, data relevant to education have become routinely available. For example, the United Kingdom's Department for Education and Employment annually publishes detailed tables of performance-related measures for primary schools, secondary schools, and colleges, which it distributes widely.2 Education authorities in Quebec, British Columbia, Alberta, and New Brunswick annually release data related to K-12 school performance.3 However, the mere availability of raw data to the public is not sufficient. Our experience in British Columbia, Alberta, and Quebec suggests that action toward improvement is encouraged when clear conclusions are drawn from the data, disseminated broadly, and debated. Education authorities in California and Oregon subscribe to this notion: both have moved beyond simply collecting and publishing performance data. California, under the authority of the Public Schools Accountability Act of 1999, uses an Academic Performance Index4 (a single statistic much like the Report Card's Overall rating out of 10 to rate its elementary, middle, and secondary schools. In Oregon, the Department of Education rates each of its public schools from Exceptional to Unacceptable in student performance, student behaviour, and school characteristics. It then uses these three ratings as the basis for an overall rating of school performance.5
In Canada, The Fraser Institute introduced the first report cards on secondary schools in British Columbia in 1998;6 in 1999, it was followed by the Report Card on Alberta's High Schools.7 In October 2000, the Fraser Institute and the Montreal Economic Institute published the inaugural edition of the Report Card on Quebec's Secondary Schools.8
First, we must talk
The Report Card will only serve its purpose when its findings are openly discussed among all those with an interest in the school. But, frank useful discussion can be difficult to initiate. When the Report Card is first introduced in a province, teachers and school officials sometimes adopt a defensive attitude. They see it as an attack on their ability as professionals. It is not. Teachers, counsellors, and school administrators should be committed to continual professional development and, as every educator knows, feedback is a necessary component of learning. The Report Card provides a variety of relevant, objective feedback.
Educators would perhaps prefer that school performance data not be made public. They may worry that parents do not have the time or the expertise to analyze and interpret such information correctly. Naturally, there are aspects of the Report Card that require interpretation but a broader understanding of school results will undoubtedly follow from discussion and debate among all those concerned with the effectiveness of our schools.
Teachers and principals may fear that parents and taxpayers will understand the results perfectly well and that as a result, if the school's performance is poor, they will demand change. Dissatisfaction among parents can be a powerful motivator of improvement. Here, in the words of its principal, is what happened at one school in rural British Columbia when it found itself at the bottom of the Report Card ranking:
the fallout or publicity it brought [my school] has allowed me great license in instituting change. For that I thank you (although my thanks is somewhat like a patient thanking a dentist after a painful root canal!!!)
Surely, when teachers, parents, students, administrators, and taxpayers all have easy access to data on school performance and they share the will to discuss it frankly and in detail, Ontario schools and, therefore, the province's students will be the better for it.
Some schools do better than others
The Ontario Report Card, like all the other editions, demonstrates that some schools do better than others. Even when we take into account factors such as the students' family background, which are commonly thought to dictate the degree of success among students, individual school results differ. This finding confirms research results from other countries.9 Indeed, it will come as no great surprise to experienced parents and educators that the data consistently suggest that what goes on in the schools makes a difference to student success and that some schools make more of a difference than others.
Unfortunately, while educators are eager to trumpet the positive aspects of their school, they are unwilling to discuss its shortcomings publicly. The Report Card provides objective results— good and bad--and offers educators an opportunity to accept poor results for what they are— a starting point from which to improve.
Comparisons are at the heart of the improvement process
When Ontario's Ministry of Education (hereafter, Ministry) designed the recently introduced uniform report cards for the province's students, it required schools to include comparative information that would help parents understand the significance of their children's results. For instance, the Provincial Report Card for grades 9 to 12, includes both the student's result and median mark for each subject in which the student is enrolled. It also shows any previous marks awarded to the student earlier in the year. Historical data like this will show if the student's marks are improving or deteriorating. All such comparative statistics enable students and parents to understand the results shown on the report card better.
Likewise, comparison of results among schools provides a better understanding of the effectiveness of each school. By comparing a school's latest results with those of earlier years, we can see if the school is improving or not. By comparing a school's results with those of neighbouring schools or of schools with similar school and student characteristics, we can identify more successful schools and learn from them. Reference to overall provincial results establishes an individual school's level of achievement in a broader context.
While the Report Card is not about which schools won and which schools lost, there is great benefit in identifying schools that are particularly successful. By studying the proven techniques used in schools where students achieve high grades, less effective schools may find ways to improve. This advantage is not lost on the United Kingdom's Department of Education and Employment. Its Beacon Schools10 program identifies schools across the country that have demonstrated expertise in a wide variety of challenging aspects of the management of schools and the teaching and counselling of their students.
Comparisons are at the heart of improvement and making comparisons between schools is made simpler and more meaningful by Report Card's indicators, ratings, and rankings.
What should we measure?
While schools in Ontario may serve different student bodies or offer specialized courses or curricula, there are certain basic tasks common to all. The school's teachers should design and execute lesson plans that take into account the differences among students inevitably present in every school. They should ensure that their students master the skills and acquire the knowledge presented in each course. They should develop and use evaluation methods that provide accurate, timely feedback to students and parents about the student's progress. Effective schools will encourage their students to complete their secondary-school studies in a timely manner. They should help their students prepare for a variety of post-secondary opportunities by encouraging them to enroll in relevant, challenging courses. The Report Card presents objective evidence of the extent to which each of the province's schools meet some of these basic goals.
Our choice of school performance indicators was largely dependent on the availability of relevant data. We selected only data generated annually and maintained by the Ministry so that we could make comparisons from school to school and from year to year.
From these data, for each school, for the six school years 1993/1994 through 1998/1999, we calculated five indicators of school performance.
1 Percentage of advanced courses taken
This shows the percentage of the grade 11, grade 12, and Ontario Academic Courses (OAC) enrolled in by the students that were designated as advanced level courses.11 A school that scores well on this indicator has successfully encouraged students to take more challenging courses regardless of their post-secondary ambitions.
2 Percentage of courses passed
This shows the percentage of the grade 11, grade 12, and OAC courses enrolled in by the students that were completed with a passing grade. Higher values here indicate that the school is effective in helping students successfully complete their courses in a timely manner.
3 Core courses taken per student
This is the number of grade 12 and OAC courses enrolled in by the school's students in the important subject areas of Language Arts (English or French depending upon the school's language of instruction), Mathematics, and the basic sciences. Choosing a variety of senior level courses in these subject areas will equip students more adequately for their post-secondary careers.
4 Gender gap: Language Arts mark difference
5 Gender gap: Mathematics mark difference
These two indicators report the difference between the sexes in the average course mark received for advanced, grade 12 level English (or French for schools designated by the Ministry as having French as the language of instruction) and in advanced, grade 12 level Mathematics. Smaller gender gaps indicate that the school is providing effective teaching to all its students and that its assessments are free of any systematic bias in favour of one sex.
From these five indicators, each school's annual Overall rating out of 10 is determined. The overall ratings are intended to answer the important question: "Generally, how is your school doing academically?"
While the five indicators chosen for the Report Card provide a useful measure of the effectiveness of the academic program at each school, it is likely that the inclusion of additional measures of school effectiveness would make the Report Card even more useful. We plan to add more indicators as relevant data become available and we encourage all interested parties to suggest new measures of school effectiveness that they believe will improve the Report Card.
The Report Card can help parents choose
Where parents can choose among several schools for their children, the Report Card provides a valuable tool for making a decision. Because it makes comparisons easy, the Report Card alerts parents to those nearby schools that appear to have more effective academic programs. Parents can also determine whether or not schools of interest are improving over time. By first studying the Report Card, parents will be better prepared to ask relevant questions when they interview the principal and teachers at the schools under consideration. Of course, the choice of a school should not be made solely on the basis of any one source of information. Nevertheless, the Report Card provides a detailed picture of each school that is not easily available elsewhere.
Taxpayers have a big stake in our schools
Finally, the vast majority of Ontario's students attend schools that are wholly or largely financed by taxpayers. For the school year 1998/1999, Ontario's taxpayers spent more than thirteen billion dollars to operate and maintain the province's elementary and secondary schools. A public expenditure of such magnitude necessitates continued, independent measurement of the system's results. The measurements should be easily available to any interested taxpayer.
Does the Report Card have an impact?
While it is too early to gauge the effect of the Report Card on school performance precisely, there is strong anecdotal evidence that where it has been introduced, its impact has been positive. In March 2001, we released the fourth annual Report Card on British Columbia's Secondary Schools. Response to its publication is evidence of its effect.
A growing number of school administrators and teachers routinely study the document in search of insights into their school's performance. We regularly receive correspondence demonstrating their interest:
Regardless of the changes we have made, the results of your report have made us seriously reflect on the academic issues of this school . . . We are not doing a very good job with our academics. That is fact. The fact your report makes it public has helped us to speed up the process of [improvement] . . . THANKS!
A school teacher in the
Greater Vancouver area
While Ministries of Education routinely produce data that may be helpful to schools, educators are often unaware of their existence and value. Administrators at one school in British Columbia's Fraser Valley only became aware of important statistics while studying the Report Card:
We are wondering from what ministry report you get the number of graduates which convert to your percentage . . . We don't seem to have a report at the school that gives us a complete list and the total number of grade 12s that complete graduation.
The Report Card encourages educators to use all relevant data about school performance when they make their improvement plans.
Parents are equally interested in the Report Card. Inquiries from parents tend to be of two kinds. First, they seek objective information about the school at which their child is currently enrolled. They use this information as the basis for discussion with school administrators. Often the Report Card is the only source of information with which they are familiar. Second, in choosing a school for their children, they ask for detailed interpretation of the individual school results. Typical of the responses from parents is the following:
A friend just pass[ed] me a copy of the Province [Newspaper] that contained a copy of your well researched report. This report card of the high school[s] is most useful for [a] new resident of the Lower Mainland.
A parent in Surrey, British Columbia
Overall, the Report Cards are seen as a useful aid to the continuing effort to improve our schools. In October of 2000, we published the first annual Report Card on Quebec's Secondary Schools. Shortly thereafter, a poll of 525 parents and non-parents in Quebec asked for opinions on the merits of a variety of methods for improving the province's schools.12 More than 75% of respondents considered the Report Card an important innovation.
info@fraserinstitute.ca
4th Floor, 1770 Burrard Street, Vancouver, BC, Canada, V6J 3G7
Tel: (604) 688-0221 Fax: (604) 688-8539 Book Orders: 1-800-665-3558 ext. 580
You can contact us at the above email address for any comments or information requests. Please report any dead links or technical problems.