Alberta Secondary Schools Report Card, 2000: Introduction
Prior to the preparation of the Second Annual Report Card on Alberta's High Schools (Report Card), we asked ourselves a single question: Why should we measure the performance of schools?
First, we measure performance so that the schools will have an objective benchmark against which to improve. The basic design of any organization's program for continuous improvement inevitably includes a repeating cycle of measurement of relevant performance indicators; the development of an improvement plan; the subsequent execution of that plan; and the re-measurement of the performance indicators to determine progress.
Such use of results measurement as the basis for improvement is widespread. In education, it is becoming routine. For example, the United Kingdom's Department for Education and Employment annually produces and widely distributes detailed tables of performance-related measures for primary schools, secondary schools, and colleges.1 Closer to home, the education authorities in California and Oregon have moved beyond simply collecting and disseminating performance data. Last year, California enacted the Public Schools Accountability Act of 1999, which requires that its State Board of Education develop an Academic Performance Index (a single statistic much like this Report Card's Overall rating out of ten) to measure the effectiveness of its elementary, middle, and secondary schools. In Oregon, a new state law requires the Department of Education to issue Performance Reports for the state's schools that rate the school on a number of dimensions. Like the Report Card, these overall indices are intended to answer an important question, "How is your school doing?"
The role of the Second Annual Report Card on Alberta's High Schools is to collect a variety of relevant, objective indicators of school performance into one, easily accessible, public document so that all interested parties--parents, students, school administrators, teachers, and taxpayers--can analyze and compare the performance of individual schools. Comparisons are at the heart of the improvement process. By comparing a school's latest results with those of earlier years, we can see if the school is improving or not. By comparing a school's results with those of neighbouring schools or schools with similar characteristics, we can identify more successful schools and learn from them. Reference to overall provincial results establishes an individual school's level of achievement in a broader context. Each of these comparisons is made simpler and more meaningful by the indicators, ratings, and rankings contained in the Report Card.
Second, and equally as important, the Report Card measures and reports on school performance so that parents and students can make a more informed choice of an education provider. Again, by making a variety of comparisons easy, the Report Card facilitates the process.
Third, the vast majority of Alberta's students attend schools that are wholly or partially financed by the provincial government. For the school year 1999/2000, Alberta Learning, the government department responsible for education in Alberta, has budgeted operating and capital expenditures of approximately 3.5 billion dollars for K-12 education. A government expenditure of such magnitude demands continued, independent measurement of the results flowing from that expenditure. The results should be easily available to any interested taxpayer.
We hope that the improvements and changes introduced in this edition allow it to serve each of these purposes better.
What is new in this year's Report Card?
What general improvements have been made?
The Second Annual Report Card on Alberta's High Schools includes several improvements of a general nature. Where private and francophone schools meet the criteria for inclusion, their results are now included. A number of the statistics have been redefined to make the results a more accurate reflection of the schools' performance. As is discussed in greater detail on page 7, diploma examination results are now reported regardless of the grade level of the writer. The measure of the number of diploma courses taken per student has also been revised (see page 10). In addition to these general improvements, several useful new school performance indicators have been added.
Do the socio-economic circumstances of the school's students affect results?
In this edition we have introduced an indicator of the socio-economic background of the school's student body. It enables parents and school officials to compare their school's results with those of schools with similar student-family characteristics. More information on this family background indicator can be found on page 12 below.
Is there a difference between the results of the two sexes at the school?
In June of last year, the Fraser Institute released Boys, Girls and Grades: Academic Gender Balance in British Columbia's Secondary Schools. Among other conclusions, the study found that especially where student assessment was made at the school level, girls were likely to do better than boys. Similar findings for Alberta students were reported in an internal study prepared for the Calgary Board of Education in 1997.2
The British Columbia study also found that the frequency and size of sex-based differences varied markedly between schools. The authors concluded: "Our findings suggest that the province's schools have great potential for improvement in the extent to which they enable learners of both genders to perform to their potential." 3 Analysis of this aspect of Alberta schools' performance indicates similar variations from school to school.
The Gender Gap indicator and ranking appears for the first time in this second edition. The indicator reports the sex that received the highest average school mark in each of two important courses--English 30 and Mathematics 30--as well as the actual difference in percentage points between the two results. It shows how effective the school has been in helping students of both sexes to succeed.
Improving school results for students of both sexes requires continued research and experimentation. School-level initiatives can make a difference. A statistical review conducted by the Notley High School4 in Essex, England, showed that boys were not doing as well as their female counterparts at school, particularly in reading comprehension. Teachers and school officials began experimenting with a variety of mechanisms to improve boys' learning while maintaining or improving that of the girls. These included teaching boys differently from girls, closer individual monitoring, early remedial work where required, and an overhaul of classroom seating strategies. Importantly, teachers reported that becoming personally aware of this systematic problem was of enormous value to them. That awareness came from the measurement and publication of results.
Do students in the school do better in some courses than in others?
In order to provide more in-depth information to parents, administrators, and other interested groups, the Report Card also includes the 1998/1999 results in the four diploma courses most frequently taken at the school. Both average final examination mark and the rate of participation are provided. Readers can easily compare a school's performance in a variety of courses and can also compare corresponding results among schools.
Both of these new indicators provide school administrators, teachers, counsellors, and parents with additional, useful objective data upon which to plan improvements that will make the school more effective.
What plans are being developed for future editions?
Is anybody there? Taking the pulse of the school by measuring student attendance levels.
Good school attendance--when it is matched with effective teaching--does matter and measures of attendance should be part of any assessment of school effectiveness. First and foremost, regular attendance at school is an important driver of academic success. A study of students in undergraduate economics classes found that "the difference in performance between a student who attends regularly and one who attends sporadically is about a full letter grade." 5 The study provided compelling evidence that attendance itself was a determinant of success in the course. But, it is not just grades that suffer when students skip classes. Research shows that grade-school truancy may lead to dropping out of school, may be a precursor to delinquent and criminal activity, and places students at higher risk of being drawn into behaviours involving drugs, alcohol, or violence.6 The same research cited the remarkable statistic that a three-week sweep for truants in Van Nuys, California, reduced shoplifting arrests during the same period by 60 percent. Since attendance matters to students' success and welfare, a measure of the effectiveness of schools in promoting good attendance is undoubtedly a valid addition to the Report Card's indicators of school performance.
Attendance data also provide a measure of the extent to which the school engages the students' interest. Secondary-school students will allocate their scarce time resources among school, leisure activities (both positive and negative in nature), and employment. The degree to which students make school their top priority will be reflected in the school's average attendance level. Schools where the attendance is high have found ways to motivate students to invest more time in their studies. If the school cannot compete with the local mall or play-centre, attendance rates will fall.
Is the school improving academically? The Progress indicator
On all but the Specific Course Results indicator (see page 13), The Report Card provides four years of data. Unlike a simple snapshot of one year's results, this historical record provides evidence of change (or lack of change) over time. Sometimes, however, it can be difficult to determine whether a school's performance is actually improving or deteriorating simply by scanning several years of data.
In order to detect trends in the performance indicators more easily, we have developed a progress indicator. It uses regression analysis to help identify those dimensions of school performance where there has been a real change rather than a fluctuation in results caused by random occurrences outside the control of the school. Because trend calculation is uncertain when only a small number of data points is available, we will delay introduction of this indicator until next year. By then, five years of school performance will have been accumulated for most schools.
A word of thanks
The improvements that are planned for the Report Card have been prompted, in part, by the comments and criticisms we received from teachers, school and district administrators, parents, and other interested individuals from across Alberta. We wish to thank all those who took the time to share their suggestions with us. We hope that the Second Annual Report Card on Alberta's High Schools will elicit even more ideas.
info@fraserinstitute.ca
4th Floor, 1770 Burrard Street, Vancouver, BC, Canada, V6J 3G7
Tel: (604) 688-0221 Fax: (604) 688-8539 Book Orders: 1-800-665-3558 ext. 580
You can contact us at the above email address for any comments or information requests. Please report any dead links or technical problems.