Tuesday, 14 October 2014

How to raise our PISA scores

A rather facetious list of recommendations based on what some of the participants have done or are doing.

PISA is the Programme for International Student Assessment. Every three years, the OECD (Organisation for Economic Co-operation and Development) tests 15 year old students around the world in math, reading, and science. A lot of Canadians are upset because Canada ranked 13th out of 65 in the 2012 math test, a slight decline from previous results. The top five ranking countries in the PISA math test were Shanghai, Singapore, Hong Kong, Taiwan, and South Korea.

The PISA math test is a two hour test. There are some multiple choice questions and some long answer ones. I don't know how many there are. A selection of the questions can be found here and here. A quick look reveals that they are predominantly of the "fake world" type that Dan Meyer so dislikes. 

Based on the PISA results, some countries have concluded that their curricula and teaching are deficient. Some, like the U.K and New Zealand, are intending to use the tests as a benchmark for their education systems or at least they are tilting in that direction, and it looks like they are getting ready to retool. (see this BBC report, and this Radio New Zealand report)

To me, using the PISA tests to draw conclusions about either our curriculum or our teachers seems iffy. Judging our entire system by how well a group of teenagers did in a two hour test is like training our athletes to compete in a triathlon and then measuring our success by how well some of them did in a 100-metre sprint. 

However, in case you are really determined to improve our PISA ranking, here are some recommendations that may do the trick without spending millions to revamp everything. But be aware that sprinters do not always turn out to be good triathletes. 

1. Increase the amount of homework that the students have to do. 

Look at the top two finishers: Shanghai students average 13.8 hours of homework per week and Singapore just under 10, while Canadian students only average between 5 and 6 hours (source : Shanghai PISA team). 

2. Set up after-school training clubs. Encourage entrepreneurial teachers to run the clubs and to publish and sell PISA-type practice questions. Encourage parents to hire tutors to help their children.

Parents in the top five jurisdictions spend large sums for extra tutoring and after-school training.  In particular, the top five are know for their notorious private "cram schools". Students attend these schools in the hope of passing exams and achieving either high school or university entrance. 

3. Import instructors from higher ranking jurisdictions to teach our teachers how to teach. 

The UK is  bringing 50 teachers from Shanghai to do just this. Actually the imported teachers will help the Brits reform their math education into a system that is centred around 32 hubs, similar to Shanghai . The cost will be 11 million pounds (CAD 17.75 million), so if we are not careful, importing teachers may lead to expensive restructuring. 

4. Don't let the bottom 20 percent of our students take the PISA tests. We can do this by barring English language learners and students with low socio-economic backgrounds.

It is pretty well established that disadvantaged children do not perform well on tests, so barring them should raise our scores. Shanghai excludes migrant children from even participating in its education system, as was confirmed to me by a teacher who taught in Shanghai. The proportion of excluded students is difficult to determine, but seems to be somewhere between 20 and 50 percent. See the damning report by Tom Loveless.

5. Put pressure on the OECD folks to release the PISA scores for individual students and schools. To further increase competitiveness, institute a set of monetary rewards for schools and provinces whose students perform the best in the PISA tests, and deny those rewards to schools whose students don't do so well. 

I'm a bit late with this recommendation. The OECD has already developed PISA-based tests for schools in the United States, England, and Spain. (Condolences to our American neighbours - more tests, just what you need!) 

Let's be honest about the PISA tests. For most of us, the only thing that counts is where we rank. And this leads to my final recommendation:

6. Let's call the PISA test what it is: a competition, not an assessment. 

I'm not sure that education should be based on competition. 


  1. Hi Ted

    Yes, PISA is definitely a competition. I agree that ranking an entire school system on a 2-hour test is hardly an accurate snapshot of our school system.

    Besides, Canada is doing well. Our non-statistically valid ‘drop’ on PISA can be attributed to many things, including the inclusion of more countries from China, differences in population sampling, and the introduction of a new computerized instrument for the exam.

    I completely disagree with your ideas for improving PISA scores. More homework, ‘cram schools,’ and selected populations for higher scores might improve scores for an already privileged few. Yet even for those privileged few, higher test scores does not always equate to higher understanding.

    Ethically, is it right to expect children to spend more time sitting at a desk studying for a math test? Memorizing for a test is not going to make children enjoy mathematics? Shouldn’t all children have the chance to struggle and find their own solutions to ‘good’ mathematics problems/tasks? To enjoy math? To play? To enjoy life?

    While Canada has an excellent school system, there is room for improvement. Adding mathematics specialist teachers to elementary school would improve mathematics learning. Children rarely encounter specialized mathematics teachers until junior high. For many, junior high is too late. They may already be lost and/or decided that they do not like mathematics.

    Many elementary teachers have humanities backgrounds with limited mathematics experience. Specialized elementary mathematics teachers can help make sure that children gain the foundations they need for junior high.