Magic and Arithmetic Series

Now your high schoolers have learned the two formulas for the sum of an AP. (Do they still call it an Arithmetic Progression?)  So you ask them this:

*   *   *   *   *

Arithmetic sequences and series. I cannot think of a more mind-numbing introduction to them than the way it was done a century ago when I was in school. And a quick googling suggests that the situation may not have improved very much — what I see often begins with a caveat that “You won’t actually need this until you take Calculus.”  Hard on the heels of this are the definitions of the first and last terms, the common difference, and so on. Then comes the formula for the general term, and finally the iconic derivation of the formula for sum of an arithmetic progression.  

Like many math teachers, I also used to tell my students the story of the clever young Gauss. It probably firmed up their belief that you have to be born with a math brain in order to do math.  Raise a glass to Kate Nowak for what she did to introduce AP's. In fact it is her post that prodded me to write this.

*

If I were able to tardis back a few years, I would probably begin with this: 

Give me the sum of three consecutive numbers.

If the students were to tell me that the sum is 72, I would tell them immediately that the three numbers are 23, 24, and 25. 

And if then I might offer this:

Give me the sum of three consecutive even numbers.

If the sum is 84, I would tell them that the three numbers are 26, 28, and 30.

Perhaps even this:

Give me the sum of five consecutive numbers.

For example, if the sum is 45, I would tell them immediately that the five numbers are 7, 8, 9, 10, and 11.

*

The secret to this is that whenever you have an arithmetic series with an odd number, n, of terms, the sum is always n times the middle term.

It is easy to convince kids that this is the case for the simple cases given above. For example, three consecutive numbers with a middle term m must always be of the form

m - 1,     m ,      m + 1.

Adding, we get 3m. To perform the trick, divide the sum by 3, and subtract 1 to get the first number.

For three consecutive even numbers, the situation is pretty much the same. The three numbers would be 

m - 2,     m,     m + 2,

Adding, the sum is again 3m.

In case you are wondering about doing the trick when you are given the sum of four consecutive numbers: dividing that sum by 2 gives the sum of the middle pair of numbers from which you can easily deduce what the four numbers are.

For example, if the sum is 50, then the sum of the middle pair is 25, so the middle pair is 12 and 13, from which we get the four numbers 11, 12, 13, 14. 

The sum of an arithmetic series with an even number, n, of terms is always n/2 times the sum of the middle pair. Interestingly, the sum of the middle pair is also the sum of the first term and last term. 

You can pursue this far enough to derive the two formulas for the sum of an arithmetic progression, but I’m not sure that I would push it that far.

Instead I would switch the question around to finding the sum of a longer list of consecutive numbers, à la Kate Nowak.

*

The possibility of introducing arithmetic sequences and series in this manner grew out of a simple trick from William Simon’s book Mathematical Magic.  

A calendar trick

Ask a student to draw a rectangle around three consecutive dates on a calendar month, like so:

This is to be done so that you, the teacher, cannot see what dates have been encircled (for example, have the student stand behind your back while you are facing the rest of the class). Ask the volunteer to show the circled dates to the rest of the students (but not to you) and to tell you the sum of the dates. You can immediately announce the dates that have been circled. 

[A personal aside: I am notoriously poor at mental arithmetic — a brief description of my troubles is contained here. Using a calendar appeals to me because it forces the three numbers to be small, thus avoiding the floundering that would occur if some cheeky person asks me “What are the three numbers if the sum is 14691?”] 

After explaining how to do the three-in-a-row trick,  the calendar itself might prompt students to ask questions like this:

How would you do it if we gave you the sum of four consecutive dates?

How would you do it if we gave you the sum of three dates in a vertical line? 

What if we gave you the sum of five consecutive dates? 

The second and third questions the students could answer themselves.

When there are four consecutive dates, there is no middle date and the sum of the four dates is not divisible by 4. I imagine this might be a stumbling block for some students. But, fingers crossed, at least some of them will actually look at what happens when they do divide by 4, and thereby open up other avenues to explore. 

They might, for example, note that dividing by 4 gives the number that is the average of the middle pair of dates.  For the four dates circled above, when I divide the sum by 4, I get 5.5, which is the average of the two middle dates 5 and 6.  And not only that, 5.5 is also the average of the first and last dates.

That is: 

Which generalizes to 

But as I said earlier, I probably wouldn't push it this far.

*   *   *   *   *

What is the sum of the following arithmetic progression? Each square represents a term in the progression. 

What about this one? 

Or this one? 

Teaching math to pre-service teachers (3)

Previously, I posted about teaching a class of students who, for all intents and purposes, lived in different mathematical time zones. Every new mathematical topic presented a challenge, most especially when it was first introduced. Mention a new math concept right off the bat and I would lose the weaker students. Make it so simple as to be trivial and I’d lose the stronger ones. 

I needed a way to introduce new mathematical material that appealed to all, one that was non-threatening to the weaker students, yet just perplexing enough to capture the interest of the stronger students. I wanted an opening with a mathematical floor so low that it didn't even appear to involve any math at all, but which at the same time contained the core idea(s) underlying the new material.

One tactic that worked for me was to introduce things via a good puzzle, and I gave an example in the previous post. Another tactic, which I'll write about here, was to start things off with a math-based magic trick. This was not quite as successful as using a puzzle, but it did work. 

As was the case for initiating a topic with a puzzle, before doing the trick I never announced what mathematics was involved.

Here, for example, is a trick that, although simple in the extreme, permits an approach to a topic that usually causes eyes to glaze over.  

The 1001 trick

You place three playing cards face down on the table.

You invite a volunteer to write a three-digit number on the board. She writes 732.

You ask the volunteer to write the six-digit number that is formed by repeating the three-digit number alongside itself. You may have to explain that abc becomes abcabc. 

She writes 732732 on the board.

Then you ask the class to choose one of the three cards. You turn over the indicated card and it is a Jack. We are using the cards as counting numbers so that  Ace = 1, Jack = 11, Queen = 12, and King = 13. 

You ask them to divide the six-digit number by the Jack: 

732732/11 = 66612.

You ask them to indicate another card. You turn it face up and it’s a 7.  You ask them to divide the new number by 7.

66612/7 = 9516.

Now you tell them that when they divide that number by their original number, the result will match the card that is left. So they divide 9516 by 732:

9516/732 = 13.

You turn the remaining card face up and it’s a King!


Explanation 

It doesn’t matter what three-digit number is chosen, but it does matter what three cards are used—they must be a 7, a Jack and a King.  (After doing the trick, I always told the students this fact. I would usually repeat the trick, having them select different cards from the 7, Jack, and King.)

The trick works because 

732732 = 732 x 1001,  

and   

1001 = 7 x 11 x 13. 

So, dividing 732732 first by 11, then by 7, and then by 732 amounts to this:

732732 x (1/11) x (1/7) x (1/732) 

which is really this:

732 x 7 x 11 x 13 x (1/11) x (1/13) x (1/732).

Doing the trick a few times lets the students see that the order in which they selected the cards does not make a difference. In the past, most of my students had heard the words “commutative law” and “associative law”, but many hadn’t remembered these laws or seen them in action. The trick would not work if those laws weren't true. So I used this trick as a jumping off point to discuss the basic laws of arithmetic.

(I should mention that the fact that 1001 = 7 x 11 x 13 is the basis of an interesting test for divisibility by 7, 11, or 13, so the trick had some relevance later in the course.)

 * * *

Here's another simple trick that I have used to introduce the same topic (I can't recall where I first saw it):

Your favourite number

Everyone has a favourite digit. You can’t have 8 because that’s mine. The digits that are left form the magic number 12345679. 

Ask a volunteer to tell you his/her favourite digit. 

Jan tells you 6. You tell her that her magic multiplier is 54, ask her to multiply the magic number by her magic multiplier,  and this happens:

12345679 x 54 = 666666666.

Henry tells you 4. You tell Henry his magic multiplier is 36 and this happens:

12345679 x 36 = 444444444.

Explanation

You get the magic multiplier by multiplying the favourite digit by 9.

It so happens that 111111111/9 = 12345679, so multiplying the favourite digit, n, by the magic multiplier (9 x n) does the following:

12345679 x (9 x n) 

= 111111111 x (1/9) x 9 x n 

= 111111111 x n =  nnnnnnnnn.

And the trick would not work if the associative law of multiplication did not hold.  

* * *

The cup of coffee swindle

You shuffle a deck of cards and place the top card face down on the table. You predict that card’s value will be the same as a randomly chosen number.

With your back to the board so you cannot see, a volunteer writes any three-digit number on the board. It can be any number as long as the digits are different. 

The volunteer is asked to scramble the digits to form a different number. The smaller number is then subtracted from the larger and some arithmetic is performed, all of this being done out of your sight. 

Apparently unknown to you, the computation results in the number zero. (Your back is still turned and you do not see the answer.)  The students are sure that you have made a mistake. However when the volunteer turns the prediction card face up, it is revealed to be totally accurate—its face is completely blank.

Explanation

(You can get a blank card from any magic supply shop or you can manufacture one yourself). 

Of course, the shuffle is fake—it’s done so that you retain the blank card on the top of the deck. And any three-digit (or two-digit or four-digit) number may be used as long as not all the digits are the same.

Here are the instructions that you give for the computation after the smaller number has been subtracted from the larger one: 

“Add  the digits of the result. Don’t tell me your answer. If your answer has two or more digits, add the digits of your answer together. Keep doing this until you get a single digit. Tell me when you get a single digit, but don't tell me what that digit is.

“Now, subtract that single digit from 9. Don’t tell me the answer, because that’s the number that the card is predicting.”

Here’s how the computation would proceed if the randomly chosen number was 128:

Scramble the digits:  812 

Subtract the smaller number from the bigger one:  812 - 128  =  684

Add the digits of the answer:   6 + 8 + 4 = 18

Two digits, so add the digits again:   1 + 8 = 9

Single digit, so subtract the answer from 9:   9 - 9 = 0.


That was how I sometimes introduced a module about digital roots and divisibility tests. My students all knew the tests for divisibility by 2, 5, and 3, but strangely, they hardly ever knew the test for divisibility by 9. 

In case you don’t know what a digital root is, it’s just the sum of the digits of the number repeatedly done until you end up with a single digit. If the result is 9 (or zero), the number is divisible by 9. 

This trick uses the fact that, no matter what number you start with, the subtraction always results in a number whose digital root is 9 (or zero). And after the trick, the students were usually interested in seeing why that happens and why the test for divisibility by 9 works.


I usually began my presentation of this trick by saying 

“I have won many cups of coffee with this test because it works almost all of the time, even though people want to bet me it can’t be right. But it’s not fair to swindle people so I won’t take any bets this time.” 

But that's not the reason I call it The cup of coffee swindle. The first time I used this trick, I didn’t preface it with anything, and one student bet me a cup of coffee that I was wrong. When I turned up for the next lecture, there was a cup of Timmy’s sitting on the table, but the student wasn’t there. She was embarrassed and angry, and she skipped the next three lectures. I feel bad about it to this day.

Teaching math to pre-service teachers (2)

In my previous post, I described a fundamental incompatibility between the students in Math 160: many students had consciously avoided math for most of their lives, while others were very confident in their abilities and even enjoyed the subject.

I ended the post with this paragraph:

When I began teaching Math 160, I had very specific ideas about how I should present the course. To me, the key to successful instruction was to concentrate on the clarity and explicitness of my lectures. It turns out that I was singing from the wrong songbook

I am reminded of that old but useful proverb: beauty is in the eye of the beholder. Or, in this case, ‘clear and explicit’ lives solely in the mind of the student.  For my Math 160 students, who differed so much from each other, there could be no interpretation of  ‘clear and explicit’ that would be suitable for all of them. Being ‘clear and explicit’ was not the way to engage them.

If you have tried to motivate students about mathematics, you know that the sticking point occurs at the very moment that you begin to talk about it. If you blow that moment, you will lose them for the duration.

So, after a period of time (too long a period), I shifted my attention towards what I could do to introduce new topics in an interesting way. 

For Math 160, there were certain constraints. I needed to make the introduction understandable without it depending upon prior knowledge. In particular, I wanted inexperienced students to be able to comprehend what was happening without a preliminary lecture or review. At the same time, I did not want the introduction to be obvious or trite to those who did have some previous knowledge. And one of my key tenets was that new terminology must be avoided so that students are not intimidated or distracted by strange jargon. And above all, it has to be interesting. 

Here’s how I usually introduced a certain challenging mathematical topic to my Math 160 students. It was my way of saying “Math 160, say hello to modular arithmetic.”

The Keystone Kidnapper

The police of Keystonia have a crack swat team. The team’s current mission is to rescue the prime minister's daughter who is being held hostage by an evil-doing kidnapper. But, through a sequence of mishaps, the swat team has allowed the kidnapper to capture them. He has herded them, along with the young lady, into a room containing seven caskets.

“Your puny intellect amuses me,” said the kidnapper. “You have one hour to figure out how to escape from this room. Sixty minutes after I leave, six of the seven caskets will disintegrate releasing one hundred angry venomous snakes. The other casket contains the key to the door. Find the key before the hour is up and you can escape. I'll give you a clue—the key is in casket number 54321. And I've done you a favour. The caskets are not locked.”

“But,” said the swat team leader, “The caskets are numbered from 1 to 7. None of them is numbered 54321.”

“I’ll give you another clue: start counting,”  and the kidnapper showed them how. 

Casket 1 was 1, casket 2 was 2, and so on until casket 7 which was 7. Then the kidnapper reversed directions: casket 6 was number 8, casket 5 was number 9, and so on until casket 1 which was number 13. Then the count reversed once more, and casket 2 was 14, casket 3 was 15. 

“You get the idea,” said the kidnapper, “Goodbye,” and he left them in the locked room.

“Well,” said the swat team leader, “Let's start counting.”

“Hold it,” said the young lady. “We have less than 60 minutes—that's 3600 seconds, and my calculator says that 54321 divided by 3600 is about 15. We would have to count 15 caskets per second and not make a mistake.” 

Luckily for the swat team, the prime minister's daughter figured it out. 

Which casket contained the key?

With its corny dialog and silly scenario, there is nothing real world about this puzzle, and there is no pretence that there is. But, as long as humans have walked the earth, people have willingly immersed themselves in tales set in unreal worlds. Using story puzzles to introduce new mathematics leans on this. It's a tactic that I used a lot, and for the most part my students let themselves be drawn in.

Spoiler Alert! If you want to solve this on your own, don't read below this line.

Here’s a short summary of how the lesson typically developed and how it fit the constraints that I mentioned earlier.

To set thing up, and to illustrate the counting process, I drew a diagram:

Usually the students asked me to write down a few more rows to clarify the counting process, and if they didn't, I wrote them down anyway, like this:

So far, all students were drawn in, and all had a clear grasp of the problem. And there was no alarming “mathematics” required. And the stronger students showed no sign of leaping to the solution. 

There was often some dead air time as they thought about the situation, and sometimes they had to be prompted with questions about what they noticed. Eventually, they started to see patterns. 

For example, some them noticed that in the leftmost and rightmost columns, successive numbers differed by 12. Or they noticed that in the second column the differences alternated between 10 and 2, and that in the third column the differences alternated between 8 and 4,  etc. 

The observations varied from class to class, and did not always occur in the same order. Sooner or later someone usually picked up on the ubiquity of the number 12, and said something like

“In all of the columns, the numbers increase by 12 for every two rows.”

With more prodding, they described what was happening by saying things like

“In the second column, all of the numbers are divisible by 12 or else their remainder is 2.”

Then, typically, someone will follow this up with:

“In the third column, when you divide by 12 the remainder is always 11 or 3.”

The key word remainder inevitably came up, but I avoided using the term until the students brought it into the conversation themselves. I sometimes had to find a way to push them towards thinking about remainders (by writing a few more counting rows and asking if they had any questions about any of the columns). I definitely never mentioned the words modulus or congruence or residue classes, even though they were grappling with these notions. They really didn't need them at this point, and there would be ample time to define them later.

As soon as someone mentioned remainders, they noticed the same sort of thing happening in other columns, which led eventually to making a chart of them:

Occasionally someone would express worries that this was leading nowhere. It is a difficult comment to handle. But, usually by this point in the course they will have learned that sometimes you need to go further down a road before you can decide where it leads. I would respond by saying something noncommittal like “Hmm.”

They noted patterns in the chart of denominators, for example, that although the remainder 10 appears repeatedly in the fourth column, it never appears in any other column. They usually also pointed out that the chart contained every remainder that you can possibly get when dividing by 12. (This sometimes initiated a conversation about whether zero should be considered to be a remainder.)

And finally a flash of understanding happened and they realized that they could identify the casket that contained the key by determining what column contained the remainder when 54321 was divided by 12. 

As I said, this is a summary, and it would be misleading to say that all classes followed the same script.  There were often lengthy pauses. There were unexpected observations that sometimes took us off-track. (As happened, for example, when someone noted that in many columns the sum of two consecutive remainders is always 14.) I tended to run with such diversions and let them play themselves out. Usually, someone interrupted and suggested something else that got us back on track.

In spite of the occasional detours, the classes always solved the puzzle well before the lecture period ended.  And I always pointed out that they found the casket well within the hour set by the kidnapper, which seemed to please them. 

* * *

The Keystone Kidnapper puzzle satisfied the constraints that I mentioned. The weaker students did not need either a preliminary lecture or new terminology to understand the puzzle. The quirkiness of the setting was captivating enough to conceal the solution from even the stronger students. And even if they knew something about clock arithmetic, they did not see the connection with the puzzle. 

The fact that the students at all levels worked through the puzzle together provided a common frame of reference. It helped overcome the non-uniformity of their backgrounds and provided a common base to build on.  Afterwards, when the new mathematical concepts of modulus, congruence, and residue classes were formally introduced, all the students could relate them to the same concrete setting. 

* * *

I don’t know the original name for The Keystone Kidnapper puzzle. I first saw it many years ago in a book by Martin Gardner I think, where fingers were being counted instead of caskets. (Several solutions for the finger counting version can be found on the internet. Almost all of the good puzzles have solutions somewhere on the web. If you use them, you might wish to alter the setting as well as the puzzle name so that students won’t be able to easily google them.)

As well as using puzzles to help introduce new topics in my courses, I occasionally used math based magic tricks. In the next post I will show how I used a couple of them, along with some pitfalls that have to be avoided. 

Teaching math to pre-service teachers (1)

They’re going to be teachers, not mathematicians. 

In Alberta, the government certifies elementary teachers, but not with regard to what they are qualified to teach. Elementary teachers are generalists, and all of them are expected to be able to teach mathematics.  

In the past, the Faculty of Education at the University of Alberta encouraged Elementary Education students to take one or more half-term courses offered by the math/stats department. Some opted for a rudimentary Calculus or Statistics course (neither of which is still offered by our department). Others took Math 160, a course which I taught many times. (It went by the spiritless and dismaying name Higher Arithmetic, and it was restricted to Elementary Education students.)

Back then, math was a bad thing for aspiring elementary teachers. Previous encounters with it had intimidated them. Math avoidance had become part of their lifestyle. The exasperating fact was that they were actually very smart, and abundantly capable of understanding the math that they needed. 

Of course, not all of my students had spent their lives sidestepping math. Some were very strong mathematically, and they actually liked the subject. 

So, there was a large variance in the students’ readiness to take on more mathematics, and it raised some concerns: Should I cater to the math avoiders and reteach them “the basics”, or should I dip right into the “higher level” topics for the stronger students? Or is there some middle-of-the-road compromise? 

Here are three options which appear to offer some resolution and which I occasionally found quite tempting. 


Option 1. Treat Math 160 as a sieve. 

In a sieve course, the professor usually has in mind standards that should be achieved, and he or she sets up the course accordingly. The standards are generally quite high and inflexible, and only those students who approach a predetermined level of attainment get a passing grade.

Rationale: Somehow, the weaker students gained admission to our university, and that’s not our fault. Teach the course at the appropriate level and let the students sink or swim. Weed out the weak ones before they become teachers.

Counterargument: It’s not always clear what the the words “appropriate level” mean. Besides, this won’t actually weed out the so-called weaker students. Remember, many of them are expert at math avoidance. If pushed, they’ll find a way to circumvent the math requirement. And when they become teachers they’ll still believe that math is a bad thing and will transmit their aversion to another generation. 

Even worse, a sieve math course will weed out some brilliant arts and humanities teachers.

Caveat: I do not advocate a complete absence of standards.


Option 2. Treat Math 160 as a math appreciation course.

By a math appreciation course, I mean one that discusses the accomplishments and personalities of mathematicians and explores how math has played a significant role in civilization.

Rationale:  Such a course shows students the human side of math and de-emphasizes the hard mathematics that frightens so many. It will show students that math is something that is worthwhile learning and will encourage them to learn more. 

Counterargument: Adulation of great mathematicians spreads the myth that you have to be a “math brain” to learn basic mathematics. An appreciation course for students with a weaker math background will be high on memorization, low on developing mathematical thinking & problem solving skills. The course may end up doing exactly what the weaker students are doing, namely avoiding math. 

Caveat: I do think that university math departments should offer courses that deal with the sociological aspects of mathematics, but, I also think such courses should have a fairly heavy dose of mathematics and therefore should be restricted to students with strong math backgrounds. 


Option 3. Treat Math 160 as a remedial course.

Ahh! The teach-them-the-basics approach. By “basics” I mean stuff that usually involves the so-called standard algorithms along with lots of shortcuts and tricks and strange rules like cross-multiplication or FOIL or BEDMAS¹. The course should include lots of drill and practice so that the students become fluent with the material that they should have already mastered.

Rationale: Too many Math 160 students lack mastery of the basic skills—for example, some cannot even carry out the long division algorithm². It’s our job to remediate the situation—the students should know the basics and should be able to perform the standard algorithms automatically. 

Counterargument: For a large number of people, the basic arithmetic facts and routines are very abstract and highly cryptic. Drill, practice, and contrived exercises will not only fail to alleviate their view that math is impenetrable, but will actually reinforce that belief. Math avoiders will become even more reluctant to learn mathematics. And the course would be a great disservice to the stronger students. To them, Math 160 would become a Mickey Mouse course. They would see it as an easy “A”, and would derive no benefit from it.

Caveat: In spite of my displeasure with the pedagogical advice by the back-to-basics people, I do believe that students should understand the fundamentals of arithmetic and be comfortable working with them. 

—

Teaching problems arise in any course where there is great disparity between the levels of the students³. I think the counterarguments above show that none of the three options resolve that situation for Math 160. In reality, all three skirt the issue by tilting the content towards one end or the other of the spectrum of student abilities.  

When I began teaching Math 160, I had very specific ideas about how I should present the course. To me, the key to successful instruction was to concentrate on the clarity and explicitness of my lectures. It turns out that I was singing from the wrong songbook.

In a subsequent post I will describe an approach for Math 160 that actually worked (at least for me).

—

[1] Incidentally, when I went to school, BEDMAS was called BOMDAS. You may called it PEDMAS or PEMDAS. The four algorithms don’t always produce the same result. James Tanton has an illuminating essay about the order of precedence for arithmetic operations. 

[2] This shocking state of affairs occurred in the era before there was any commotion about “discovery math”, so don’t go placing the blame there.

[3] Judging by what I see in teachers’ tweets and blog posts, the disparity in K-12 courses is even greater than in university courses. 

PISA: snapshots with a fogged up camera

When I was an undergrad I had a friend who lived by the motto “It’s easier to be infamous than it is to be famous.” He was a notorious “sh!+ disturber”. He caused havoc all over the campus, and he enjoyed it.

A certain Edmonton Journal columnist seems to have inherited my friend’s mantle. He enjoys tweaking the noses of the education community. I suspect his “sh!+ disturbing” also gives him much enjoyment. His weapon of mass disturbation is PISA.

Every three years the OECD tests a large sample of 15-year olds in Mathematics, Science, and Reading. This is PISA, the Programme for International Student Assessment. The OECD describes the programme as providing a snapshot of the state of education in the participating countries. Their objective is clearly stated: PISA is intended to guide the development of education policies around the world. 

Following each round of PISA exams, OECD releases a welter of results, presenting them as league tables, that is, as a list with the highest scoring countries at the top and the lowest scoring countries at the bottom. Newspaper columnists love the league tables.

Although I like to see how our education systems stack up internationally, I am not a supporter of PISA. There are lots of questions that I find unsettling. Here are two.

How reliable are the PISA results?  

Imagine that you are a teacher, and that you have decided to use a two-hour final exam to assess your students. You concoct a list of questions covering all aspects of the course, but you realize that it would take a student up to four hours to complete all of these questions. 

Here’s how you get around the problem: You have a lot of additional information about each student (assignments, quizzes, midterms, classroom observations). So you set a final exam by using a subset of the questions and you fill in the missing data by using all that extra information. 

Now imagine that you are the PISA examiner. To fully evaluate the students you also need a four hour exam. As before, each exam paper is only permitted to be two hours in length. 

But now there is a real problem: you do not have access to all that wonderful additional information. If you set a two-hour exam, there will be an awful lot of missing data, and you will have no way to compensate for it. However, in order to get reliable results, you absolutely need to use all the questions that you have concocted.

Here’s how PISA gets around the problem. The examiners are really not interested in individual results. They only want the aggregate picture. So they distribute the questions among different exam papers. The exams are not the same—the questions on Mary’s paper will be different than the ones on Peter’s paper. In aggregate, your class answers all test questions, but for each student there is a large chunk of missing data. PISA generates the missing data by using a psychometric model called the Rasch model. 

The situation may be described like this. Let's suppose we are testing mathematics. Everybody named Mary gets a question about area, but nobody named Peter does. Nevertheless, everyone named Peter still gets a grade for the area question, and the grade is determined by how the Marys answered the question.

In some circumstances, the Rasch model works, but Svend Kreiner, a Danish biomedical statistician, says that PISA uses it incorrectly. The use of the Rasch model has also been criticized by David Spiegelhalter, professor of the public understanding of risk at Cambridge University.

Some people say that Kreiner is wrong and that it is acceptable to use the Rasch model. I am not a statistician, and  it is not clear to me who is correct. (However, I find it relevant that Kreiner was one of Rasch’s students and that he has used the Rasch model for 40 years.)

How should we interpret the PISA league tables?

Let’s dismiss any misgivings about the reliability of the results and take the PISA league tables as valid. But keep in mind that some data is generated statistically, and the students taking the exams are only a sample of the 15-year old cohort. Consequently, there is some uncertainty in the published figures. 

The league tables show a plausible single score for each country, but the uncertainty means that there is actually an interval range of plausible scores. If the intervals for two countries overlap then you cannot really conclude that the scores are different. 

For example, in the 2012 PISA mathematics table, Finland finished 12th with an average score of 519. Canada placed 13th with an average score of 518. The intervals for Finland and Canada overlapped considerably. Do the results really allow us to say with any degree of certainty that Finland finished ahead of Canada? 

The uncertainty allows some bending of the league tables to suit your argument.

If you want to argue that Canada is slipping badly in math education, you use the league tables based on the single scores: In 2006 Canada was sixth from the top, in 2009 Canada was tenth from the top, and in 2012 Canada was thirteenth from the top. From sixth to thirteenth is a significant drop.

If you want to argue that we are actually not dropping very much, you use the statistical intervals. Based on these, in 2006 Canada was tied for fifth, and in both 2009 and 2012 Canada was tied for tenth. So, no drop from 2009 to 2012.

This still looks like a drop in performance since 2006. But wait: in 2006 neither Shanghai nor Singapore participated in PISA. So you can argue that if we want to compare the 2009 and 2012 results to 2006, we should exclude those “countries”. When we exclude them, we have Canada finishing fifth in 2006 and eighth in 2009 and 2012. And now the drop somehow doesn’t seem as drastic.

- - -

I don't think I've answered the two questions. 

Nevertheless, as OECD intended, PISA provides people with data that they can use to press their governments about their education system. It’s unfortunate that league tables may be flawed and that they are presented in a way that allows the data to be bent to fit one’s agenda. 

Thanks for reading.

- - -

Sources

The arguments against the use of the Rasch model:

An academic paper by Svend Kreiner .

A paper by Svend Kreiner and Karl Bang Christensen .

(To be honest, I didn’t read the above papers. They are highly technical and beyond my expertise.)

Two articles by David Speigelhalter here and here (these are readable).

Rebuttals of the above:

Ray Adams (member of PISA) has a longish article defending PISA’s methodology.

Jan-Eric Gustafsson (university of Oslo) discounts Kreiner and Christensen’s critique. (Gustafsson, however, has other reasons to doubt the valdity of the league tables.)

The PISA data:

The OECD-PISA league tables can be found at the OECD PISA site. CMEC also has them here:

PISA 2006,  PISA 2009, PISA 2012

The original arguments claiming that PISA is flawed were published some time ago. Here are some more recent ones: 

Matthew Smith in Education Week in review (April 2014).

William Stewart, a long time education reporter for TES. (Last Updated: 27 September, 2014).

Benjamin Reilly, Founder of Deans for Impact (Feb 2014).

Catherine Wolff, an education writer in New Zealand (Dec 2013).

Rate your prof(ession)

Imagine that you are the newly appointed minister of education. You are deciding who should have input into the K-9 math curriculum.  There are many groups that want to have a say. Here’s your chance to rate three of them according to how you value their opinions on a scale from 1 (barely worthwhile) to 5 (absolutely necessary). 

Fill in the appropriate  circle.

From all that I have read, one thing is evident: people from the different groups think that their own views are paramount, and they have little regard for each other. It makes it difficult to have a conversation. 

Are you a teacher? Do you believe that math profs and ed profs have not spent enough time in the classroom and do not come close to grasping the complexity of the K-9 environment?

Are you an ed prof? Do you believe that math profs and teachers are largely ignorant about how people learn mathematics?

Are you a math prof? Do you believe that teachers do not know enough mathematics? Do you think that much of what ed profs say is faddish and based on dodgy research?

If you belong to one of the three groups, do you believe that it is possible to even discuss math education with the other groups, or are the differences so great that they cannot be overcome? 

There was an insightful statement by Andrew Coyne in the June 4 edition of the Edmonton Journal.

To construct a theory of the unbridgeable difference between two groups, after all, you have to have in mind a stereotype of each, to be compared and contrasted.

Coyne was talking about Jacques Parizeau and the politics of separatism, but I think the statement is apropos regarding math education.