How should reading literacy be measured? PISA thinks globally …

especially Most of us think that reading is looking at written words and de-coding them ─ and understanding what the words are saying. And, to test students’ skills at reading, that’s what you focus on ─ how well did students de-code and how well did they comprehend what they read.

But there’s so much more to reading, especially after students master the basics. What comes next then?

Think globally and check out PISA’s definition of reading literacy: ‘the capacity to understand, use, reflect on and engage with written texts, in order to achieve goals and develop knowledge and potential ─ and participate in society.’[1]

The PISA view is ─ you can’t really participate in society without reading literacy ….

Who is PISA and why should we care about this definition of reading literacy?

PISA stands for The Programme for International Student Assessment.[2] PISA is an international test conducted by the Organisation for Economic Co-operation and Development (OECD) of 15-year-old school students’ performance on math, science, and reading. The purpose of these assessments is to try to improve education by delivering information to policymakers and educators about how students are doing.

Every three years since the year 2000, 15-year-old students from randomly selected schools throughout the world take PISA assessments in the core subjects of reading, math, and science ─ and beginning in 2012, some nations also elected to participate in an optional assessment for problem solving and financial literacy. In each year of the assessment, PISA especially focuses on one of the core subjects.

What are the PISA assessments like?

The two-hour long PISA assessments are a mixture of open-ended and multiple-choice questions that present reading passages in real-life situations. A total of about 390 minutes of test items are covered. The students and their school principals also answer questionnaires to provide information about the students’ backgrounds, their schools and learning experiences ─ and about the broader school system. This background information is important for understanding how to interpret information collected from schools in many different systems and nations.

Why should we care how 15-year-olds in the U.S. stack up to 15-year olds in other nations?

Consider how reading literacy is assessed and then you decide …

In PISA, reading literacy is assessed in three ways: (1) the format of the text, (2) the processes of reading, and (3) situations for reading.

The format of the text: In typical tests of reading in the U.S, students are asked to read text presented in a series of sentences and paragraphs; and they’re tested on ability to read and comprehend from that format. But PISA tests additional types of reading formats such as information contained in lists, graphs, and diagrams; and reading that occurs in varied formats ─ story-telling, application forms, and advertisements. The thinking behind this is that individuals will encounter all types of written materials in their life and we should know how prepared they are to read these varied materials. In my early career, I worked as a K-12 resource teacher in a big public school district and recall so clearly the Superintendent of Schools remarking that a good teacher could teach reading from “the Yellow Pages.” Back then, that was the book that contained alphabetical lists of businesses and non-profits in the city; and it did contain the names of businesses and their addresses, phone numbers, ads, pictures, and charts about who they served. His words ring ever more true about our need to be sure our students can indeed interpret the world around them – through reading all kinds of materials, though the Yellow Pages have by and large given way to the world of websites now for this information.

The processes of reading: The PISA test does not assess 15-year olds on basic reading skills because it assumes that most students by this age will have these skills. What PSA does focus on is how well students can access and retrieve information; and whether they can form a general understanding of the text─ and interpret it and reflect on its contents and its form and features.

Situations: The PISA test wants to know how well a student reads for four different “situational” uses ─ reading for private use (personal); reading for public use; reading for work (occupational); and reading for education. For example, reading a novel, personal letter or biography is for private use; reading official documents or announcements are for public use; reading a manual or report is for occupational use; and reading a textbook or worksheet is for educational use. Some groups may perform better in one reading situation than another, so the assessment includes a range of reading situations among the test items.

Who among us could argue persuasively that our students need only to read well in one or two of these four situations?

In the most recent PISA assessments, some 70 nations participated. The PISA results are tabulated by country ─ and recent testing cycles have separate provincial or regional results for some countries. The public attention, as you would expect, focuses on rankings: the average scores of countries and how they stack up next to one another. PISA never combines math, science and reading scores into an overall score to create a “best of show.” However, some folks have combined test results from all three areas (math, science, reading) to create such a ranking system. OECD does not endorse this type of summarizing of the outcomes of these assessments, however.

How does the U.S. stack up in the most recent reading assessment for our 15 year-olds?

PISA 2012 testing results were announced on December 3, 2013, based on tests of some half a million (510,000) participating students in all 34 OECD member countries plus 31 partner countries. Here are the rankings by country for reading. You have to scan pretty far down the list to find the U.S. at 24^th on the list among 65 nations!

Reading scores PISA 2012 (reported 2013)

Rank            Country                         Score

The press reports on PISA going back to its origins in 2000, and more recently, have stirred up considerable debate about how our school systems should be changed in light of what many folks think has been a lukewarm performance by U.S. 15-year olds.

Regardless of what our ranking is among nations, we should be asking ourselves: do we want our children to achieve a use-able level of reading literacy in the four situations we know reading is vital for ─ to read for private use; to read documents for public uses; to read for work/occupational uses; and to read for education uses?

I’m a fan of thinking globally when it comes to assessing the real-life reading literacy skills of our students. I can hear my former Superintendent of Schools calling for us to teach reading from the Yellow Pages ─ he understood decades ago that reading was indeed a skill our students would need to apply in multiple situations.

———————

[1] OECD, 2009: http://www.acer.edu.au/ozpisa/reading-literacy

[2] The U.S. rendition of their name is, The Program for International Student Assessment.