A revised PIRLS 2011 to 2016 trend for South Africa and the importance of analysing the underlying microdata
Stellenbosch Working Paper Series No. WP02/2020Publication date: February 2020
Author(s):
Given South Africa’s weak performance in international testing programmes, there is a strong interest in gauging improvements within these programmes. The finding that South Africa saw no progress between 2011 and 2016 in the Progress in International Reading Literacy Study (PIRLS) programme, which tests Grade 4 reading competencies, was inconsistent with considerable progress seen in a couple of other testing programmes. Moreover, an earlier PIRLS average score for Grade 4 from 2006 suggested that the 2011 mean score used to determine the flat 2011 to 2016 flat trend was problematic. The current paper uses the underlying microdata for PIRLS 2011 and 2016, which are publicly available, to examine the trend. It is clear that the 2011 mean score used by the international PIRLS analysts to arrive at the flat trend cannot be correct. It should be considerably lower. It should be noted that the 2011 mean for South Africa involved an unusual process. South Africa was the only country for which an original mean on an easier scale, prePIRLS, had to be recalibrated to the main PIRLS scale. This was because South Africa was the only country participating in the easier prePIRLS in 2011 and in some form of PIRLS in 2016. There was clearly something wrong with the 2011 recalibration. In correspondence, the International Association for the Evaluation of Educational Achievement (IEA), the body that conducts PIRLS globally, acknowledged, in part on the basis of a preliminary version of the current paper, that the originally published 2011 to 2016 South Africa trend should now not be considered reliable. They also confirmed that the classical score gains for South Africa reflected in the paper are correct. The method used in the paper is essentially to examine classical score gains between 2011 and 2016 with respect to common items, and then to recalibrate that to the main PIRLS scale. As an additional verification, the paper checks that gains remain after one controls for socio-economic status. The paper concludes that South Africa in fact saw a large gain between 2011 and 2016, equal to around 0.05 standard deviations a year. This is a fast rate of improvement by international standards. Of the 43 countries with 2011 to 2016 trends in PIRLS, South Africa displayed the third-steepest improvement, after Morocco and Oman.
JEL Classification:C13, C89, I21
Keywords:South Africa, PIRLS, international testing systems, reading
Download: PDF (390 KB)Login
(for staff & registered students)
Upcoming Seminars
Monday 21 July 202512:00-13:00
Izak Odendaal: Old Mutual Wealth Chief Investment Strategist
Topic: "Diverging fiscal policies and what it means for markets"
12:00-13:00
Dr Neil Rankin: Ceo Of Predictive Insights & Stellenbosch University
Topic: "TBC"
12:00-13:00
Prof Willem Boshoff: Stellenbosch University
Topic: "Two competing approaches in South African competition policy: merger control and anti-cartel enforcement over the past 30 years"
BER Weekly
27 Jun 2025 Another setback for the GNU, but oil markets breathe a little easierThis week was marked by heightened tensions both domestically and internationally. At home, friction intensified between the two largest parties in the Government of National Unity (GNU), the ANC and the DA, following the firing of one of the DA's deputy ministers. Internationally, the US conducted airstrikes on three Iranian nuclear facilities using...
Read the full issue
Upcoming Seminars
Monday 21 July 202512:00-13:00
Izak Odendaal: Old Mutual Wealth Chief Investment Strategist
Topic: "Diverging fiscal policies and what it means for markets"
12:00-13:00
Dr Neil Rankin: Ceo Of Predictive Insights & Stellenbosch University
Topic: "TBC"
12:00-13:00
Prof Willem Boshoff: Stellenbosch University
Topic: "Two competing approaches in South African competition policy: merger control and anti-cartel enforcement over the past 30 years"
BER Weekly
27 Jun 2025 Another setback for the GNU, but oil markets breathe a little easierThis week was marked by heightened tensions both domestically and internationally. At home, friction intensified between the two largest parties in the Government of National Unity (GNU), the ANC and the DA, following the firing of one of the DA's deputy ministers. Internationally, the US conducted airstrikes on three Iranian nuclear facilities using...
Read the full issue