A revised PIRLS 2011 to 2016 trend for South Africa and the importance of analysing the underlying microdata

Stellenbosch Working Paper Series No. WP02/2020
 
Publication date: February 2020
 
Author(s):
[protected email address] (ReSEP, Stellenbosch University and Department of Basic Education)
 
Abstract:

Given South Africa’s weak performance in international testing programmes, there is a strong interest in gauging improvements within these programmes. The finding that South Africa saw no progress between 2011 and 2016 in the Progress in International Reading Literacy Study (PIRLS) programme, which tests Grade 4 reading competencies, was inconsistent with considerable progress seen in a couple of other testing programmes. Moreover, an earlier PIRLS average score for Grade 4 from 2006 suggested that the 2011 mean score used to determine the flat 2011 to 2016 flat trend was problematic. The current paper uses the underlying microdata for PIRLS 2011 and 2016, which are publicly available, to examine the trend. It is clear that the 2011 mean score used by the international PIRLS analysts to arrive at the flat trend cannot be correct. It should be considerably lower. It should be noted that the 2011 mean for South Africa involved an unusual process. South Africa was the only country for which an original mean on an easier scale, prePIRLS, had to be recalibrated to the main PIRLS scale. This was because South Africa was the only country participating in the easier prePIRLS in 2011 and in some form of PIRLS in 2016. There was clearly something wrong with the 2011 recalibration. In correspondence, the International Association for the Evaluation of Educational Achievement (IEA), the body that conducts PIRLS globally, acknowledged, in part on the basis of a preliminary version of the current paper, that the originally published 2011 to 2016 South Africa trend should now not be considered reliable. They also confirmed that the classical score gains for South Africa reflected in the paper are correct. The method used in the paper is essentially to examine classical score gains between 2011 and 2016 with respect to common items, and then to recalibrate that to the main PIRLS scale. As an additional verification, the paper checks that gains remain after one controls for socio-economic status. The paper concludes that South Africa in fact saw a large gain between 2011 and 2016, equal to around 0.05 standard deviations a year. This is a fast rate of improvement by international standards. Of the 43 countries with 2011 to 2016 trends in PIRLS, South Africa displayed the third-steepest improvement, after Morocco and Oman.

 
JEL Classification:

C13, C89, I21

Keywords:

South Africa, PIRLS, international testing systems, reading

Download: PDF (390 KB)

Login

(for staff & registered students)



Need a password?
Forgot your password?

Upcoming Seminars

No seminars are currently listed. Please check back soon.
 
More...

BER Weekly

26 January 2024
Domestically, the theme of the week centred around monetary policy and inflation, with the SA Reserve Bank (SARB) making its first repo rate decision of the year on Thursday. Furthermore, Stats SA released both consumer and producer price inflation data for December. Globally, monetary policy was also important, with the European Central Bank (ECB),...

Read the full issue
 

Upcoming Seminars

No seminars are currently listed. Please check back soon.
 
More...

BER Weekly

26 January 2024
Domestically, the theme of the week centred around monetary policy and inflation, with the SA Reserve Bank (SARB) making its first repo rate decision of the year on Thursday. Furthermore, Stats SA released both consumer and producer price inflation data for December. Globally, monetary policy was also important, with the European Central Bank (ECB),...

Read the full issue