Measuring Educational Impacts

BBC Radio 4 – The Educators – John Hattie
(Click on the link above to listen to the discussion)

Here’s a very interesting audio recording of an interview done for the BBC Radio 4 series – ‘The Educators’. The interviewee was Dr John Hattie of Melbourne University, Australia. He’s particularly talking about the large meta-analysis he’d carried out in to educational impacts of various things. In other words – research that pulls together the results of a large body of research studies from across the world.

Whilst it’s interesting, especially from an academic perspective, to hear him talk about the relative impacts of different things, such as homework, class sizes, streaming or parental choice of schools the burning question that came in to my mind was – “what are we measuring to define IMPACT/ EFFECT and are they the things we should be measuring?”

It is inevitable that during the discussion the PISA tests came up for discussion. To many educators, and especially those who want to believe that data and statistics are all that’s needed to enable them to drive change/ improvement in the whole education system it is a very simple piece of logic to say that PISA tests what children need to be learning in school and that, therefore, comparisons of relative performance on PISA are the right way to assess the quality of an education system.

How comfortably does this sit with ideas related to the need, in the Twenty First Century to develop lifelong learners, to develop high levels of EQ, empathy, communication skills and other softer attributes when its very clear that these aren’t figuring at all in the analysis of PISA or any other effects/ impact/ outcomes?

A school, District or even an individual teacher can decide to do certain things that will have a higher chance of producing higher PISA scores or scores on other standardised testing systems. However, does that mean that we must say that that automatically represents good education, the ideal? Well, arguably, when Shanghai scored top on PISA it was acknowledged by education authorities in China that their methods were very good at drilling the children in what they needed to know to do well in the exams, BUT were probably very bad at inculcating and developing the skills those young people would need to be effective in the 21st century. As a result, the Chinese have been looking outside their own education systems for ways to change so as to have an education system that prepares young people for the reality of the future they will face.

Here’s the problem – the future of children and their contribution to society doesn’t lend itself to being tested, picked apart for analytical debate half as easily as standardised test results. And this is why we can so easily fall in to the trap of being so comfortable with the kinds of debates here in this interview. All through, the discussion is about the relative merits of different actions that can take place in education, all based on a criteria of judgement about worthwhile outcomes that may not stand up to scrutiny.

Being able to climb a ladder better or faster is meaningless if we’re leaning the ladder against the wrong wall.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: