How To Mess Up A Good Thing

Formative assessment has been a vitally important element in education, as educators seek to have ways to move away from the old fashioned models based endlessly on the use of standardised examinations to provide data and evidence of student learning. Incidentally, on that, I read yesterday that in Japan, children don’t take any kind of full summative assessment until the age of 10.

Formative assessment goes by many names, the most common other name being assessment for learning (AfL) to differentiate it from assessment of learning (AoL) AoL can be compared to cracking eggs open on a regular basis to find out how the chicks are developing. When we consider that the man who invented standardised tests (especially of the multiple choice variety) actually came out after a year or two to say that they were too crude and unsophisticated to use to measure school pupils’ knowledge, skills or academic performance (and was promptly sacked for his honesty!) we have to say that if we are to have assessment it has to be something a great deal more advanced.

AfL sets out to pay more attention to the future, rather than looking solely backwards like summative testing. It includes a variety of techniques and methods to gather clues as to how well a student is progressing in some learning and to have clarity about where they need to go next to best build on to their existing knowledge, skills and competence. Also, and perhaps most critically, AfL is not just predicated on the need to produce a set of data for the teacher, but aims to have the learner themselves reflect on their learning, the journey they have taken up to the point in time and where they are going next – including what they will need to do to get there. It

In November 2008 I was very fortunate to attend a presentation in Mumbai, India arranged by Cambridge International Examinations (CIE). The keynote speaker was professor Sue Swaffield of Cambridge University, a leading proponent of AfL and researcher in educational leadership, assessment and school improvement. She presented powerful research evidence about the ways in which grades and marks demotivate all but the strongest of academic achievers, lead to negative approaches to learning from children and teachers and advocating strongly for stronger training of teachers in the techniques and methods of AfL.

So, I was very interested recently to come across this article written by professor Swaffield nearly a year later that really highlights how good ideas can get mangled and abused in the education domain. The needs of those in power for data, control and top down dictating of how things are done is the very opposite of what we see throughout the world in the most dynamic, creative learning organisations.

The Misrepresentation of Assessment For Learning
(To download the document as a PDF you will need to have Adobe reader or some similar programme loaded)

I particularly loved the section on the second page of the paper that talks of the origins of the word ‘assessment’ in latin, deriving from a word that means “sitting beside”. Tell that to the exam invigilators – they would call that cheating!

Confessions of a Grade Fetishist

An interesting TedX video from a teacher about the benefits she and her pupils experienced when she broke her love affair with student grades as a measure of everything important.

"The score is what's remembered which essentially means nothing out of context."

One of the points she makes is something that has always stood out to me - there can be a whole bunch of children with the same grade, but who have very different learner profiles, needs and requirements as future learners. The truth is they tell us so little as to be useless.

She is at her most animated when she talks about what the process has done for her students.

This is well-worth watching to develop the debate about what's possible in learning.

The Future of Assessment

Insights in to to the future of assessment, from the perspective of Pearson Education.

Chasing Bits of Paper

Teaching Times – English GCSEs Could Be Harder For Pupils

In a logical and rational world children, parents and educators would want that examinations are a fair test, skillfully put together in such a way that all students get to demonstrate their capabilities and what they’ve learned to the greatest extent. Those who have embraced the learning best would be able to demonstrate that.

Instead, what we’ve increasingly finished up with is dumbed down simplified tests that enable vast numbers to score the top marks, to the point where they no longer act as a good and effective test of who are the best students. One of the results is that Universities increasingly struggle to find any value in them to determine who would be the best students to permit to pursue further studies. Sadly, all parties conspire, consciously or unconsciously to dumb the system down and render it worthless.

If we could hav an education system in which everyone really embraced a love for learning, what would the assessments look like? How would we reward and recognise those teachers who embrace the work of enabling every pupil to learn to their best and how would we hold those who cynically tech to the tests to account?

Leadership For Teacher Learning

The ‘growth mindset’ of Stanford professor Carol Dweck is as important for teachers as it is for pupils, if we are to have schools that achieve effective learning for all students.

Dr Dylan Wiliams is a British educator who has become a renowned authority on strategic formative assessment. I was very glad to come across this webinar he did this week for Learning Sciences International. It's based on a new book he's just published.

He shares some fascinating research. What i particularly liked was that he challenges and questions some of the assumptions that have been drawn from past research and offers some alternative explanations. As he highlights, this is particularly important in terms of how leaders in education interpret research, draw conclusions and decide what to do in their schools as a result.

Unfortunately, I found that there were too many occasions when the visuals of the webinar weren't matching with what was being said. Fortunately, I was separately able to find a copy of the presentation slides, so that it's possible to see them independently from the audio. They can be found here:

Dylan Williams Center - leadership For Teacher Learning Slides

There are many points he raises. One I found interesting was the suggestion that, so far, there really are no effective ways to be able to tell who are the great teachers and who are not. This is a big issue when we know that the quality of teachers has a big impact for children. I liked his idea that if a teacher doesn't talk in terms of believing that they can gt better then this should be seen as a warning sign. Teachers who are committed to their own continuous improvement offer the best chance for raising educational standards. Teachers who believe they are as good as they can get (fixed mindset) are a risk in the classroom.

I was especially interested in his findings about formative assessment and also the research on comparing the costs of particular initiatives, the quality of evidence in favour of the intervention and the size of benefit it achieved.

The five processes of formative assessment was really valuable;

a) Clarifying, sharing and understanding learning intentions,
b) Eliciting evidence of prior learning,
c) Providing (motivating) feedback that moves learners forward,
d) Activating students as learning resources for each other (peer learning),
e) Activating students as owners of their own learning

I think that Wiliam's conclusions are spot on when he suggests that if teachers are to be growing and improving practice continually, there is an onus on both the teachers themselves and their leaders to create the climate and environment for this to happen. He also suggests some useful signs that progress is being made.

I would urge all teachers and education leaders who are regular readers of this blog to invest the time to watch the webinar. I think you'll find it time well spent.

Testing – For What?

“If we are insistent on having exams for these Primary School children, then what are we testing for?”

“Well, of course, we want to know what they have learned and what they can do.”

“In that case, is there any legitimate good reason why there should be time limits on the exam? Doesn’t that test something else?”

The room full of teachers looked at me with such horror that I felt I might need to do a quick check to see that I hadn’t just grown a second head.

I’ve long believed that there are times when, as leaders, one of our most valuable responsibilities is to ask naive and simple questions that challenge and question those things that are taken for granted within a profession. The reality is education and teaching have many of these things – practices and processes that are applied without question or application of curiosity to see whether they really make sense. This is a particular issue in schools that espouse a desire to move to more child-centric and learner oriented education methods (especially personalisation) but still do a multitude of things that aren’t really compatible with those goals – simply because they haven’t been questioned.

The conversation at the top of this post really happened. It was about 11 years ago, but I’ve also had similar conversations with teachers in other places much more recently. So, I was interested when i saw this article from New York Times and wanted to share it;

New York Times – State Will Shed Clock For Some Statewide Tests

Seeing the headline and the initial part of the article one might have reason to believe that this was all positive and a recognition and response to do something that makes sense for positive progress in education. However, reflection on the final part of the article suggests that the intent is more manipulative and potentially a devious move by those hell-bent on pushing forward the agenda of the standards movement in US public education. Could it be that this is a sop to appease increasingly frustrated and angry teachers and parents? These are people (politicians particularly) who are convinced that the way to raise standards in education and have a higher level of quality is to change the nature of teachers’ jobs by linking their remuneration and even job security to performance in standardised tests.

We only need to stop for a second to see that what they’re doing doesn’t make sense. If the question in their minds was about how to have a system of assessing children’s learning that was beneficial, meaningful and led to genuine progress n leaning for every child, then why continue with the standardised formal exams? Why not advocate for a form of more formative assessments and application of a fuller range of assessment tools and strategies? Secondly, if the time limit on the exams is removed, how does this create a level playing field for all teachers such that the data nerated can be trusted as a basis for judging their performance, determining salary increases and even issues of job security and tenure?

I continue to believe that if we are truly putting the children/ learners first – then there’s no place for examinations in the earlier years of their education at all. There are so many more effective ways of assessing progress that provide meaningful ways to plan their way forward for continuous learning progress. Exams are not some holy sanctified process for which years of learning and practice are necessary – we want to create great citizens and young people who can make a meaningful contribution to the world, not exam ninjas!

In the meantime, I will consider it my duty to continue to ask naive questions, challenge and probe so that together educators can bring positive reform in our profession.

Noam Chomsky – Dangers of Standardized Testing

“Not everything important is measurable and not everything measurable is important.”
Elliot Eisner, Educator

In this short video (under 7 1/2 minutes), Noam Chomsky very neatly summarises what's wrong with today's obsession with standardized testing and the harm that it's doing in school systems, individual schools and at the level of the individual teacher and student.

For those who might want the interview in written form, the link is here:
Creative Systems Thinking - Noam Chomsky

Experience tells us, throughout the world, that when these standardized systems of assessment are challenged the people who defend them are the children who performed highest (or expected to) and their parents under the rigid system. They happen to be the children who have figured out how to 'play the game' of giving testers what they want and are happy for their A grades to just keep rolling in without any real effort. However, I would argue that even they are failed by the standardized system as they don't get stretched to fulfill their potential or challenged to go ahead of where they are. More formative assessment processes focus more on momentum and progress forwards for each and every student.

Probably the biggest hurdle to getting real, across the board, effective change is that the testing industry is now vast and highly profitable. I've seen suggestions that in the US alone the revenues from testing are around one and half times the revenues earned by cinema box offices. This is a powerful force with strong political connections that is determined to spread their approach throughout the world. A further challenge comes in those places where the teachers see their lives as easier where there is standardized testing and therefore don't speak up. Administering some multiple choice based exams developed by an external party takes far less effort than engaging mentally and continuously with the formative process of figuring out for each and every student in the class what they need, where they need to go and how they need to learn to progress from where they are.

We know there's something wrong when the teachers and learning have become subjugated to the testing, instead of the other way around. Never mind that the tests fail to give feedback of any real value or merit, especially when it comes to the development of twenty first century competencies and skills. There is very important work to be done in this area by educators throughout the world.

Getting in a Mess Over Testing

In the last week or so, the debate about testing in the US took a significant new direction with a decree from the President, Barack Obama with the headline – “We’ve been doing too much testing.” Here’s a New York Times story with more background on the issues:

New York Times – Calls for Limits on Testing in Schools

There is real irony in this story and the way that it’s reported. Politicians dictating to educators about how children will be educated. For example, no clues are given as to how they’ve arrived at this figure of 2% of instructional time to be given to testing. As the article quite rightly suggests, with testing not defined, who decides what forms part of the 2%? Also, if the testing is rubbish it doesn’t matter whether it occupies 2% of a student’s learning time in school – it’s still going to be wasted time.

Further, we have to be realistic. part of the problem with high stakes testing isn’t the actual time the tests take to conduct. it’s all the other associated lost time that worries me. If, as is often the case, teachers are going to find their performance assessed according to the performance of the students in the exams, we can hardly be surprised that the teachers turn over large amounts of learning time to ‘test preparation’. Before you know it, even with one annual cycle of exams, way more time is lost in revision lessons, exam priming sessions, practicing for the processes of answering the exam questions etc. Before you know it, learning to excel in the exam has become far more important than learning.

Then, in most Indian schools, my experience suggests that combined pressure from the parents, the students and the teachers will see one exam take place per day, with the remainder of the day written off for the child to go home and ‘mug up’ for the next. Then, there are often some days declared as ‘non-instructional’ while the teachers do the marking of the exam papers. Then, we lose some more learning focus and time whilst everyone fills themselves with angst about the results afterwards.

The net effect in an academic year that typically amounts to about 190 school days is that, easily over 10% of learning time is lost to this process. I have always felt that this was truly bizarre if the purpose is really to check progress and point the way for future learning.

So, while the US tackles the mess it’s got itself into with a new limit on testing time, the Indian educators need to take a long cold hard look at the entrenched habits of examinations.

School must be for learning, not testing.

When We Stop Rating People

I have often found something odd about the frequency of hearing teachers especially talk against performance management systems that lead to rating of their work. The arguments may vary somewhat, but usually consist of claims that ratings de-humanise them, that they attempt to objectify their work which is inherently subjective and that it goes against the wishes expressed by management to have people work collaboratively and as teams.

When educators spend so much of their time, traditionally, putting marks and/ or grades on students’ work – both the outcomes and the effort, I find the first tow arguments at times a bit hypocritical. However, I do see a fair amount of justification in the third argument. On the first two, I’m not necessarily saying i disagree with the teachers, but that if they want people to stop putting numbers/ grades and ratings on them I’d love to see the same enthusiasm for finding alternative ways to provide feedback on students’ work! What’s good for the teachers is also good for the students!

When i was in Delhi, we experimented with one term in the year when parents of pupils up to class 8 would receive a ‘Comments only’ report for their child, instead of the traditional ones where parents inevitably focus upon the grades and marks (and particularly those they want to see higher!). One father summed up the feedback of many parents when he told me that this report completely changed the nature of the dialogue he had with his child. He had initially been sceptical. However, what he found was that he and his daughter had a far more open discussion around the teacher feedback, uncluttered by the grades and marks. They talked about strengths as well as areas for development and much more about what help, if any, the child might need to make the targeted improvements.

I was reminded of that when I read this recent article from Harvard business Review about large companies in the US who are shifting away from forced ranking, grades etc. to measure performance of people in the workplace;

Harvard Business Review – Ditching Performance Ratings

The reality is whether we’re talking of students, teachers or employees in any kind of organisation any attempt to distil the essence of who they are and the contribution they make to numbers, grades or other quantifiables will feel like a blunt weapon and a poor way to motivate, inspire or guide to higher levels of achievement and performance. We can, even must, strive to more positive means of motivating and aiding people of all ages to fulfil their potential and to manifest their best self.

Do We All Mean the Same Thing When We Refer To ‘Personalization’?

The simple answer to that question is – no we don’t. For pure profit motives this term which means a perfectly laudable aim and intention in progressive education has been latched on to, corrupted and made to mean something else when used by the EdTech companies when they come peddling their latest snake oils, charms and amulets.

This article sets out well the confusion that has been caused by the mangling distortions of the word. Plainly, to me, the definition used by progressive teachers that emphasises creativity and freedom of learning paths is the most appropriate use of the word.

Mindshift – Big Ideas Article

One interesting aspect of the article is the reference to current educational testing goals being incompatible with personalisation. I’m really not sure this is necessarily the case. I see personalisation as being very closely entwined with differentiation (not mentioned in the article). As one of the comments below the article points out, school systems will and probably should all have common end goals for every pupil. For example, every student should acquire the skills to carry out algebraic equations to a certain level of competency. However, differentiation and personalisation offer the idea that whilst the eventual end goals may be the same for different pupils, modern educational methods (including those that harness the benefits of IT) enable different students to take different paths to the same destination.

Even differentiation gets subjected to a lot of abusive corruption where it often appears to be a simplistic process of setting students in to different ability groups and then adopting different paths with each group that almost always pre-suppose different levels of eventual outcome (high, medium and low end goal expectations). Instead, I see differentiation and persoanalisation as harnessing all the tools available to educators (include ICT) to enable different students to take different paths, different sequences of units and activities, different pacing and methods, but with THE SAME level of end expectations and goals.

In this way, personalisation isn’t incompatible with common end tests and exams.