The expert is Rick Stiggins, who is the founder of ETS's Assement Training Institute in Portland OR. The sage commentary appears in the current )Oct 17) edition of Education Week, the national weekly indispensable for news on Education (and even if you are not a subscriber you can register for free to read two articles a week). The article to which I refer is entitled Five Assessment Myths and Their Consequences (this link might require you to register before you can use it). Below I will offer all five of the myths, a bit of what Stiggins has to say about each, and as per my custom, offer a few comments of my own.
Stiggins begins his commentary with the following introduction:
America has spent 60 years building layer upon layer of district, state, national, and international assessments at immense cost—and with little evidence that our assessment practices have improved learning. True, testing data have revealed achievement problems. But revealing problems and helping fix them are two entirely different things.He then tells the reader that assessments impact on school improvement has been seriously impacted by erroneous myths, five of which he wants to share with us.
As a member of the measurement community, I find this legacy very discouraging. It causes me to reflect deeply on my role and function. Are we helping students and teachers with our assessment practices, or contributing to their problems?
Myth 1: The path to school improvement is paved with standardized tests. Stiggins traces the historic growth of standardized testing, then notes how it fails to meet most of the needs of those who contribute most to the effectiveness of schools, teachers and students. He concludes his examination of this myth as follows:
We have almost completely neglected classroom assessment in our obsession with standardized testing. Had we not, our path to school improvement would have been far more productive.
Myth 2: School and community leaders know how to use assessment to improve schools. Stiggins points out that most people have no idea of how to use the information from standardized tests, and that they way they are being used - rewarding and punishing - is demoralizing to the classroom teachers and schools who receive the latter. I note that I have described this approach, which is an essential part of NCLB, as "the beatings will continue until the morale improves."
Myth 3: Teachers are trained to assess productively. Stiggins begins by cutting to the heart of this myth:
Teachers can spend a quarter or more of their professional time involved in assessment-related activities. If they assess accurately and use results effectively, their students can prosper. Administrators, too, use assessment to make crucial curriculum and resource-allocation decisions that can improve school quality.Of course the problem is most classroom teachers are NOT properly trained either in creating nor applying assessments. Sadly, few administrative personnel other than those formally trained in assessment have sufficient training.
Myth 4: Adult decisions drive school effectiveness. Stiggins points out this totally ignores the role of students and the decisions they make, which can be strongly influenced by whether or not they believe they can succeed, because if they don't, they give up. As he rightly notes:
The most valid and reliable “high stakes” test, if it causes students to give up in hopelessness, cannot be regarded as productive. It does more harm than good.
Myth 5: Grades and test scores maximize student motivation and learning. The heart of this myth can be seen in a short snippet of what Stiggins offers here:
Schools operated on the belief that if I fail you or threaten to do so, it will cause you to try harder. This was only true for those who felt in control of the success contingencies. For the others, chronic failure resulted, and the intimidation minimized their learning. True hopelessness always trumps pressure to learn.Stiggins points out that if our goal is that all children succeed (aka "No Child Left Behind") we must provide a model where success is frequent and failure infrequent. This provides a framework of optimism, in which the student believes s/he can succeed. I will return to this point anon.
Stiggins argues that we have learned that we need to asses for learning rather than do what we have been doing, which is assessment of learning. He points out there is a well-established research base to prefer the former over the latter.
He also argues that our approach to assessment now, which is isolated, even if it includes benchmarks and interims as well as annual assessments, needs to be replaced with a system that includes ongoing in-class assessment. For this people at all levels of the educational system need to be properly trained. Stiggins also points out that policy makers - often themselves NOT educators - also need appropriate training in and understanding of assessment. I will return to these points as well.
Before I offer a few more comments of my own, I want to offer the close from Stiggins, because I think it is crucial in understanding the nature of the issue of proper use of assessment:
Of greatest importance, however, is that we acknowledge the key role of the learner in the assessment-learning connection. We must begin to use classroom assessment to help all students experience continuous success and come to believe in themselves as learners.
Let me begin with that last quote. I am a firm believer that one important - and often ignored - purpose of our schools should be to empower all of our students to be life-long learners. That will not happen unless they experience success in the controlled learning environment of school. How we use assessment can help or hinder that learning process. As a classroom teacher, my experience is that if students can learn from their mistakes and self-correct, if they do not see assessment as something punitive, but rather as supportive, they have less fear of being assessed. They become more confident, and as a result perform better, even on assessments that are quite difficult. Our current approach to testing and to test preparation seems totally contrary to my experience.
Far too many involved with education do not properly understand assessment. It is a minor part of teacher preparation programs, and is not always part of administration and leadership programs. Increasingly teachers are pressured to use pre-prepared testing material provided by textbook vendors. That does not mean that one is necessarily using canned tests, although that happens far too often. Instead there are item pools, groups of questions from which one can select. In theory these items have been vetted to ensure that they are not biased or poorly constructed, but the reality is far different.
Often in courses with high stakes end of course tests one is inclined to construct tests from released items from former tests. Of course, that presumes that such items were themselves properly constructed, which from my experience in Maryland I will assure is NOT always the case. I will let my students go through such items to familiarize themselves with test construction, and the kinds of items they will encounter. I generally will not use such items for an actual graded test, but that is my idiosyncratic approach: I want my students able to provide their own answer to a question, not merely pick which among four or five choices stinks the least.
If I were to focus on only three points from Stiggins, it would be these:
(1) we need far better understanding of the proper uses of assessment at all levels of the educational process, from the highest level of policy maker to the individual classroom teaching
(2) Most assessment will continue to be done at the classroom level: teachers and students need the more immediate feedback classroom level assessment provides. It seems to me that our assessment for accountability purposes should not be in isolation from or in ontradiction to in class and in school assessment, but instead should use these as basic building blocks. This requires rethinking how we do assessment, but we are not on totally new ground. Here I strongly recommend those interested in this subject explore what Nebraska has been doing. One can look at the website for STARS, which is the state's "School-based Teacher-led Assessment Reporting System." I also suggest looking at the webpage of Doug Christensen, who is Nebraska's Commissioner of Education (and who served as a panelist at Yearlykos 2007).
(3) If our goal is truly to leave no child behind, perhaps rather than insisting that the child be squeezed into a standardized model of assessment that is not fair to all children we think about ensuring that we provide assessments upon which the child can build, that is, experience success, have some understanding of her own learning, and thus gain some control over his own success.
I do not believe we will totally do away with mass-produced tests, largely prepared by for-profit entities (and even the ostensibly no-profit ETS acts more and more like a for profit entity). Such tests are one form of assessment. They should not be the only form, nor should they be the primary method of how we assess learning. Assessment should be much more geared towards being formative, towards helping the learner and the instructor - helping both in enabling the learner to improve her learning. Unfortunately our approach has been increasingly summative, measuring at the end when it is too late for that measurement to provide meaning for the student, or information to improve instruction for the cohort of students being tested. And then we use the performance on such summative measure to punish and praise, to rank and compare. That seems inimical to the idea of leaving no child behind, if the result is to say too bad, you don't graduate or your school is no good. ha ha. Too bad you wasted a year or more.
I am not an assessment expert. Stiggins is, and those who would like to know more about his ideas can go to the website for the Assessment Training Institute, or perhaps peruse the links at this google search.
Maryland schools were closed today. I was catching up on things, including dental appointments and back reading. I did not get to the Stiggins commentary until mid-afternoon. I hope this diary gets some visibility, because I think the ideas are important, but I am not going to obsess over it.
Do with it what you will.
Peace.
