Pop Quiz: How useful are tests, anyway?
As some of you may or may not be aware, most PhD students outside the discipline of Education are never required to take any classes in paedagogy in order to “learn” how to teach at the university level, or indeed at any other. Rather, it’s kind of magically expected that after so many years on the receiving end of classroom instruction in one’s discipline, planning, balancing, and executing a course that covers the same material should somehow just come naturally. Although the most fortunate students are sometimes afforded the experience of a teaching assistantship under a gifted instructor, by and large the first classroom teaching experience is a bit of a trial by fire for the newly-minted teacher.
Universities and their constituent departments have developed a few strategies to try to deal with this glaring weakness in the system. Many institutions, such as my own, offer optional (or even mandatory) teacher training seminars to graduate students and first-time adjuncts. It is also becoming standard in my discipline for search committees to ask job applicants for a sample syllabus or even a video of the applicant teaching a class well before the traditional classroom teaching demo in the final stages of the interview. To compensate for this aspect of the job market, my own programme has incorporated course design into the comprehensive exams we must pass to reach ABD status. Students are expected to design an upper-level undergraduate course related to their specialisation or dissertation topic, and then orally defend the syllabus in front of a panel of faculty members.
Curiously, this process does not involve actually teaching any material to undergraduates.*
The glaring weaknesses of this system of, er, “teacher training” should be pretty evident, but while possible reforms to it are a perennial topic of conversation, paedagogical training generally seems to end up at the very bottom of the priority list when it comes to actually making changes. This is unfortunate, since failing to train students in evidence-based instructional methods essentially perpetuates an educational “model” that consists of little more than a giant appeal to tradition with a healthy helping of confirmation bias.
Let me illustrate with an anecdote.
I’ve never really cared for tests as a paedagogical tool, though I do grudgingly accept their necessity in certain situations where other forms of evaluation are impractical. In fact, I often get the sense that tests are a distraction from actual learning and mastery, since they shift the student’s primary goal away from learning itself and toward passing the test. In my student days I did not find this distinction terribly problematic, but only because I was never particularly concerned with grades: I learned the material that interested me, and rarely if ever took extra steps to prepare for tests (sometimes with detrimental results). As a teacher, though, tests drive me crazy. Because so many students are so terribly concerned with their grades, entire class sessions are lost to exam protocols, reviews, and other minutia that distract from the business of learning.
So I decided, when designing my sample course, to evaluate the students based only on a number of writing assignments and a final paper. That’s right, no tests! Of course, when I brought the syllabus in for oral defence, I was immediately confronted with the received wisdom.
Them: Why don’t you have any tests?
Me: I don’t think they are as useful a gauge of students’ mastery as writing assignments.
Them: But without tests, what motivates them to learn?
Me: Well, they still have to learn the material for the writing assignments. Otherwise they wouldn’t be able to complete them successfully.
Them: I think you really need tests, or students won’t be motivated to learn. I always test my students to make sure they learn certain key concepts, and even then some of them always fail.
Me: Wait, doesn’t that show that the test doesn’t actually motivate everyone?
Them: …
What do you think of tests? In what situations are they useful paedagogical tools? If you received teacher training, how was testing addressed?
The Pop Quiz is a question posed to you, the Scholars of Doubt. Look for it to appear Mondays, Wednesdays and Fridays at 3pm ET.
*In my specialization, anyway. Other programmes in our department do require students to teach a class as part of these exams. We also have an informal system of classroom observation and feedback for graduate students who are lucky enough to be assigned courses to teach.
To me the word “test” is vague. I teach both science students (physiology, neuroscience, and zoology) and health science professional students (med, dent etc.) and I use many assessment tools during the term (including clickers, lab exit tests, lab practical tests, anatomy rat runs, lab reports, mini abstracts, case study assessments, and guided learning modules). I am not sure if any of them qualify as a test or not. I think what you are asking is what makes a good or useful assessment.
I get a lot of what I call tell me what I need to know to get an A students, I like to tell these students something one of my biochemistry lecturers once told me “take care of the learning and the marks will take care of themselves”. Assessment should help students take care of the learning. By that I mean that assessments should be designed to identify any misconceptions students might have so that said misconception can be corrected before the student gets to the final exam (or worse the big wide world of research science or medicine). Assessment should be designed to help students identify what they do not yet know. Alternatively, assessment should measure important practical skills that might not be assessable in a typical end of year exam setting.
The forgotten side of good assessment is that it should not only guide student learning but also help a teacher improve their teaching.
I’m using “test” in its common meaning, a classroom exercise wherein students must answer questions without talking to each other or referencing other material. Assessments can take many other forms, and in fact what I am asking is if these forms of assessment contribute more to the act of learning than do tests.
In any case, if a test is used to help students highlight what they don’t know, it would appear to serve this purpose whether or not it constitutes a part of the final mark/grade for the course (which is what causes the anxiety and distraction). Perhaps it’s best to use them in this way if possible, although in most cases this will create a workload problem since other forms of assessment will also need to be evaluated and recorded.
Also, if I said “take care of the learning and the marks will take care of themselves” to some of my students I’m pretty sure they would have a heart attack on the spot. I think this is indicative of a real problem in academic culture that is the direct result of increased emphasis on testing.
Yes, they probably will have a heart attack (or hissy fit) – let them. When they recover, calmly explain to them that your job is to teach them course content. Tell them that you are a teacher who cares (or at least it sounds like you do) and who will do whatever it takes to make sure they understand (and hopefully love) the entire course material (or at least the bit you are responsible for teaching). If they really understand the course material then they will pass any reasonable test or exam. The same for those “entire class sessions” you are losing to “exam protocols, reviews, and other minutia that distract from the business of learning”. Don’t do this, reclaim the classroom and give those sessions back to your subject.
On a side note: I don’t think there has been much of a shift to an “increased emphasis on testing”, I do think that students have become more concerned about passing exams as job markets have gotten tighter and, in NZ and Australia at least, university courses have become more expensive.
In most of the courses I currently teach (rather than the fantasy course I designed) I have found that dismissing these concerns just compounds the problem and creates more work for me. Instead of dedicating thirty minutes of class time to discussing the exam format, for example, I will get 80 emails asking about it. As a sessional instructor on a per-course fee I have to draw the line somewhere. Even without all the extra work my TAs earn a higher hourly wage than I do.
This problem is particularly intractable in large lecture-oriented classes, particularly electives for non-majors (many of whom really just want a fine art credit and an A for med school and don’t care about the material).
I have to agree with you that tests don’t always prove that someone is learning. Many homeschoolers don’t use tests when we teach because we don’t really see the point. A good bit of us as parents remember learning the facts for the tests and quickly forgetting them after the test is over and done with. Also just teaching to the test doesn’t leave room for deviation especially when something has sparked interest and you just want to follow that. I think if you can find a way to teach and prove that there is learning happening without a test that’s for the better.
In school I was always told that by writing and rewriting my notes I would retain the information better. It would stand to reason that if I had to no only take notes but then organize thoughts into a writing assignment I may learn more.
In a marvelous “coincidence,” this very issue came up in a recent on campus interview for a small undergraduate college in Pennsylvania–our dialogue was almost exactly the same. I work on John Dewey…the very epitome of the philosophy of education. For that reason and others, I have gone far, far out of my way to take pedagogy classes, practice my pedagogy, and look into the vast empirical literature surrounding pedagogy. An important point that your “Them” above and my interview committee often miss is that testing (in the humanities at least) is almost always based on in-class lecture and at home reading. Occasionally there will be other factors. The problem is that we know (almost for a certainty) that the most effective methods of learning are through teaching others (+/- 90% retention), actually doing an activity…writing papers, etc. (+/- 75% retention), and interactive group discussion (+/- 50% retention). Whereas the long term retention of topics that are put on exams have a MUCH lower level of retention. Lectures (+/- 5% retention), readings (+/- 10% retention), and even audio-visual–powerpoint, etc. (+/- 15% retention) are much less effective as pedagogical tools than the more interactive aspects of class.
So it is interesting to focus the question of “testing” by looking at the hidden assumptions about the goals of education. One rarely finds group work, in-class student teaching student, or student activities on a written test–though they are empirically the most effective ways for student to learn. Rather, we find our least effective tools–lecture, reading, audio-visual on the test. Even though we know the average long-term retention (whatever grades they may get on the exam) of these methods is very low.
It would seem that most teachers and professors (for complex social, logistical, and administrative reasons) are more interested in an easily quantifiable exam. Even though we know the students are unlikely to remember the information several months later. It would seem that most teachers and professors are more interested in a number than they are in actual learning….
That’s pretty much what my reasoning was in designing the sample course. In addition to frequent writing assignments and an analytical paper, the students were also supposed to give presentations on their final projects in order to “teach” their classmates what they had learned from their own research outside of the core readings.
Sadly in real life I mainly get stuck with talking-head powerpoint stadium lectures where it’s nearly impossible to give them anything but scantron multi-choice tests. One can dream…