Tag: Testing

If You’re Going to Do Test Prep, Don’t Be Horrible (20/365)

Photo by ShareGrid on Unsplash

We will, at a later date, examine me railing against test preparation. For today, though, let’s take a look at the problem below.

It is not unlike a problem a well-meaning teacher might put in front of their students. Because exposure/preparation/familiarity. Our well-meaning teacher would take this or something like it from a released bank of items on Standardized Exam Ultra and give it to their students to complete. Why not? The items are aligned to standards, the students will see things like this when they take the real Standardized Exam Ultra, so let’s get them started.

Pedagogy aside (and it’s quite difficult not to go off that way), a completed and scored set of these practice items would give the teacher little to no useful information by which he could shift his practice.

Let’s say 90 percent of the class misses question three. What would scoring this question tell our teacher? Well, it would tell him, in a class of 30 students, 27 got the question wrong. What he’d be likely to claim is this shows his students don’t understand verb tenses, commas, and apostrophes. That’s quite a bit to fit into a single question. In reality, there are at least 27 different paths each of these incorrect students could have taken to reaching the wrong answer. Getting this question wrong only informs instruction insofar as our teacher was wondering whether students got it wrong or right.

If our teacher insists on using this form of assessment, I’d point to a few easy tweaks that would provide greater clarity while increasing the cognitive load on students. I’d even go so far as to claim these tweaks would increase the likelihood of the students arriving at the correct answer.

My first alteration is to include a space under each question asking students how they reached the answer they chose. This reveals the paths students took not only to incorrect answers, but correct answers as well. Suddenly, our teacher is able to better understand process. It also moves student reflection into the assessment itself. Rather than asking students why they missed a problem post hoc, we are building a mechanism for students to consider the why of their choices in the moment. Here is the first place I’d argue our group of 27 is more likely to shrink as students pause to think about what they’re doing in ways exercises like these don’t naturally demand. We even put value on answers of “I don’t know.”

What if the shift above isn’t the only move we make? What if we add another space for feedback. This time, though, our students have passed their papers to another classmate and our teacher has asked each classmate to look at the answer and justification of her peer and then reply with her thoughts on the justification? At this point, we’ve begun a conversation, if small, where our students are no longer looking at the answers, but looking at the reasoning and asking if they arrived anywhere worth going.

Think, then, of the information our teacher is working with when he collects these papers. Rather than simply knowing if a given student got a question correct or incorrect, he now also knows how they got there and has given all the class an opportunity to chime in on that thinking at the same time.

The next day, he might pull together students who took similar approaches to reaching the incorrect answer and offer some targeted instruction in correcting their missteps. He might not point out the errors, but put our three correct students in a group with 9 other peers and have them work in small groups to consider how they answered and why they made the choices they did.

It is conceivable our class might grow frustrated with our teacher’s lack of simply telling the correct answer. This is good. I’m assuming these students have computers disguised as phones in their pockets and bookbags. When the discussion reaches its highest frustration, our teacher might say, “Okay, see if you can find help online.”

Here, we’ve taken something boring and inauthentic and built a community around it. We’ve manufactured value to something that originally would have told us if students picked the correct letter or the incorrect letter. While this is not preferable to some authentic, contextualized tasks that would also lead students to practice these skills, it is certainly better than we found it.

Things I Know 328 of 365: My brother has finals today

He’s 12.

He’s in the sixth grade.

He has three of them on one day.

Scantron’s included.

Someone stop this.

Things I Know 234 of 365: Testing is killing the curriculum

Too many professors feel right at home talking at students instead of fostering an engaging and interactive learning environment. Students are expected to sit there, take notes, and find some way to stay awake. The suck-it-up-and-endure-a-mind-numbing-lecture mindset is so ingrained in college, schools even assign room names like “Lecture Hall 4”.

– Liz Dwyer

A few months ago, a friend raised an argument to me, “We’re not teaching to the test.”

It was the first time in a while I’d heard someone make this particular case.

The temptation – the overwhelming urge – was to shout, “Of course you are! You are and you have been for years. Mountains of curricular history have been shifted so that exactly what you are doing is teaching to the test.”

Instead, I asked, “I see, then you’re teaching away from it, are you?”

According to Wayne Au of California State University, Fullerton, my initial response would have been the correct one.

In 2007, Au compared 49 studies of how standardized testing had shaped curriculum across 10 different states. He wanted to know what the trends were across studies of high-stakes testing and curricula.

According to Au, “The primary effect of high-stakes testing is that curricular content is narrowed to tested subjects, subject area knowledge is fragmented into test-related pieces, and teachers increase the use of teacher-centered pedagogies.”

Well, there you have it.

But Au found more.

As he began coding the data of his metasynthesis, he found the results breaking down into three categories:

  • subject matter content alignment/contraction vs. subject matter content alignment/expansion
  • form of knowledge changed/fractured vs. form of knowledge changed/integrated
  • pedagogic change to teacher-centered vs. pedagogic change to student-centered

After Au’s data was coded, he started to look for trends in studies that included two or three of the categories.

Were there trends in shifts toward teacher-centered lessons coupled with curriculum contraction.

He found them.

Most frequently, Au found content contraction coupled with a shift toward teacher-centered pedagogy. Teachers, the studies predominantly found, were contracting what they were teaching and teaching in such a way that they were positioning themselves as the sources and makers of knowledge in their classes.

In considering triplets where three of the coded data sets were present in 28 of the 49 studies, the most frequent trio was contracting curriculum, fragmented knowledge and teacher-centered pedagogy.

That sound you hear is the rolling over of John Dewey and Paolo Freiere in their graves.

Au’s reports that some curricula were actually expanding in connection to high-stakes testing was initially heartening. This was short-lived as he wrote that such expansion was often social studies teachers expanding their curriculum to take on those skills tested by English language arts assessments.

Au concludes his report claiming such constrictions were the end goal of policymakers from the outset.

The intent wasn’t to move the mountain. The intent was to chip away, re-shape and grind down the mountain of human knowledge so that students can carry around the pebbles of the human experience as mementos of what once was.

“Given the central findings of this study, however, a crucial

question is raised,” writes Au, “Are test-driven curriculum and teacher-centered instruction good or bad for teachers, students, schools, communities, and education in general?”

Things I Know 203 of 365: My dad could climb a rope

If I were to draw on a paper what gym does for me, I would make one dot and then I would erase it.

– Elizabeth Berg

On matters of policy, my father and I are traditionally at odds. Fiscal, foreign, defense, entitlements, everything.

Education is no exception.

While I’m able to steer clear of most of the others when we get together, I’m not so great at keeping my mouth shut when my dad starts talking about education policy.

Last night, we started talking about testing and Sec. Duncan’s decree easing the expectations of schools around the country to get to 100% proficiency as called for by No Child Left Behind.

My father is of the, “I guess that’s just another thing we don’t expect of our kids anymore” mindset.

He works in the technology office of the school district from which he graduated. As anyone can imagine, this means he often sees the worst from teachers. Rarely do faculty members bake cookies for tech team.

To further illustrate his point of the lowering of the bar for today’s students, my dad talked about a rope.

He first encountered the climbing rope on his first day of middle school P.E.

It kicked his butt.

A competitive swimmer from way back, my dad thought he should have been able to make his way up the rope with no problem. Such was not the case.

For weeks, my father struggled to make it to the top of the climbing rope.

For weeks, he could not make it.

This, for my father, was the bar to which all students should be held.

“I walked through the gym the other day, and do you know what I saw?”

Not pausing for a response, my dad continued, “The rope has knots in it.”

I was confused.

He explained.

The same climbing rope, which had been my father’s adversary for weeks in his youth, had single knots running in it every few feet up to the rafters.

Dad explained this and sat looking at me for a moment.

“It took me weeks to get up that rope, but when I did, I knew I could.”

He lamented the knotting of the rope the same way he was lamenting the easing of NCLB’s testing requirements.

“Are we too worried kids aren’t going to feel good about themselves, so we make everything easy on them?”

I see his point – I really do.

For the same reason folks are worried playgrounds are becoming too safe, learning should have some scraped knees, some trial and error.

My problem with my dad’s point accepts his metaphor and rejects his premise.

What are we still asking kids to climb ropes?

Maybe, in dad’s day, the climbing rope was the best we could do to figure a upper-body strength and endurance. Maybe, way back when, we had no other choice than to make a student’s learning and abilities a matter of public display. Maybe, when my old man was coming up, we didn’t know any better.

I doubt any of that was the case, but I’m willing to give him the benefit of the doubt.

Today, though, we certainly have better methods and tools at our disposal.

Cut down the ropes, find out the best ways to figure out if kids are fit and healthy, and then truly teach them how to do it better for reasons other than their peers will laugh at them if they don’t.

Yeah. The metaphor works.

Let’s cut down some ropes.

Things I Know 182 of 365: Sometimes, I’m ready to move on to the next discussion

Discussion is an exchange of knowledge; argument an exchange of ignorance.

– Robert Quillen

I won’t call it an argument.

What I got into this evening that lasted from the beginning of dinner, through settling the check, the drive to the movie theater and up to the point at which we took our seats for the previews was a discussion.

We discussed education.

In a party of 9, my friend and I ignored all others and searched for common ground.

He discussed from a deficit model.

He discussed the importance of standardized tests.

He discussed there were far more teachers who should be fired than were excellent.

He discussed end-of-year assessment and the ability to write a well-reasoned essay as the marks of a highly-qualified teacher.

I argued against it all.

I argued against it all except the very last point.

I wanted to. In my argument for project-based assessment, for the value of asking students what they can create and teaching to their passions, for the idea that having students read from textbooks precludes the idea that a teacher has created a constructivist classroom – in all of this – I could not make my way to the argument that writing an essay should no longer be the measure.

This thought hung on the coatrack in the back of my mind as I attempted to make my exit from the discussion.

He wasn’t ready for the idea that what might deserve our focus is teaching students to make arguments, but that writing them down – on paper or screen – mightn’t need to be the standard by which we measure their rhetorical abilities.

It was incredibly frustrating to realize how many layers of discussion were necessary before I would be able to get to an idea I recognized as truly progressive.

I wanted to suggest having kids write with video integrating links, tags and annotations a la youtube videos could liberate voice, deepen understanding and lead to more dynamics arguments. I wanted to suggest that writing in words wasn’t native to the human experience, that doing something because it’s what’s been done for centuries isn’t answer enough.

Instead of this, I had a discussion I’ve had time and again regarding truths I take to be self-evident. It was a moment of frustration. I want to be having a better discussion based on a common belief that learning and adequate yearly progress are not the same thing. Tonight, I had hoped that our conversation of what education can be could come from a mutual belief that teaching is a respectable profession and that we must care for teachers as we would care for students. It turned out, we weren’t ready for the conversation.

I’ll find myself in some iteration of tonight’s conversation again (soon, I’m sure). I will listen and question and push with as much vehemence as I did tonight each time I’m allowed.

Still, in some moments, it would be nice if we all decided we were ready for the next big conversations.

Things I Know 178 of 365: Report cards should be better

Zachary has been a joy to work with this year. I will miss him next fall.

– Mary Cavitt, my kindergarten teacher

From kindergarten forward, the majority of schools get progressively worse at telling students and parents what’s being learned and how well students are learning it.

A few days ago, my mom stumbled upon a folder marked “ZAC – School” while searching for immunization records.

Not the least of the documents in the folder were both my first and my last report cards as a K-12 student.

I found my final high school report card first.

When I saw it, my eyes flashed to class rank, then GPA, then a quick scan to remind me of my final courses and teachers.

A few pieces of paper later, 24 years after it was issued, I found my kindergarten report card.

It required a little more time for consumption. Nowhere did it tell me where I ranked among the other 5 and 6 year olds. I had no idea as to my kindergarten GPA either.

The only real use for the report card was a detailed accounting of my progression as a student throughout the year.

Groan.

I could remember every piece of information from that first year of K-12. I would be hard-pressed to recount half of 1% of anything covered in my senior macroeconomics class. I’d honestly forgotten I’d taken macroeconomics until I saw the report card.

In the comments section for each quarter, Mrs. Cavitt wrote a short message to my mom alerting her to my progress and letting her know I was being seen by my teacher.

In the “Comment Explanation” section of my senior report card – nothing.

As a kindergartener, I had little use for my report card. It was a document for the adults in my life to examine and use as a starting place for conversation.

In my later years, the report card held much value. It was a quarterly mile marker of my progress toward college and beyond. Still a communication between the adults in my life, it raised more questions than answers. I have no idea how I got that B in my first quarter of English IV, nor do I know what improved in the second quarter that led to an A.

I’m certain my parents asked questions on these very topics. I’m sure I stumbled through my answers and took stabs at the multitude of possible reasons for my grades.

I try to imagine, though, what would have transpired were I not as successful as a student.

If I’d been lost in the tall grass of high school with Cs, Ds and the occasional F, this report card would have served no purpose other than to reinforce my failures and dumbfound my parents.

If I’d not had such dedicated parents, the conversations would have stopped there and the frustrations would have continued to mount.

The modern middle and high school report card is an arcane relic made supremely ironic in light of the millions of dollars spent nationally in the name of gathering data.

I’m certainly aware of the systemic impediments in place, but improving communications with students and parents on individual learning need not include standardized tests and computer-generated reports.

At the end of my kindergarten year, in math, I could name the four basic shapes, count to 43, add, subtract, print my numerals and much more.

At the end of my senior year of high school, in math, I got a B.

Which measure focused on the learning?

Things I Know 89 of 365: It’s the testing holidays

Not everything that counts can be counted, and not everything that can be counted counts.

– Albert Einstein

Used to be it took a proclamation from Congress to create a holiday. Though education reform initiatives over the last decade haven’t exactly been proclamations, they’ve certainly created a holiday. A month-long holiday.

For SLA, the holiday season started last week with the advent of Pennsylvania’s standardized tests, the PSSA’s.

Math and reading were last week, writing finished this week and we’re looking forward to the science test in two weeks as a break from teaching.

When I met with one of my G11 classes today, we actually took a couple of minutes to greet each other and catch up as the testing schedule has kept us apart over the last few weeks.

We took separate vacations.

And while the tests have meant a break from the meaningful and authentic learning our G11 students engage in at SLA, some schools treat the testing holidays as though they are the time of year when their students do any real work.

Phone calls are made home to remind students to show up to school. Some schools cater breakfasts for students.

Students know they will be rewarded with the educational equivalent of Christmas bonuses if they’re shown to have made top gains when the testing results come in.

And while G9, 10 and 12 students at SLA attended classes as usual (with some room switches), in other schools the testing holiday looked like a real holiday for those students not being proctored.

Oh, the proctoring.

Again, treating the holiday season as though its more important than when actual learning is taking place, proctors face rooms of testees with attitudes that are, well, testy.

Looks of scorn and dictatorial attitudes are assumed in an effort I can only assume to frighten the smarts out of the students.

I guess I missed the study showing stressed students preform better.

On the other hand, there’s a way to treat students like people – even during the testing holidays.

Talking to kids as they enter the room, providing them with peppermints, smiling, treating to them the same way you treated them before the break and the same way you will treat them when classes resume.

If we must test (and for now we must), let’s treat it like school.

I know it’s not – not the best versions of school, anyway.

Still, let’s pretend so we can avoid those humanity gaps that we know can occur over breaks in learning.

Things I Know 79 of 365: My students are readers

Books let us into their souls and lay open to us the secrets of our own.

-William Hazlitt

As my students amassed this afternoon, I met them outside my classroom with the door closed and waited for the last stragglers to, well, straggle.

“Partner up with the person in class who you think is the best researcher,” I said, “When you have a partner, you may enter the room.”

As they partnered and entered, I told each partnership that one of them should open a blank Word doc.

“I’m going to ask you a series of questions,” I said.

For each question, the partners needed to sniff out the answer, document their source and, if the source was a PDF, document the number. Answers needed to be in complete sentences, preferably restating the question as a statement.

Before I began with the questions, I told the class about running into a friend this weekend at the coffee shop near my house.

A fellow educator who knows the belief structure of SLA, with a smile in her voice she asked, “So, have you guys just been drilling and killing?”

We both laughed.

“Not so much,” I said, “I did bring it up last week. I figured, if they’re going to take the test, we should probably talk about it.”

It’s true.

That’s what I said to her and how I brought it up with my students.

Tomorrow, my G11 students will take the first two sections of this year’s standardized tests.

Today, we prepared.

Rather than prepare a slidedeck explaining the inane nuances of the test, those same inanities became the questions for our research today.

“How many sections of reading are their on the G11 Reading PSSA?”

“How many of each type of question is in each section?”

“What are the possible genres of reading passages on the test?”

And they searched and found and filled in the holes. Some were frustrated, others downright competitive.

The moment that struck me and the moment that let me know we were doing the right thing was when one of my students offered up, “It feels like we’re searching for classified information.”

I flashed to David Perkins and Making Learning Whole and everything he had to say about learning the hidden game.

I know Perkins was talking about the hidden game in real, worthwhile learning and not standardized tests. In the eyes of the state, sadly, the next few weeks represent the realest of real learning my students will be doing this year.

The grant project and the building history project will mean nothing, nor will the multiple books the kids have blazed through and the conversations they’ve had as readers.

Perkins talks about the hidden game as the pieces of learning that are unspoken and unknown except to those who know how to play well. They might not even been understood by those who play well – they just are.

I suppose, aside from some practice in researching, that was the other goal of today’s exercise. I wanted them to know they will find 22 multiple-choice and 2 open-ended questions tomorrow before they sat down so they don’t need to worry about the rules. All they’ll need to worry about tomorrow is reading.

They can do that.

They can read, question and converse better than many undergrads and grads I’ve known. They know what they look for in a book and can tell you. They can tell you why a book is boring and why it’s exciting. And, they’re working on learning to read more closely than most people I know.

They are readers.

I told them that.

I told them that, and I told them to slow the frak down tomorrow.

It’s the best way to play the game.

Cheating (in) the System

6 August 09

So tired today. It may be the teaching of backward design. It may be the fact that I made the mistake of wearing long pants and the room I was teaching in had no air flow to speak of. It may be any  number of things. My money is on the idea that we’ve been going for about 5 weeks now and there are bound to be days that are more difficult than others.

Today was that day.

I’d been told that some teachers will purchase pre-written exams and administer them to their learners without reading the contents. This, I was told would lead to learners attempting to answer questions about material they hadn’t learned or even encountered in class. Try as they might, many would fail and need to repeat a class because their teacher was too lazy to write a decent exam.

I hadn’t believed it when I’d heard it.

When discussing the importance of designing assessment in Stage 2 of the Understanding by Design process, I repeated the story I’d heard.

“Does this really happen?” I asked.

“It does,” some responded while the others simply shook their heads at their lesser colleague’s actions.

I suppose my main ire is at the idea that, if you’re going to be an exam-centered educational system, at least do it well. Have a system you can be proud of or at least defend.

I’ve started to identify in some inexplicable way with belonging here. The unexpected affirmation of what I’d heard from my colleagues made me angry. Too much stands in the way of education for teachers to be pulling a bait and switch because they can’t bothered to do their jobs. More than just this, I think I’d enjoyed the idea that the similarities I’ve found between the South African, Kenyan, Canadian and American educational systems ended with what we do right. I didn’t want a reminder that our lowest common denominators are, indeed, common.

Maybe I’m just tired, and I should remind myself of the teachers here this week and their seemingly endless dedication, curiosity and hope for being able to do even better by their students.

Yeah.