What’s going to be on the exam?

Standard

Do you love it when students waste office hours with questions that don’t help them learn? Do you want to cultivate anxious emails from students sent at 3 in the morning? Do you want your students to wager their grades by guessing what you think is the most important material?

Then don’t tell your students what is going to be on the exam.

Continue reading

Time limits and test anxiety

Standard

When the clock is going TICK TICK TICK, it can be hard to think clearly, because you’re anxious about the clock.

Math anxiety is well understood, and no small part of this comes from the pressure of timed tests. Ultimately, some people take tests faster than other people. I would hope that you want your tests to measure how much students have learned, not their ability to take tests under pressure. If this is the case, then everybody taking the test needs to feel that they have adequate time. Continue reading

The case for open book exams

Standard

In the sciences, most exams are a closed-book affair. Is this a good thing?

Open_book_nae_02On some tests, I’ve allowed students a 3×5 card, or a one page “cheat sheet.” This is usually met with relief, or joy, or gratitude. When I tell students that they can bring in their textbook for the exam, they get even more relieved.

I might say, “Don’t be so happy, because this just raises the bar for what I’m asking on the exam.” But then, my students say that they feel like it’s not useful for them to have to memorize stuff. And they would prefer solving problems and applying information in novel ways. Even if memorizing stuff is important, it causes a lot of anxiety. Continue reading

Standards-based grading

Standard

As we start up the new semester, this is an apt time to evaluate, and update or change, our grading schemes.

I don’t like giving grades. I wouldn’t assign grades if I didn’t have to, because grades typically are not a good measure of actual learning.

Over the least year, I’ve heard more about a new approach to assigning grades, that has a lot of appeal: “standards based grading,” in which students get grades based on how well they meet a detailed set of very clearly defined expectations. This is apparently a thing in K-12 education and now some university instructors are following suit. Continue reading

Students say the darndest things!

Standard

Oftentimes, professors make sport of sharing humorously incorrect exam answers. I’ve seen a bunch of these during this end-of-semester grading season.

When students don’t know the answer, they sometimes entertain us with witty, technically correct answers that don’t answer the intended question. (There’s a well-selling book about this. And at least one website, too). But that’s not what I’m talking about. Continue reading

“What’s going to be on the exam?”

Standard

Do you love it when students waste office hours with questions that don’t help them learn? Do you want to cultivate anxious emails from students sent at 3 in the morning?  Do you want your students to wager their grades by guessing what you think is the most important material?

Then don’t tell your students what is going to be on the exam.

It is entirely legitimate for a student to be told the basis of their evaluation. Students take a course, and earn a grade. They should be made aware, as specifically as possible, the foundation for this grade before they do what it takes to earn it. The less they know about the basis of their evaluation, the less fair we are to our students.

The more specific you are about what is on the exam, the happier the students will be. Moreover, specificity gives you control over the material that they will study. I have often heard colleagues frustrated that students aren’t focusing on studying the right material, or asking the right questions while studying. I seriously don’t get these questions from students, and I think that both they and myself are better off for it.

If students ask you what will be on the exam, please don’t reply, “Whatever I think is important.” That just will help the students who are better mind readers. (Which are probably those with a social and cultural background most similar to yourself.)

If students ask you what will be on the exam, please don’t reply, “Everything in the lectures and everything in the chapters.” This vague set of expectations will prevent students from focusing on facts and concepts which are most important, and may lead to some students wasting their time on minutia.

The more vague you are about what will be on the exam, the less control you have over what they study. The worry about the mysterious contents of the exam detracts from learning. Our value in the classroom isn’t our content knowledge itself, but our expertise that allows us to parse the useful, meaningful and relevant. Asking the students to master too much information will result in no mastery at all.

Several years ago, I decided that the exam guesswork was a bad thing. Now, I administer three types of exams, all of which are designed to remove guesswork on the part of students:

  • I give students a full list of potential exam questions in advance. I then select a subset of these questions for the exam itself, choosing at random, haphazardly, or with a specific rationale. Here is an example of one, from a non-majors Environmental Biology lecture course.

  • I give students a comprehensive exam preparation sheet, one or two weeks before the exam. I give them a solemn promise that everything on the exam will be covered by one or more items on the review sheet. Sometimes these items are very narrow but other times they could be rather open-ended. But they are never intended to be vague. I tell the students that if any question on the exam isn’t based on one of these review items, then I’ll drop it from the exam. I am also tempted to hand these out at the beginning of the semester, but I call too many audibles to make this a wise choice. Here is an example of one, from the first exam in a biostatistics course. You’ll note that there’s a lot of material on there. I can’t ask questions to cover every one of those items. But I can make sure that students study them all, but also make the scope narrow enough that it is do-able.

  • I can give a take-home exam. I only do this if I’m blessed with a small class. Because are a variety of problems associated with take-home exams, I typically only do this with a small graduate course.

One of the more annoying questions that a student can ask is, “What’s going to be on the exam?” I just have to answer that with a single piece of paper.

Sometimes some students will email me, “What’s the answer to number 8 on the exam prep sheet,” or they’ll write me an answer and ask me how it meets my expectations. I make a point to not evaluate their responses or give them any information, unless I do so for the entire class. I might clarify a question or an item, if a student doesn’t understand the words.  Under all circumstances, I assiduously avoid evaluating providing privileged information for the students who feel more comfortable with approaching me for private studying advice, because that would be unfair to the students who don’t email me. I might send a reply to a question to the entire course.

I always schedule time during a class session prior to the exam so that students can ask me questions about any of the review items. Sometimes this lasts just a few minutes, and sometimes the bulk of the class period. (I do not hold separate reviews outside regular class hours, as I’ve mentioned before.) Usually when students email me a question, I ask them to save it for class, so that everyone can benefit from their question. But most of the review session is me saying, “I’m not going to reteach that entire lesson, but this is the nutshell version.”

The better I construct the exam prep information, the less time we spend in review during class, and the more time students spend studying with each other, which is where the real learning takes place.

All faculty need academic freedom to protect their students

Standard

Sometimes I hear questions like, “Why is academic freedom so important? Why should university professors should have total control over what they teach?”

Let me answer those questions with a cautionary tale.

Last semester, a shortage of academic freedom in one department at my university caused what can only be characterized as a tragic boondoggle. This is causing an entire cohort of students to graduate one year late.

Over fifty Biology majors were enrolled in the second semester of General Chemistry. An adjunct lecturer planned and taught this course. The tenure-track faculty in Chemistry implemented their own common internal exam to be administered to all General Chemistry students. The instructor was not privy to the contents of this exam while she was teaching this course. Consequently, over the entire semester, the lectures and homework assignments did not correspond to the material that the students were tested on at the end of the course.

The students, who had been performing well throughout the semester, were blindsided with an exam that looked nothing like they had been studying for the whole semester. This class historically has a pass rate exceeding 80%. Last semester, however, more than 80% of the students failed. The instructor of record for this course, who taught the whole semester, did not apparently have authority over the grading of the exams, nor final authority over the grades that she was directed to submit to the university. This sounds outrageous, but also sounds like the only sensible explanation for what transpired.

Most of these students clearly did not deserve to fail. They did not deserve an exam that did not reflect the content of the course itself. They deserved an instructor who has the authority to control the grades assigned in the course.

The chair of the department is not making any accommodation for the students who got screwed over in her department. The chair claims that the students simply weren’t prepared for the exam. I don’t dispute that fact, but in this circumstance the lack of student preparation is the fault of the Chemistry department, not the students. The students fulfilled the academic expectations of the instructor, but that had no connection to their grade. That is flat-out unethical.

The consequences of this F go well beyond this single course. None of the students can retake the course this semester, because those sections were filled by those who passed preceding course in the sequence.

The soonest these victims can retake the course is one year after they were originally enrolled, but now we have twice as many students trying to take this course and the Chemistry Department refused to offer any additional sections to its victims from last semester.

This course is a prerequisite to Organic Chemistry, which is a prerequisite for other courses. Nearly all of our majors in this section – more than fifty students – are now going to graduate at least one year later than they had planned.

What’s the worst part of all this? It happened two months ago, and as far as I can tell, the only people who aware and troubled are the ones who have no power to change anything.

If any of our students had families donating large sums of money to the school, this situation would have been resolved lickety-split. If anybody with authority in Chemistry actually cared about the students, this would have been fixed before the semester ended. If department had any confidence in their trained contingent faculty, then this unjust situation wouldn’t have emerged.

The students can file a grade grievance, but that won’t fix the problem. It takes at least a year for that process to go through the system. (I served once as a “preliminary investigator” for a grade grievance claim, and the incident happened three semesters earlier.)

You might ask, “Aren’t common exams an effective way to make sure that there is consistency in grading when section are taught by different instructors?” The answer to that question is yes. However, that consistency has a price. In this case, the price is reasonable academic progress for scores of students. Keep in mind that most of our students work long hours in addition to a full class load, and also have substantial family concerns at home. Being in school is a great challenge, and we just made made the climb to graduation even steeper.

The required use of common exams deprives instructors of the academic freedom to evaluate their own students.

If similar events had taken place in any of the three private institutions in which I’ve taught (as adjunct, visiting, and tenure-track), this disgrace would be unthinkable and scandalous. There would be mass protest. But at this disadvantaged university, it’s just one more injustice.

At this point, I’m not even sure if our administrators are aware of this incident. I have a huge amount of confidence in the Dean and the President, who I imagine would do everything they can to resolve this situation, insofar as it is possible. The fact that this problem wasn’t a howling and yelling crisis at their doorstep at the end of last semester is a sad testament to the fact that our students are just accustomed to being disempowered, and they just roll with being wronged. It’s our job, as faculty, to prevent these wrongs from at the outset. That starts with giving all instructors that academic freedom over their own workload.

If any instructor is good enough to be hired as to teach for the university, then they’re good enough to be trusted by the university to carry out their job independently. Any department that lacks the faith that its own instructors can teach appropriately has huge problems that can’t be fixed by imposing a top-down exam.

As a postscript, I should note that common exams are not always a disaster, though I think they are inadvisable. In grad school, I used to teach three sections in a class that had more than 40 sections. All of the TAs gave the same exam, and we had little control over this exam. We didn’t even get to see it until a few days before we taught, because it was a practicum set up at the last moment. I see the need for consistency among sections taught by graduate students with little to not teaching experience. I don’t see the need, however, for this particular solution.

How the heck was I supposed to know what to teach when I didn’t know the basis on which students were going to be evaluated? This was obviously a problem for students. (I also lacked the experience and professionalism to deal with this situation effectively.) This was mostly an annoyance, though, and the students did just fine in the end as best as I can recall. The lab was not overly detailed, and the exams weren’t overly idiosyncratic. As a novice instructor, I found the system to be unfair to both myself and the students. If instructors are teaching a course, they should be able to construct or choose their own evaluation. If for some reason that doesn’t happen, at the very least the faculty need to know exactly what is in exams before the start of the semester.

Don’t waste tenure

Standard

Tenure gives us academic freedom.

This doesn’t have to be an empty concept.

Before you get tenure, you need to live by the rules. After you get tenure, you have to follow the written rules but not live by them. You can choose to ignore as many unwritten rules as you think is wise. You have a lot of latitude.

I find it lamentable that, for most people, including myself, tenure changes very little.

It’s as if six years has been calculated as the amount of time that it takes for us to become a cog in the system.

Here’s another way to think about the time you spend in the system prior to tenure. Being a professor is like being a musician. Don’t they say that great musicians know all of the rules, and after developing technical expertise they know which rules to break, including how and when to do it, to to make extraordinary music? (I’m not talking about John Cage, but maybe the Beatles or the Pixies.)

Do you think the same is true for our academic careers, that we need to know which conventions have to be broken in order to excel?

In terms of research, I have always attempted projects with the notion that their success or failure wouldn’t alter my risk of unemployment. I have a variety of high-risk-but-potentially-really-cool projects happening, that I probably wouldn’t be inclined to take a chance on if I needed to focus on getting more papers and grants. (I’m able to take these research risks not because of tenure, though, but instead because my university isn’t in a position to expect much research productivity.)

Here’s one major change that I’ve made, that I wouldn’t have done pre-tenure: I rarely lecture. When I step into a classroom, I have a set of activities, discussion items and problems to be solved. I might occasionally bust into a 3-15 minute explanation of some topic if the circumstances require it. (As a caveat, I haven’t taught an introductory majors course in a number of years.) There are two reasons I wouldn’t have done this change pre-tenure. I wouldn’t risk a potential tank in my evaluation scores (which actually moved very little). And I wouldn’t have risked doing things so differently from my department mates who would be sizing up my tenure file. (My last department chair at my old job who observed me was actually put out when I stopped lecturing for a 2-minute think-pair-share activity, as I mentioned in the teaching/tenure post on Monday.) How do I find the time to do teach like this? I actually find that doing this takes less time than preparing a decent traditional lecture. I am actually concerned about being accused about not putting enough effort into my lessons, because the real work is being done by the students and not by myself. I just arrange the circumstances for them to learn. That is good teaching, in my view, but at most universities the lecture predominates and a departure from that practice might be viewed upon with suspicion, especially by scientists who aren’t trained in education. I haven’t had much formal education training, either, and by no means am I convinced that I’m doing it the best way. Which is why I call this change a risk that I can do with the benefit of tenure.

Here one risk that I’ve considered in my post-tenure era, but not had the guts to implement yet: In my Biostatistics class, I’d like to entirely do away with all quizzes and exams, and simply implement an oral performance-based final for 100% of the grade. By the end of the semester, the expected outcomes are so straightforward that a student should be able to demonstrate competency or mastery in the context of a short conversation and ten minutes with the software. (Last semester, gave my class the option to do this instead of doing a big take-home final, but everyone picked the time-intensive final.)

Here’s another risk I have yet to do: Instead of offering 0% participation points in a class (that’s a forthcoming post, at some point), switch to 100%. Grade students wholly on perceived effort. That’s the beauty of academic freedom. I can do this. I can’t be fired for it, I don’t think. Imagine if students were getting grades for trying, rather than for guessing right. Wouldn’t that be beautiful? Again, I’ve yet to try this.

This blog is a risk I took only because I have tenure. I’ve put effort into remaining civil and positive. However, a variety of things that I’ve written already could have been a huge liability before tenure, about my former dean, former president, how my campus is tragically underfunded, and how I interact with students. The only thing preventing me from telling it like it is, is my lack of confidence that I really understand how it is. That’s a nice perk of tenure. There are plenty of other pre-tenure bloggers, such as Dr. Becca among many others who I’ve linked to previously, but their identities are often hidden.

Once you are tenured, what risks would you want to take with your research and teaching? If you have tenure, what new risks have you taken or are you afraid to try?

Efficient teaching: exam writing vs. exam grading

Standard

Grading sucks. I hate grading. I guess even the best job in the world has its downside.

I hate grading because it makes me think about all of the less un-fun things that I could be doing at that moment. And I often do those things (like laundry, or dishes) instead of grading. Which only makes grading worse.

I’ve never actually given a scantron exam in a class that I’ve had the freedom to teach how I wanted. I haven’t had a massive class that’s required this approach (or a small army of student graders). I’m not inherently opposed to multiple choice exams, but I’ve mostly been in places where they were not appreciated, at least not in the kinds of classes I have been assigned.

One of the reasons I haven’t liked multiple choice exams is that writing a good one takes a lot of time. (And, even when I’ve used some of these questions on a paper exam, I find that they can contain a lot of hidden cultural biases that only come out when talking with students afterwards.)

If your exams aren’t multiple choice, then how can you do spare yourself grading hell?

You can’t. But you can lessen it.

Which is more annoying, writing an exam or grading?

Which is more annoying, writing an exam or grading?

This is just a working hypothesis. It’d be interesting to really know. But not interesting enough to delve into the education literature.

On one extreme, you could write an exam in ten seconds. It would ask:

Explain in detail the five most important ideas that you learned while studying for this class in the past month.

Very easy to write, very hard to grade.

When underthinking exam questions, then we could be in for a world of hurt when we have to grade the responses. You can’t necessarily create a perfect rubric up front, because you might get correct but unanticipated answers.

Students can put all kinds of crazy stuff down when you ask questions on exams. Sometimes, this crazy stuff is actually factually correct and directly answers your question. Even if the answer is not addressing the content that you were expecting the question evaluate?

Badly Worded Question: When the earth had more oxygen in the atmosphere, would the sky appear orange or not? Explain your answer with a sentence.

Correct answer: Yes. The sky would either be orange, or it would not be orange.

I have a really hard time marking points off a question which is fully answered correctly, even if the correct answer isn’t what I anticipated. (In fact, I promise I won’t do this to my students, as a part of a policy of transparency and fairness.) The onus is on me to write questions that directly get at what I need to know to assess their content knowledge and the ability to process it (Blooms taxonomy, yadda yadda)

So, if you make sure your exam is airtight going in, then grading it should be easier.

But writing exams is no fun either. So we don’t write the best exams in the first place. Maybe the memories of painful grading are enough of a stick to make us write tighter exams.

hat tip to Prof-Like Substance for venting about grading.