I’d like to extend a topic that I brought up recently – about the difficulty in evaluating excellence in teaching.
I danced around the central conundrum, by highlighting all the ways that we do, and don’t, evaluate teaching on our campuses. What I didn’t bring up is: what is it, exactly, that we are sizing up when we decide how good someone’s teaching is?
What I argued at the time is that most people use the template of their own teaching (or perhaps one’s aspirations for one’s own teaching) as a model for evaluating others. This operational definition of how excellence is measured doesn’t actually specify what characteristics define excellence.
Let’s set aside the evaluation problem, and ask: What is excellent teaching?
Nearly all of our evaluations focus on the process of teaching: what happens in class on a day to day basis.
What really matters is what happens in the end: what is the product of teaching?
If we focus on process, then we measure things like specific items in the curriculum, demeanor, what the assignments and exams look like, and classroom performance in the theatrical sense.
If we focus on product, then we look at outcomes: exam performance, student satisfaction, long-term professional growth, and — from the administration’s perspective — alumni giving.
You imagine that a quality process leads to a quality product, however a veneer of quality might not lead to a long-term high quality product. What educational practices that get high ranks in the evaluation system actually contribute to positive outcomes in the long term? How much of what we do in teaching doesn’t result in long-term learning but looks like quality teaching in the short term? For starters, the first thing that comes to mind is the use of overly detailed powerpoints that are shared with the class online for cramming studying for exams. Students love it, it raises satisfaction, the professor looks well prepared and: will students know any of it the next day, or how about five years later?
Just as we can design an individual class with backward planning, we can think of the whole college experience this way to inform what students really need to get what we want them to get out of the college experience. What constitutes an “excellent” college education? Whatever it is, then excellence in faculty is when they deliver on that goal.
In my view, when you graduate from college, the school has done its job if the product is:
- articulate in thought, words and writing
- able to differentiate between opinion and reason and has personal values informed by both
- broadly interested in the world and all it has to offer
- conversant in literature, art, history, geography, science, mathematics, philosophy, civics and more
- able to separate style from substance
- struggling to understand the faiths, or the lack thereof, of those in their midst
- able to readily explain to any novice the basic tenets of their specialty
- can perform admirably on a summative exam in the subject of one’s specialty (e.g., subject GRE)
- reading the goddamn newspaper or its equivalent on a regular basis
- able to whiff out pseudoacademic thinking (like Malcom Gladwell), pseudoscience and shabby reasoning
- not taking oneself too seriously
- treats other people as they would like to be treated
Those are my aspirations for what college should do for everyone. I wrote this on the fly but I imagine it would resemble to some degree what you think a “liberal arts education” is supposed to look like.
Just as it’s not the job of a K-12 teacher to merely teach the state standards, it’s not my job to cover the expected learning outcomes of my course. I’m working on providing one piece of a holistic college experience. Am I successful at this? No. But it’s a goal. Since I’m at a state comprehensive institution and all the talk is “career-training-this” and “job-preparation-that” we are really botching these fundamentals. My heart soared when our newish provost said, in his first address to the faculty, that he didn’t want to see any more business regarding what a college degree does for students, he wanted to hear more about what a college education does for students. In our environment, that idea is hard to deliver in deed, but I’m excited to see the emphasis nonetheless.
How do we measure teaching excellence? Foremost, students need to emerge from the class knowing the course material. In particular, they need to know it deeply: well beyond their graduation dates. Truly excellent teachers inspire lifelong learning, excitement tied to discovery in the discipline, and an academic ethos that pervades all aspects of life. There is a multiplicity of routes to this destination.
This is often why teaching is considered to be an art, because what works for one person might not work for another. The trickier part, though, is knowing what works. This might not be measurable at the end of a semester.
Small Pond Science 
Isn’t excellence in teaching about both process and product? Inputs and outputs? The trouble with just focusing on outputs is that the outputs are affected by lots of things out of the teacher’s control.
… which is why teaching in some environments is more challenging than others.
If a teacher is being evaluated based on outcomes, ignoring external factors, that’s messed up, as any K-12 teacher will tell you.
However, if you look at the process, without evaluating the effectiveness of the process by looking at the product, then that’s meaningless.
Process matters, because process has to be good for the product to be good. But if you’re not focusing on the product, then say whatever the hell you want about the process, it won’t matter because you’ll never know what’s effective by watching it.
The only way we can look at process and say, “that’s good teaching” is that if the specific items in the process that are being evaluated are being shown to be effective instruction. At a lot of different places, what science departments perceive as high quality process results in very poor long-term learning. That’s NOT excellent teaching. It’s an excellent performance. It’s customer service.
If you look at (final-initial) this could be a good way to think about it. In my view, ‘final’ really matters long after a student graduates. The assessment nerds are all about ‘learning outcomes.’ I’m not a member of the cult but one thing that the assessment movement has right is that the bottom line isn’t what the professor is doing, but what changes in the students over the course of a semester and a program. What the professor does is what gets tweaked to make the changes in the students more favorable. The outcome informs the teaching method.
Let’s say we’re cooks. I don’t care what the hell happens in the kitchen if nobody gets hurt and the meal comes out delicious. If the cook needs to improve, we need to look at the process in the kitchen. But what characterizes excellent teaching isn’t what happens in the kitchen, it’s what gets served on the table. You don’t watch a chef and say, “what an amazing cook!” (though admittedly the knife skills of pros are might cool), instead what you do is you eat at their restaurant and you make that decision.
A lot of science professors have lots of kitchen appliances, excellent raw ingredients, good knife skills, and a top notch waitstaff, but the meal still ends up unsatisfying. And that’s because their teaching is evaluated by overemphasizing process without looking at product. Okay, that metaphor is stretched too far.
Unfortunately it’s difficult it is to do a really good assessment of final outcomes of the sort you’re rightly focused on even right after the student graduates. Never mind many years after.
I agree with much of what you’re saying here Terry. But I’m not clear on how, even an ideal world, we could ever do the particular sort of outcome-based assessment you seem to want to see. Apologies if I’m misunderstanding. Can you clarify or elaborate? How do we actually *do* the sort of ideal outcome-based assessments you’d like to see, and how do we isolate the contribution of college teaching to those ideal, far-down-the-road, fundamental outcomes?
Oh, I just point out the problems, I don’t know the answers!
It’s all f’ed up, that’s all I know. I have no policy recommendations.
I think, as faculty, we need to eat at the restaurant more. Meaning, we should see how our students are progressing from a holistic standpoint. Intro faculty should evaluate their teaching by teaching the same students in the upper division, for example. (Also, we should be asking our students about what novels they’re reading, chat about what happening in the news, and stuff like that.)
p.s. on re-reading your comments again, I’m guessing I may be misunderstanding you on when you’d like to see outcomes assessed, and you’re not actually suggesting that they be assessed many years post-graduation, not even in an “ideal” world?
No, no suggestions. Maybe one. It would be interesting to have a graduating senior retake an exam from intro bio. I think it would really make the professor question what they’re doing when it goes in one ear and out the other over the course of a few years.
I lived in Japan in the summer of 1996, part of the NSF East Asia Institute. I had two hours of language classes each day. I had a whole notebook. Now, I can’t not only not understand it, but I can’t even read it. It’s on my shelf as a reminder. Hey, that’s a good post. I have to stop commenting and save it up!
Regarding students not remembering what was taught a few years (or even a semester) later: I am certainly guilty of this. I would never score as high on my exams now as I did when I was enrolled in the classes. However, I still understand many of the concepts (if not the details of specific examples) and would be able to re-learn the material rather quickly. I’ve been told multiple times that the point of school is more to teach people how to learn than to get them to remember the material. At the same time though, I hope that biology majors, for example, still understand natural selection years after they’ve graduated.
In my experience, the incentives in college are to memorize as much as possible as quickly as possible in order to do well on exams, rather than to learn material in such a way that you’ll remember it. This is definitely a problem when courses have prerequisites and no one remembers anything from them.
I have to say that I always perceived excellence in teaching as more of an evaluation of its effectiveness in broadening or changing mindsets. How do you evaluate such change?