Clinical Prediction Fails Again

I just gave the first Psych 144 exam of the semester. Before I graded them, though, I tried to predict everyone's score based on my experience talking with them, observing them work on activities, and so on. (I did not know how anyone had scored on quizzes or assignments because my TAs grade those.) My intuition told me that my predictions would be pretty good and I was expecting a correlation between my predictions and the actual scores of at least +.50 and probably higher.

The reality, though, was that the correlation was a measly +.17. (I won't show the scatterplot because people might be able to tell from it how I had predicted they would do. But the pattern in the data is barely detectable in it.)

To put things in perspective, I also computed the correlation between the order in which students finished the exam and their scores. It was -.40. (The negative correlation just means that students with lower ranks tended to score higher.) In other words, simply looking at the order in which students finished provided much better predictions than I did using my "professional judgment" based on experience.

[Consider also that this was on a day with horrendous traffic because of the "Get Motivated" event at the Save Mart Center, which I'm sure motivated a lot of complaints to the University Administration. Anyway, this made many students late and affected the order in which they finished. The correlation would probably have been stronger than -.40 otherwise.]

These results are actually typical of tons of research on "clinical prediction" (prediction done based on experience and intuition) and "actuarial prediction" (prediction based on a simple rule usually derived statistically from past data ... but in this case just pulled out of thin air). This research shows that although psychologists, doctors, teachers, and others tend to be quite confident in their clinical predictions, actuarial predictions are always more accurate overall.

One of the best books about psychology (in my opinion, of course) is House of Cards by Robyn Dawes, in which he discusses the problems with clinical prediction and their implications for clinical psychology. I strongly recommend it.

Dawes, by the way, is one of my own intellectual heroes. Unfortunately, he died just recently. (Here is an obituary for him.) Although I had chances to meet him, I never had the nerve to walk up and introduce myself. I definitely wish I had.

Alphabetical Order

Researchers Kurt Carlson and Jacqueline Conard at the Georgetown University School of Business published a very cool article in the Journal of Consumer Research. (By the way, much of the research that goes on in business schools is essentially applied psychology.)

They found that people whose last names are further toward the end of the alphabet respond more quickly to offers of free basketball tickets or an opportunity to complete a survey in return for a bottle of wine. Their theory is that people whose names are further toward the end of the alphabet have spent a lifetime being at the end of the line and waiting their turn, which causes them to jump more quickly at these kinds of opportunities.

Here is a more complete popular summary of this research and the research article itself.

There is really a lot to think about here. But one striking thing is the very respectable strength of the relationship. For example, the correlation between people's last names (A = 1 ... Z = 26) and the time it took them to respond to the basketball ticket offer was -.27. According to Cohen's guidelines, this is a medium strength relationship. And this for a relationship that no one (except for Carlson and Conard, I suppose) would have guessed existed at all.

Anyone who has dabbled in psychological research knows that many variables that seem like they ought to be related turn out not to be (or at only trivial levels). I can't count the number of times students in Psych 144 or Psych 42 have looked at the relationship between the number of units one is enrolled in and one's stress level and found ... bupkis.

So although the research seems well conducted and analyzed, I'm waiting to seeing some more replications--especially ones that use different methods and come from different labs. I love counterintuitive results (probably much more than the next guy), but the more counterintuitive they are, the more confirmation I want.

Stay tuned for more ... I hope.

Psychology, Science, and Life

Sometimes I get the sense that many of my students--especially those in Psych 60T and Psych 144-- think the only thing I care about is science or that I think that science is the only legitimate approach to understanding things.

But this is pretty far from the truth. I'm interested in all sorts of non-scientific things. For example, I love to read novels and play the guitar and banjo. And I do these things not because scientific research has proved them to be beneficial to people's health or happiness, but because I enjoy them. In the case of novels, I believe that the good ones also give me new ways to understand the world, including myself. And playing the banjo gives me a sense of accomplishment and helps define who I am.

I'm also interested in a lot of big philosophical questions, including ones about the nature of knowledge (e.g., What does it mean to know something? How do we know what we know?) and even the limitations of science. I'm extremely interested in ethics. How should we behave? Is it right for me to have as much money, food, and stuff as I have while there are billions of people in the world who have nothing ... and in fact are starving to death? (I feel pretty sure the answer is "no" but I'm not at all sure what to do about it.)

And I certainly didn't choose my spouse according to some computer algorithm or plan my family based on some scientific principle. (Maybe that explains why my two kids are a high school freshman and a kindergartener.)

I suspect that no one would be surprised by a chemistry professor who is "scientific" when doing and teaching chemistry but who has a wide variety of other interests and approaches to life. But it seems that taking a strong scientific approach to psychology implies a severely stunted world view. "If the guy can reduce questions about even love to numbers and graphs, then he must have some kind of a problem."

But here's a news flash. I'm not a literature, music, or philosophy professor. I'm a psychology professor. And psychology, by definition, is the scientific study of human behavior. Who says it's scientific? Well, the American Psychological Association, for starters. And just about every introductory textbook that's out there. The entire faculty in psychology at Fresno State says so, as do the faculty at just about every other university you can think of. Even Wikipedia says so!

So that's what I teach--psychology as an empirical science. And I bring into the classroom all the assumptions and attitudes that go along with that: empiricism, skepticism, and so on.

But I am not trying to convince anyone that science is the only legitimate approach to understanding things--least of all human behavior. What I do want is for my students to understand the scientific approach itself and appreciate it for what it does well: establishing general principles about the way things are. (More on this soon, I hope.)

OK, I've got to go put on my lab coat and safety goggles and then fire up the ol' Bunsen burner. Just kidding, of course. I think I'll play my banjo and listen to the birds sing.

The Learning Curve

What follows is a thinly veiled excuse for posting yet another wav file of myself playing the banjo. This time it's a tune called "Whiskey Before Breakfast."

I've been playing for about three and a half months and have made a lot of progress. In fact, I think I've climbed the steepest part of the famous "learning curve."

Now I'm guessing you've heard of this learning curve ... but I'm also guessing that many of you don't have a good sense of what it is. If you're in Psych 42 or 144 I can explain it using language from class.

The learning curve is the generic statistical relationship between the amount of time one spends studying information or practicing a skill (X, the independent variable) and one's knowledge or skill level (Y, the dependent variable). It's called a "curve" because it's a nonlinear relationship, with skill level increasing quickly at first but then slowing down and eventually leveling off.

For example, in my first week of banjo playing, I went from barely being able to strum a chord in good form to playing simple songs--a huge improvement. In my second week, I went from being able to play simple songs to being able to play somewhat more complex songs. I was on the "steep part" of the curve. But then it probably took two more months to make any really noticeable improvement. I was moving toward the "flat part" of the curve.

So the bad news is that my rate of improvement is slowing down quite dramatically. The good news, though, is that I'll probably continue to improve--even if the increments are very small--until either my hands or my brain start to give out. I don't worry about my eyes. I can play with them closed already.

Banjo Therapy

Well ... the football team lost to Wisconsin.

Immediately afterward, I sat alone in my darkened living room and composed a new tune. It's a bit more sad and reflective than my previous arrangement. I call this one "What Might Have Been."

Fight Varsity!

I've completed and recorded my very first banjo arrangement! It's a song called "Fight Varsity!" You might not recognize the name but see if you recognize the tune.

Fresno State vs. Wisconsin: Why Do I Care?

Like many people here in Fresno, I'm pretty excited about the upcoming football game against the University of Wisconsin. By kickoff, I'll be downright agitated.

In fact, I tend to get so worked up during big games that watching them is not even enjoyable. I'm usually on my feet, pacing, waking in and out of the room, and yelling things like, "Don't field the damn punt inside the ten!"

But, really, why do I care?

I don't know anyone on the Fresno State team, and my family and friends don't really care about the outcome. I do have friends at other universities for whom a Fresno State win would provide me with "bragging rights." But that just begs the question. What would I have to brag about?

So logically, being a diehard fan of a sports team doesn't make much sense. Psycho-logically, however, it's a different story.

Social psychologists have long maintained that we define ourselves in part by our many group memberships. This means that our groups' successes contribute to our own personal self-esteem. In Henri Tajfel's classic social identity theory, people's need for self-esteem leads them to exaggerate the good qualities of the groups they belong to ("ingroups") and diminish those of the groups they don't belong to ("outgroups"). Furthermore, they do this even when the groups are created arbitrarily. (I was able to use this fact a few years ago to show that people's probability judgments are biased in favor of what they want to be true. Here is the abstract of that study.)

So maybe this is why I care. Even though my association with Fresno State is essentially arbitrary (they happened to be hiring psych faculty with my area of expertise at the time I happened to be looking for a tenure-track job), my self-esteem goes up when the football team wins because this reflects favorably on my group ... and it goes down when they lose.

Pathetic, really.

Go 'Dogs!