The Psychological Rambler

Clinical Prediction Fails Again

2011-02-16T13:52:00.001-08:00

I just gave the first Psych 144 exam of the semester. Before I graded them, though, I tried to predict everyone's score based on my experience talking with them, observing them work on activities, and so on. (I did not know how anyone had scored on quizzes or assignments because my TAs grade those.) My intuition told me that my predictions would be pretty good and I was expecting a correlation between my predictions and the actual scores of at least +.50 and probably higher.

The reality, though, was that the correlation was a measly +.17. (I won't show the scatterplot because people might be able to tell from it how I had predicted they would do. But the pattern in the data is barely detectable in it.)

To put things in perspective, I also computed the correlation between the order in which students finished the exam and their scores. It was -.40. (The negative correlation just means that students with lower ranks tended to score higher.) In other words, simply looking at the order in which students finished provided much better predictions than I did using my "professional judgment" based on experience.

[Consider also that this was on a day with horrendous traffic because of the "Get Motivated" event at the Save Mart Center, which I'm sure motivated a lot of complaints to the University Administration. Anyway, this made many students late and affected the order in which they finished. The correlation would probably have been stronger than -.40 otherwise.]

These results are actually typical of tons of research on "clinical prediction" (prediction done based on experience and intuition) and "actuarial prediction" (prediction based on a simple rule usually derived statistically from past data ... but in this case just pulled out of thin air). This research shows that although psychologists, doctors, teachers, and others tend to be quite confident in their clinical predictions, actuarial predictions are always more accurate overall.

One of the best books about psychology (in my opinion, of course) is House of Cards by Robyn Dawes, in which he discusses the problems with clinical prediction and their implications for clinical psychology. I strongly recommend it.

Dawes, by the way, is one of my own intellectual heroes. Unfortunately, he died just recently. (Here is an obituary for him.) Although I had chances to meet him, I never had the nerve to walk up and introduce myself. I definitely wish I had.

Alphabetical Order

2011-02-04T08:31:00.000-08:00

Researchers Kurt Carlson and Jacqueline Conard at the Georgetown University School of Business published a very cool article in the Journal of Consumer Research. (By the way, much of the research that goes on in business schools is essentially applied psychology.)

They found that people whose last names are further toward the end of the alphabet respond more quickly to offers of free basketball tickets or an opportunity to complete a survey in return for a bottle of wine. Their theory is that people whose names are further toward the end of the alphabet have spent a lifetime being at the end of the line and waiting their turn, which causes them to jump more quickly at these kinds of opportunities.

Here is a more complete popular summary of this research and the research article itself.

There is really a lot to think about here. But one striking thing is the very respectable strength of the relationship. For example, the correlation between people's last names (A = 1 ... Z = 26) and the time it took them to respond to the basketball ticket offer was -.27. According to Cohen's guidelines, this is a medium strength relationship. And this for a relationship that no one (except for Carlson and Conard, I suppose) would have guessed existed at all.

Anyone who has dabbled in psychological research knows that many variables that seem like they ought to be related turn out not to be (or at only trivial levels). I can't count the number of times students in Psych 144 or Psych 42 have looked at the relationship between the number of units one is enrolled in and one's stress level and found ... bupkis.

So although the research seems well conducted and analyzed, I'm waiting to seeing some more replications--especially ones that use different methods and come from different labs. I love counterintuitive results (probably much more than the next guy), but the more counterintuitive they are, the more confirmation I want.

Stay tuned for more ... I hope.

Psychology, Science, and Life

2009-02-10T09:25:00.000-08:00

Sometimes I get the sense that many of my students--especially those in Psych 60T and Psych 144-- think the only thing I care about is science or that I think that science is the only legitimate approach to understanding things.

But this is pretty far from the truth. I'm interested in all sorts of non-scientific things. For example, I love to read novels and play the guitar and banjo. And I do these things not because scientific research has proved them to be beneficial to people's health or happiness, but because I enjoy them. In the case of novels, I believe that the good ones also give me new ways to understand the world, including myself. And playing the banjo gives me a sense of accomplishment and helps define who I am.

I'm also interested in a lot of big philosophical questions, including ones about the nature of knowledge (e.g., What does it mean to know something? How do we know what we know?) and even the limitations of science. I'm extremely interested in ethics. How should we behave? Is it right for me to have as much money, food, and stuff as I have while there are billions of people in the world who have nothing ... and in fact are starving to death? (I feel pretty sure the answer is "no" but I'm not at all sure what to do about it.)

And I certainly didn't choose my spouse according to some computer algorithm or plan my family based on some scientific principle. (Maybe that explains why my two kids are a high school freshman and a kindergartener.)

I suspect that no one would be surprised by a chemistry professor who is "scientific" when doing and teaching chemistry but who has a wide variety of other interests and approaches to life. But it seems that taking a strong scientific approach to psychology implies a severely stunted world view. "If the guy can reduce questions about even love to numbers and graphs, then he must have some kind of a problem."

But here's a news flash. I'm not a literature, music, or philosophy professor. I'm a psychology professor. And psychology, by definition, is the scientific study of human behavior. Who says it's scientific? Well, the American Psychological Association, for starters. And just about every introductory textbook that's out there. The entire faculty in psychology at Fresno State says so, as do the faculty at just about every other university you can think of. Even Wikipedia says so!

So that's what I teach--psychology as an empirical science. And I bring into the classroom all the assumptions and attitudes that go along with that: empiricism, skepticism, and so on.

But I am not trying to convince anyone that science is the only legitimate approach to understanding things--least of all human behavior. What I do want is for my students to understand the scientific approach itself and appreciate it for what it does well: establishing general principles about the way things are. (More on this soon, I hope.)

OK, I've got to go put on my lab coat and safety goggles and then fire up the ol' Bunsen burner. Just kidding, of course. I think I'll play my banjo and listen to the birds sing.

The Learning Curve

2008-10-13T19:59:00.000-07:00

What follows is a thinly veiled excuse for posting yet another wav file of myself playing the banjo. This time it's a tune called "Whiskey Before Breakfast."

I've been playing for about three and a half months and have made a lot of progress. In fact, I think I've climbed the steepest part of the famous "learning curve."

Now I'm guessing you've heard of this learning curve ... but I'm also guessing that many of you don't have a good sense of what it is. If you're in Psych 42 or 144 I can explain it using language from class.

The learning curve is the generic statistical relationship between the amount of time one spends studying information or practicing a skill (X, the independent variable) and one's knowledge or skill level (Y, the dependent variable). It's called a "curve" because it's a nonlinear relationship, with skill level increasing quickly at first but then slowing down and eventually leveling off.

For example, in my first week of banjo playing, I went from barely being able to strum a chord in good form to playing simple songs--a huge improvement. In my second week, I went from being able to play simple songs to being able to play somewhat more complex songs. I was on the "steep part" of the curve. But then it probably took two more months to make any really noticeable improvement. I was moving toward the "flat part" of the curve.

So the bad news is that my rate of improvement is slowing down quite dramatically. The good news, though, is that I'll probably continue to improve--even if the increments are very small--until either my hands or my brain start to give out. I don't worry about my eyes. I can play with them closed already.

Banjo Therapy

2008-09-15T08:50:00.000-07:00

Well ... the football team lost to Wisconsin.

Immediately afterward, I sat alone in my darkened living room and composed a new tune. It's a bit more sad and reflective than my previous arrangement. I call this one "What Might Have Been."

Fight Varsity!

2008-09-12T08:48:00.000-07:00

I've completed and recorded my very first banjo arrangement! It's a song called "Fight Varsity!" You might not recognize the name but see if you recognize the tune.

Fresno State vs. Wisconsin: Why Do I Care?

2008-09-11T13:44:00.000-07:00

Like many people here in Fresno, I'm pretty excited about the upcoming football game against the University of Wisconsin. By kickoff, I'll be downright agitated.

In fact, I tend to get so worked up during big games that watching them is not even enjoyable. I'm usually on my feet, pacing, waking in and out of the room, and yelling things like, "Don't field the damn punt inside the ten!"

But, really, why do I care?

I don't know anyone on the Fresno State team, and my family and friends don't really care about the outcome. I do have friends at other universities for whom a Fresno State win would provide me with "bragging rights." But that just begs the question. What would I have to brag about?

So logically, being a diehard fan of a sports team doesn't make much sense. Psycho-logically, however, it's a different story.

Social psychologists have long maintained that we define ourselves in part by our many group memberships. This means that our groups' successes contribute to our own personal self-esteem. In Henri Tajfel's classic social identity theory, people's need for self-esteem leads them to exaggerate the good qualities of the groups they belong to ("ingroups") and diminish those of the groups they don't belong to ("outgroups"). Furthermore, they do this even when the groups are created arbitrarily. (I was able to use this fact a few years ago to show that people's probability judgments are biased in favor of what they want to be true. Here is the abstract of that study.)

So maybe this is why I care. Even though my association with Fresno State is essentially arbitrary (they happened to be hiring psych faculty with my area of expertise at the time I happened to be looking for a tenure-track job), my self-esteem goes up when the football team wins because this reflects favorably on my group ... and it goes down when they lose.

Pathetic, really.

Go 'Dogs!

The Psychology of Clawhammer Banjo

2008-08-19T09:02:00.000-07:00

My big accomplishment of the summer was to start playing the banjo. I've played the guitar for about 15 years but have always wanted to try the banjo. I guess my wife got tired of hearing me talk about it so she got me one for my birthday. Now she's tired of hearing me play it.

Anyway, playing the banjo has got me thinking a lot about the process of learning a new physical skill. For the style I play (called "clawhammer"), the basic right-hand technique involves curling your hand up like a claw and plucking downward at the strings with the back of the fingernail of your index finger, which feels very awkward at first. The banjo is also tuned differently from the guitar, has five strings instead of six, and the fifth string only reaches about three-fourths of the way from the bridge to the peghead. This means that when you look down at your right hand while playing, you see five strings. But when you look up at your left, you see four. This can be confusing when you're looking back and forth between your hands while struggling with "Cluck Old Hen."

Slowly, however, I am getting the hang of it. Moves that seemed next to impossible at first are now fairly easy and some of the simpler songs that I've been learning are starting to sound pretty good. I'm starting to think that I might even be a "real" banjo player someday.

Experiencing and thinking about this reminded me that there is a well established area of psychology that is concerned with just this kind of thing. It is referred to as the study of "motor performance" or perhaps the study of "perception and action." There are laboratories, graduate programs, scientific journals, and textbooks devoted to understanding how we generate, control, and refine voluntary movements to do things like brush our teeth, catch a ball, drive a car, type, and so on.

Yes, a man might play the banjo as an act of passive aggression against his wife (I say might), and everyone would recognize this motivation as something "psychological." But the act itself of playing the banjo is psychological too. And many of us think that it is at least as worthy of study.

Now I've got to get back to working on "Four Wet Pigs."

Mere Exposure and What Everybody Knows

2008-03-20T22:18:00.001-07:00

Students in my Intro to the Psych Major course recently had to read and review a pop psychology book. This is a great assignment, by the way, and I really enjoy reading the papers. If only I didn't have a zillion other things to read: research paper introductions, thesis drafts, articles submitted for publication that I've been asked to review, etc.

Anyway, one student reviewed a book on anger, in which the author made something like the following assertion: "You cannot forgive others until you forgive yourself." In his review, the student was using this critically as an example of something that everyone knows anyway. And of course I've heard this before too and was about to let it pass.

But then I thought, "Why not? Why can't you forgive others until you forgive yourself?"

And I sat there for a few minutes trying to think of any reason that this might be true. Not empirical evidence, mind you. Just any semi-logical, halfway plausible reason for the proposed relationship between other-forgiveness and self-forgiveness. But I couldn't do it. In fact, the more I thought about it, the more meaningless the phrase became.

I suspect that the real reason that this assertion seems obvious--along with the kindred, "You cannot love another until you love yourself"--is just that we've heard them before. There is a phenomenon that research psychologists have termed the "mere exposure effect." Simply having been exposed to a stimulus before (even subliminally) causes that stimulus to be processed more easily, to seem more familiar, to be liked more, and--in the case of assertions--be perceived as more true.

This is making me think of all sorts of other related research-based stuff having to do with the automatic tendency to accept assertions as true and with different ways of processing persuasive messages. But I should probably get back to grading papers.

Evidence Shmevidence

2008-03-16T12:11:00.000-07:00

I just logged in after a long absence and I have 13 comments! I'm famous! I'll go through those soon and get them posted, but in the meantime ...

I was reading an article on herbal remedies in Parade magazine today (the one that comes with the Sunday paper) and saw a section on Ginkgo biloba. Ginkgo is a tree that we have a lot of on the Fresno State campus. It has funny fan shaped leaves that turn bright yellow in the fall. Anyway, the section on ginkgo contains two interesting sentences. The first says, "Evidence suggests that ginkgo biloba has a positive effect on the vascular system ...." The second says, "Some doctors recommend it to boost memory ...."

One way to read this is that 1) ginkgo has a positive effect on the vascular system and 2) ginkgo boosts memory. The different wordings--"evidence suggests" versus "some doctors recommend"--might be just for the sake of variety. But I don't think this is the case. The reason is that while there may be scientific evidence that ginkgo has positive effects on the vascular system, the best current evidence is that it does not boost memory. So although it cannot truthfully be said that "evidence suggests" that it works, it can be said that "some doctors recommend" it. The change in wording shifts responsibility for what is essentially a false statement onto those unnamed doctors, while still generating positive interest in ginkgo.

Although the average person might not even notice the "evidence suggests" versus "some doctors recommend" difference, I maintain that this is exactly the kind of thing that psychology majors should be learning to notice. There is a huge difference between knowledge based on empirical evidence and "knowledge" based on opinion--even expert opinion. This is especially true in psychology where we have lots of scientific knowledge about the diagnosis and treatment of mental disorders, but we also have people (with Ph.D.s, no less) who claim they will hypnotize you and take you back to your past lives to identify the traumatic experiences that are causing your current psychological problems.

"Some psychologists recommend ..." should definitely put you on guard.

Theory Generation

2008-02-21T12:46:00.000-08:00

In Psych 144, I always emphasize the importance of distinguishing phenomena (things that we observe) from theories (explanations for those phenomena). For example, there are sex differences in scores on the math portion of the SAT. No question about it. But there are many different ways to explain this fact that might involve biased tests, a biased society, genetics, and so on.

But this blog has nothing to do with sex differences. What it does have to do with, though, is the fact that no one has commented on any of my blogs. That's the phenomenon of interest for today. And in an exercise straight out of Psych 144, I'm going to generate as many theories to explain this phenomenon as I can. I'll worry about which ones are correct later.

1. Students never look at blog.
2. Students do not know how to post comments.
3. Blogs are too boring to comment on.
4. Blogs are incomprehensible.
5. Students are too dense to think up comments.
6. Settings do not allow comments (because blogger is too dense).
7. Students are afraid to interact with instructor.
8. Students are afraid of appearing to be apple polishers.
9. Students find topics uninteresting.

Other suggestions?

In Praise of Statistical Thinking

2008-02-08T15:41:00.000-08:00

In an essay in the Chronicle of Higher education, Eric G. Wilson (an English professor at Wake Forest) writes "In Praise of Melancholy."

Melancholy, by the way, is just a fancy word for depression. The word itself derives from the ancient Greek meaning "black bile," because it was once thought that depression was caused by an excess of black bile in the body.

Anyway, the author of the essay mentions that recent polls show that 85% of Americans rate themselves as happy. He also claims that this is a disturbing trend because melancholy has been the inspiration for much of the world's great art, music, and literature. Yet we are content to "annihilate" it through positive psychology, psychotherapy, and the use of antidepressants like Prozac.

Although there are a lot of interesting things about this argument, there is a fundamental problem that prevents it from ever getting off the ground ... and it's something that I emphasize over and over in Psych 144. In the abstract it's this: evaluating a claim about a statistical relationship requires a comparison of one variable across levels of the other. In the concrete: the author claims that people are happier than they used to be, but he only presents data on how happy they are now. Who's to say they weren't just as happy in the "olden days?" If they were, then the whole idea that we are in the process of "annihilating" melancholy falls flat.

I'm not an expert on happiness (or what psychologists are more likely to refer to as "subjective well being"), but I'm pretty sure there are data out there that show whether or not there has been a change in the happiness levels of Americans across time. I'm also pretty sure that those data show that there really hasn't been much change over time--certainly not changes on the order of what the Wilson suggests in his essay.

But this is a blog, not a journal article. So you'll have to look up those data yourself.

The Meaning of Imply

2008-02-01T10:28:00.000-08:00

While lecturing the other day on correlational research, I had a thought about the famous dictum, "Correlation does not imply causation." No, I'm not going to try to debunk it. But I am going to suggest that it be reworded because the word imply is ambiguous.

One meaning of imply is something like guarantee. This is the intended meaning in "Correlation does not imply causation." The simple fact that two variables are statistically related does not guarantee that the underlying relationship between them is causal. It could be causal, but this is not a sure thing.

A second meaning of imply--which is probably more common in everyday conversation--is something like hint at. So if my wife casually notes how full the trash can in the garage is getting, some might say that she is implying that I carry it out to the dumpster. (Note that I wouldn't say this ... but some might.) So "Correlation does not imply causation" could be interpreted to mean that the fact that two variables are statistically related does not even hint at there being a causal connection between them.

But this isn't right because it does hint at it. For example, if caffeine consumption is correlated with later miscarriages, then this suggests that researchers ought to take a look at caffeine consumption as a possible cause of miscarriages. It doesn't guarantee that caffeine consumption is a cause ... but it could be.

So let me propose a rewording of the old dictum: "Correlation does not guarantee causation."

You CAN Do Anything with a Bachelors in Psych

2008-01-27T20:26:00.000-08:00

"You can't do anything with a bachelors degree in psych."

In addition to its being plain incorrect, this claim bothers me for several reasons.

The first is that, according to Tara Kuther in The Psychology Major's Handbook, the majority of psychology majors do not go to graduate school ... ever. But of course they must be doing something.

The second is that it implies that there is something unique about psych majors in terms of their ability to get jobs with only their bachelors degrees ... as though sociology, philosophy, and chemistry majors are being handed jobs left and right. Yes, the technical knowledge obtained by chemistry majors gives them the upper hand for getting certain kinds of jobs, but the technical knowledge obtained by psychology majors (e.g., behavioral principles, statistical analysis) gives them the upper hand for others.

The third reason this claim bothers me is that it reflects a misunderstanding of the traditional purpose of an American-style college education, which is not to be trained for a particular career. Yes there are some exceptions; nursing and engineering come to mind. But a bachelor's degree in psychology is not supposed to turn you into a psychologist any more than a bachelor's degree in biology is supposed to turn you into a physician. What both of those bachelor's degrees are supposed to do is turn you into an educated human being who knows a little about a wide variety of fields, a little more about one specific field, and has lots of important general skills like the ability to write and speak clearly, analyze complex problems, motivate yourself, and work effectively with others.

The fact is that a bachelor's degree in psychology can be one step on the way to doing just about anything--including becoming a psychologist, a doctor, a lawyer, a newspaper reporter, an advertising executive, ....

Jumping to Causal Conclusions

2008-01-25T08:02:00.000-08:00

There's been a rash of studies in the news purporting to show that some medication, food, or behavior (call it X) has an effect on some aspect of health (call it Y). In Psych 42, I've talked recently about new studies on caffeine and miscarriage and anger suppression and mortality (i.e., death). The latest study concerns birth control pills and ovarian cancer.

All of these studies do clearly show an association between X and Y. However, because none of them is an experiment--where the independent variable is manipulated and extraneous variables are controlled--they do not show that X causes Y. And it doesn't matter how many times the journalists or researchers state that they do.

In the latest example, the researchers showed that women who took the Pill were less likely than women who did not take the pill to develop ovarian cancer. This could be because they took the Pill ... but it is also possible that women who took the Pill differed from women who did not in some other way. For example, women who took the Pill might also have been women who tended to think about and act on their health and well-being more than women who did not--and this difference might have been responsible for the reduction in ovarian cancer.

Yet the article linked above is titled "Pill Prevents Ovarian Cancer for Decades." The word "prevents" clearly implies causation. But if the risk reduction the researchers found was due to something other than the Pill, then then it is not the Pill that is doing the "preventing" and going on the Pill will not reduce anyone's risk.

There are other issues here--such as the effectiveness of statistically controlling for confounding variables--but I'll cover those in another blog. For now, let me give my standard disclaimer about this sort of thing. I am not saying there is anything wrong with this research. It is interesting and important and tells us that there is an association between X and Y that might be a causal one. But researchers, journalists, and the general public need to be much more cautious about jumping to causal conclusions.

From "Grand Theft Auto" to Inferential Statistics

2008-01-21T00:02:00.000-08:00

A Research Methods student from last semester, Jordan Prendez, sent me the following link: http://www.gamespot.com/news/6147420.html. It is a short article describing a research study showing that players of a more violent video game are more prone to violence themselves than players of a less violent video game.

First, let me point out that the study, as described, is a good example of a two-group randomized experiment with multiple dependent variables. It is an experiment because it has a manipulated independent variable and the researchers appear to have tried to control other variables.

Second--and this was Jordan's point--it is interesting how the gamers who commented on the article attacked the research (presumably without reading the original study). This is reminiscent of a famous study by Lord, Lepper, and Ross (1979), showing that people are good at finding the flaws in a research study ... but only when they already disagree with the study's conclusions.

Statistics students, in particular, should note the argument that appears in a couple of the comments. In essence, "There's no way that a sample of 100 [the number of participants in the study] can represent the whole population of gamers." Intuitively, this seems right because 100 is a very small fraction of the total number of gamers out there. If there were 10 million gamers in the population, for example, this sample would only represent 1/100,000th of that population. But this intuition is wrong. A sample of 100 will generally be much more similar to the population--even a very large population--than most people realize. And this is what the field of inferential statistics is all about.

Thanks, Jordan!

Why Indeed?

2008-01-20T23:27:00.000-08:00

I guess this blog started out as a test. I wanted to have students in my Intro to the Psych Major course (Psych 60T) create and maintain blogs for short, informal, reflective writing assignments. So I had to create one myself to see how it worked. Then I had to try posting to it. Then I realized it would be a good forum for communicating with my students, especially about ideas that in class would be considered "digressions." (I'm a big digressor.) I suspect, however, that these ideas are probably more interesting and useful than the "course material" in many cases. I also realized that I could do this without cluttering up the Announcements space on the course Blackboard site ... and I could make these ideas available to students in all of my classes ... and rest of the world too. Welcome to the 21st century, Dr. Price.

First Post

2008-01-14T23:15:00.000-08:00

It's late and classes start tomorrow for the Spring 2008 semester at Fresno State. I'm not nearly ready but I'm killing valuable time by writing a blog. Why?