I’ll bet some of you regular Doc Gurley readers already guessed that we’d be handing out a BOGUS Award to that widely-publicized example of deeply flawed research–the study stating that you can tell if someone’s into love or lust, purely from their facial features. Slut or not? – just look at the width of her eyes. Yes siree, folks, this study joins those august embarrassing fields of pseudo-science – phrenology and eugenics – where claims were made in earlier centuries about what “data” you could get from face/head features. Bizarre indicators, from the bumps on your noggin, to racist descriptions about the width of your nostrils, were claimed to be clear indicators of such “obvious” things as intelligence and stamina.
But maybe you’re asking yourself, how could a study be that bad if even Scientific American covered it?
Think critically to yourself (something health reporters nationwide failed to do) while I describe the study design: Get a bunch of early-twenty-somethings. Ask them to describe their sexual attitudes. In other words, lean forward and ask: so tell me, are you a slut? Or looking for true love? Take a picture of them (after their self-description). Then show that picture to other early-twenty-somethings and have them guess. Slut or not?
What was “success”? Getting the self-description right more than 50% of the time (including people who guessed right 51% of the time).
Hmm. Anyone seeing a problem here?
First, the math. When the numbers involved are this small, generalizing from statistics doesn’t work well. One research subject might, just from chance alone, get 60 right and 40 wrong. None of it would have anything to do with an ability to predict the answer. A study’s chances of getting a skewed result go way up with a smaller number of people, purely, again, from chance. Their first study sample used 153 people–NOT many. But the biggest problem here is their definition of success. Calling more than 50% right a “success” means you’re not doing much better than a coin toss. Let’s take away the sex distraction and see how it looks. What if, instead of faces + sluttiness, we instead were asking if someone could predict heads or tails on a coin toss? A research subject then could get 49 wrong, and 51 right out of 100 tosses, and be declared a psychic. No one needs a lot of fancy power calculations or advanced courses in epi to realize this sucker of a study fails the common sense test (what Doc Gurley likes to call the Grandma Check). Take a minute and ask yourself, as an honorary skeptical Grandma – do you think getting 51 right and 49 wrong is good enough to say that someone can verifiably predict sluttiness? Nah, didn’t think so. In mathematical summary, when you combine the small study numbers, and the stretched-past-the-breaking-point definition of “success,” and remove the intentional distraction of sex, what you end up with is serious bogosity: (teeny Number + inflated Outcome – gimmick Sex = BOGUS) or, as a reduced equation, ( N+O – Sex = BOGUS).
Second problem, the study design. Ignoring the issues of math, how many of you think that if you ask a 20-something, hey, are you a player? – and they say definitely (then you snap their picture), could you then tell what they wanted to convey to the world at large? My theory is that if you really want to know if a static face can tell you someone’s sexual tendencies, take a life history from 90-year-olds, then, months or years later, go to the funeral home and snap some pics. Exactly why is it the researchers didn’t take pictures of these subjects, scrubbed of make-up and wearing a swimcap, while the self-described lust-hound was, say, having their blood drawn? That would be a more accurate assessment of whether the width of someone’s eyes (or the manliness of their clenched jaw) could predict sexual tendencies.
The journal that published this trash study is called Evolution in Human Behavior. Seems like, despite the title, no one there has evolved from similar cringe-worthy studies of the 1800’s.
But here’s the most humiliating thing (for all scientists everywhere) – this study was published the exact same week that the incredible Mary Roach launched her latest wonderful, funny book, an expose’ of the idiotic nature of “scientific” sex research, titled Bonk. All I can say is, must we give her more material, right as her book hits the shelves? Couldn’t we collectively, as scientists, wait at least a week or two?
Sigh. Therefore, we award this pseudo-science sex study a Doc Gurley (Bonk? – Oh Groan, it’s Us, Still) BOGUS award.