A Rival Blog

There's another blog on roughly the same topic but somewhat more active.

Four Small Studies

According to four studies (sample sizes 85, 38, 36, and 34), people give more allegedly right-wing answers to survey questions when they are less able to think. The only one that even came close to a decent size was also the one most subject to selection effects. (It found that drunker people in bars gave more right-wing answers. The lesson we might learn is that high-IQ conservatives don't go to bars.)

Maybe they should come back when they have real data.

Is There a Slant to Small Samples?

Maybe I should start checking to see if studies with a sample size of 17 are likely to bolster ridiculous feminist cliches. I suspect I can find at least 17 of them.

Are Drivers of Expensive Cars @ssholes?

On the one hand, it seems intuitively obvious and there's even some empirical data to back up this conclusion. On the other hand, the data for car classes 2–5 are well within random variation, which means the conclusion is ultimately drawn from a sample of the five cars in class 1. (I also suspect that the lines between the classifications were drawn in order to derive a predetermined conclusion.)

Besides, the intuitively-obvious result would have class 5 be the outlier instead of class 1.

20 Subjects vs. 428 Subjects

Wired is reporting that a study with 428 subjects indicated that being sleepy might aid creativity and that a study with 20 subjects (I checked the ScienceNews report) purportedly indicated that being drunk can aid creativity. I have a question: Why is a study with 20 subjects reported as though it were as sound as a study with 428 subjects? It looks like there is some actual evidence that being sleepy can aid creativity. The claim that being drunk can aid creativity is still as anecdotal as ever.

An Unforgettable Small Sample

The most unforgettable scientific lecture ever was based on a sample size of 1. Professor G. S. Brindley tested an experimental treatment for erectile dysfunction on himself and, to prove it worked, dropped his pants during a presentation of the paper at a scientific conference. After that it got weird…

More Brains!

Studying to be a London taxi driver can increase the brain size of at least 39 people.

A few years ago, I would have taken a small study like this more seriously. Now it is, at best, an indication that a real study might be worthwhile.

This Would Be Creepier …

… if it were less dubious.

According to a study with 10 subjects (seen via io9)

"The most surprising thing in this study is that mere inductions of neural activation patterns...led to visual performance improvement...without presenting the feature or subjects' awareness of what was to be learned," said lead researcher Takeo Watanabe. He continues:

We found that subjects were not aware of what was to be learned while behavioral data obtained before and after the neurofeedback training showed that subjects' visual performance improved specifically for the target orientation, which was used in the neurofeedback training.

Is this research mind-blowing and exciting? Absolutely. I mean come on — automated learning? Yes. Sign me up. But according to research co-author Mitsuo Kawato, the neurofeedback mechanism could just as soon be used for purposes of hypnosis or covert mind control. And that... I'm not so keen on.

"We have to be careful," he explains, "so that this method is not used in an unethical way."

I think taking a study with a sample size of 10 seriously is unethical enough.

Besides, earlier mind-control research went nowhere … or at least that's what THEY want us to think …

Don't Tell the Tobacco Companies

According to a study with 36 subjects, carbon monoxide makes people better able to tolerate noise pollution.

On the other hand, maybe it makes them deaf.

They're Starting to Take Care of It

I'm sure many of my fellow nerds have seen an article purporting to claim that observers can see evidence of a possible ”love gene” in brief video clips (original article here). This far-reaching conclusion was based on a sample size of 23. The good news is that experts are starting to react:

But the paper has drawn harsh criticism from geneticists because of its small sample size. Daniel MacArthur, who blogs at Genetic Future, said, “[A simple size of] 23 for genetics means the paper might as well not exist. It carries no useful info. Without a larger sample and independent replication, it’s safest to simply assume these results are false.” Joe Pickrell from Harvard Medical School, agreed: “If the sample size is 23… there’s no way that’s a real association.”And Chris Gunter from the HudsonAlpha Institute for Biotechnology added, “The literature is full of behavioural genetics studies which don’t replicate, with similarly small numbers.”

Thin Evidence about Thin People Believing in Thin Ectoplasm

There is some evidence that skinny people are afraid of ghosts:

There are indications that subjects with schizotypal personality have a lower Body Mass Index. Also schizotypal personality is linked to a higher incidence of paranormal belief. In this study we examined whether low Body Mass Index is also linked to paranormal belief. In a pilot study 48 students of psychology (85.4% women) between the ages of 20 and 27 years were administered a questionnaire assessing weight, height, and paranormal belief. Analysis suggested an association between belief in paranormal phenomena and low Body Mass Index.

On the other hand, the evidence disappeared once they did a real study:

In a follow-up study with 300 subjects and equal sex distribution, the relationship was examined under control of schizotypy. The results for Body Mass Index could not be confirmed; however, paranormal belief was heavily associated with the cognitive-perceptual component of schizotypy.

Aren't ghosts also supposed to vanish in the light?

It's Been Done

I'm not the first to point out the absurdly-small sample sizes of social-science research. Tversky and Kahneman (seen via The Wall Street Journal) beat me to it by several decades.

On the other hand, their sample size was only 84. On the gripping hand, that might be a dividing line between social-science research that should be taken seriously and social-science research that should be regarded as tentative.

One more point: My usual reaction to reports of research on cognitive bias is to wonder why we don't see more research into the cognitive biases of psychologists. It turns out they've done such research, at least as far as the cognitive bias of believing in small samples is concerned. The problem is that it isn't as widely reported as first-order cognitive-bias research.

Has there been any research done on the cognitive biases of science reporters? (I wouldn't be surprised to find there has been such research and the reporters don't mention it … for obvious reasons.)

Life, the Universe, and Everything

The great question of Life, the Universe, and Everything turns out to be “What was the sample size for the paper ‘Subcortical processing of speech regularities underlies reading and music aptitude in children’?”

Digression: The problem with small-sample research isn't that it is necessarily invalid. A small-sample paper might be about something real. One problem is that if it were about a real phenomenon we should expect it to be backed up by large samples and frequently it isn't. Another problem is that it is usually reported as something definitive and small-sample research is always tentative.

Do Neuroscientists Have This Fold?

According to an article in Discover, there's a fold in the brain that helps keep reality and imagination separate. After seeing the sample size involved:

Of the 53 people selected for the study, some had this fold on both sides of their brain, some had it on one side, and some had no fold.

I wondered what part of the brain is responsible for taking studies with small sample sizes seriously.

I Reject Information That's Based on Small Samples

According to scientists studying optimism:

In a study published October 9 in Nature Neuroscience, researchers at the Wellcome Trust Centre for Neuroimaging at UCL (University College London) show that people who are very optimistic about the outcome of events tend to learn only from information that reinforces their rose-tinted view of the world. This is related to 'faulty' function of their frontal lobes.

………

Nineteen volunteers were presented with a series of negative life events, such as car theft or Parkinson's disease, whilst lying in a functional magnetic resonance imaging (fMRI) scanner, which measures activity in the brain. They were asked to estimate the probability that this event would happen to them in the future. After a short pause, the volunteers were told the average probability of this event to occur. In total, the participants saw eighty such events.

This was mentioned in an article cited by John Derbyshire.

We Know Free Will Doesn't Exist …

… because of experiments on five subjects:

The recordable cerebral activity (readiness-potential, RP) that precedes a freely voluntary, fully endogenous motor act was directly compared with the reportable time (W) for appearance of the subjective experience of 'wanting' or intending to act. The onset of cerebral activity clearly preceded by at least several hundred milliseconds the reported time of conscious intention to act. This relationship held even for those series (with 'type II' RPs) in which subjects reported that all of the 40 self-initiated movements in the series appeared 'spontaneously' and capriciously. Data were obtained in at least 6 different experimental sessions with each of 5 subjects.

There's better data, though. Another study had 14 subjects:

Haynes and his colleagues imaged the brains of 14 volunteers while they performed a decision-making task. The volunteers were asked to press one of two buttons when they felt the urge to. Each button was operated by a different hand. At the same time, a stream of letters were presented on a screen at half-second intervals, and the volunteers had to remember which letter was showing when they decided to press their button.

I think this was the study I discussed here.

Transcranial Magnetic Stimulation

After noticing that two of the articles mentioned were about Transcranial Magnetic Stimulation, I started getting suspicious. So looked at a few more on Pubmed.

According to Excitatory repetitive transcranial magnetic stimulation induces improvements in chronic post-stroke aphasia.:

We enrolled 8 patients with moderate or severe aphasia >1 year after LMCA stroke.

Sample size 8.

According to Accelerated repetitive transcranial magnetic stimulation (aTMS) for treatment-resistant depression

Patients included 9 men, 5 women with a median age of 51 years (range 20–74 years).

Sample size 14.

According to Revisiting the Backward Masking Deficit in Schizophrenia: Individual Differences in Performance and Modeling With Transcranial Magnetic Stimulation:

Seventeen patients (5 female) were recruited from the New York State Psychiatric Institute (NYSPI) Schizophrenia Research Unit.

Sample size 17.

According to Ten sessions of adjunctive left prefrontal rTMS significantly reduces fibromyalgia pain: A randomized, controlled pilot study.:

We recruited 20 patients with fibromyalgia, defined by American College of Rheumatology criteria, and randomized them to receive 4000 pulses at 10Hz TMS (n=10), or sham TMS (n=10) treatment for 10 sessions over 2weeks along with their standard medications, which were fixed and stable for at least 4weeks before starting sessions.

Sample size 20.

According to Worry Facilitates Corticospinal Motor Response to Transcranial Magnetic Stimulation

The 21 participants were self-identified right-handed males who ranged in age from 18 to 38 years (M=20.33, SD=4.22).

Sample size 21.

According to A bi-directional assessment of the human brain-anorectal axis.

Cortical evoked potentials was assessed following electrical stimulations of anus and rectum with bipolar electrodes in 26 healthy subjects.

Sample size 26.

Finally, according to Noninvasive brain stimulation with high-frequency and low-intensity repetitive transcranial magnetic stimulation treatment for posttraumatic stress disorder.:

In this double-blind, placebo-controlled phase II trial conducted between October 2005 and July 2008, 30 patients with DSM-IV-diagnosed PTSD were randomly assigned to receive 1 of the following treatments: active 20 Hz rTMS of the right DLPFC, active 20 Hz rTMS of the left DLPFC, or sham rTMS.

Sample size 30.

I've looked at nine of these papers. By small-sample standards, I could publish something. By the way, are there any large studies on Transcranial Magnetic Stimulation?

Analgesia for Freshmen

According to NCBI ROFL (I don't think they take this seriously either):

Two studies assessed whether playing video games would significantly distract participants from painful stimulation via a cold pressor test. In Study 1, participants (8 men, 22 women, M age = 18.5 yr., SD = 1.3) in an action-oriented game condition tolerated pain for a longer time period and reported lower pain intensity ratings than those in a nonaction-oriented game or a nongame control condition. No differences were found on scores of aggressiveness, competitiveness, or prior video game experience, suggesting that these factors play little role. In Study 2, participants (14 men, 13 women, M age = 19.7 yr., SD = 1.3) engaged in six video game conditions (action, fighting, puzzle, sports, arcade, and boxing) and a nongame control condition. Video game play produced an increase in pulse, which was greatest during the action, fighting, sports, and boxing games. Pain tolerance was greatest during the sports and fighting games. Thus, certain games produce greater distraction, which may have implications for the medical field as an adjunct to pain management.

Maybe this only applies if you're also given extra credit for a psychology class.

More Mesmerism

Magnets are again being used in a small sample study (seen via futurisms):

In the study, the researchers asked 49 female volunteers to take part in a study where they were asked to rate the attractiveness of 220 photographs of female faces, but they were allowed to change their ratings after seeing what others in the study had scored.

When Transcranial Electromagnetic Stimulation (TMS) was used to inhibit the activity of the neurons in the posterior medial frontal cortex, the participants did not change their ratings of the photographs so they were more in line with the rest of the group.

Even if this does pan out, I doubt if it will have much of an effect on society. After all, the people most likely to want to use this are the people who need it the least.

Didn't Mesmer Pioneer This?

According to a recent study:

Here, we show that when subjects can name the colour of presented objects correctly or incorrectly at their free will, the tendency to stick to truthful answers can be manipulated by stimulation targeted at dorsolateral prefrontal cortex.

According to the Guardian's summary:

Inga Karton and Talis Bachmann of the University of Tartu adopted a different and novel approach, by examining the natural propensity to lie spontaneously during situations in which deception has no consequences. They recruited 16 volunteers, and showed them red and blue discs, which were presented randomly on a computer screen. The participants were asked to name the colour of each disc, and that they could do so correctly or incorrectly at their free will.

How do we know the experimenters weren't influenced by magnets?