Twenty Celsius

I started breathing 68 years ago.

Is This a Record Sample Size for a Transcranial Magnetic Stimulation Study?

A recent study of Transcranial Magnetic Stimulation had 80 participants.

Wow.

Wordy People with a Large Vocabulary Are Telling the Truth!

According to recent research on how to get terrorists to talk (seen via BuzzFeed (seen via GeekPress)):

We hypothesized that deceptive participants would speak less and use fewer unique words than would truthful participants when interviewed about their activities.

They were able to confirm their hypothesis using a WEIRD sample of 64 people. You can be sure they were telling the truth because they used very many long words to explain it. (That's why I used the hypothetical statement above; it was much shorter than the later summaries.)

Cross posted to my main blog.

\(\sf\LaTeX\) vs. Word

Who's doing the half-time show?

According to research by psychologists, a controlled experiment of \(\sf\LaTeX\) vs. Word found that \(\sf\LaTeX\) users were slower and made more mistakes. Needless to say, they had pathetically-small sample sizes (10 in each group). This might be related to their goals, since they apparently wanted research published as fast as possible and the time needed to gather and test an adequate sample would interfere with that. (In other words, this is the scholarly equivalent of an all-nighter.)

I have more comments on my main blog.

Are These Large Samples?

According to Transcranial magnetic stimulation: a new tool in the fight against depression:

During the year 2000, three relatively large studies (Grunhaus et al,³⁸ George et al,³⁷ and Pridmore et al⁴²) have reported significant antidepressant effects for rTMS administered over the LDLPFC. George et al conducted a parallel, double-masked, sham-controlled study of rTMS over the LDLPFC in patients with nondelusional MDD.³⁷

…

37. George MS., Nahas Z., Molloy M., et al. A controlled trial of daily prefrontal cortex TMS for treating depression. Biol Psychiatry. 2000;48:962–970. [PubMed]

38. Grunhaus L., Dannon P., Schreiber S., et al. Repetitive transcranial magnetic stimulation is as effective as electroconvulsive therapy in the treatment of nondelusional major depressive disorder: an open study. Biol Psychiatry. 2000;47:314–324. [PubMed]

…

42. Pridmore S., Raimondo B., Turnier-Shea Y., Reid P., Rybak M. Comparison of unlimited numbers of rapid transcranial magnetic stimulation (rTMS) and ECT treatment sessions in major depressive episode. Intj Neuropsychopharmacol. 2000;3:129–134. [PubMed]

When we look at the actual studies, we see in Ref. 37:

Thirty medication-free adult outpatients with nonpsychotic, major depressive (n = 21) or bipolar (n = 9) (depressed phase) disorder who were in a current major depression (Hamilton Rating Scale for Depression [HRSD] 21-item score of >18) were treated each weekday for 2 weeks. Subjects were randomly assigned to receive either daily active (20 subjects) or sham (10 subjects) stimulation. Additionally, the 20 active subjects were equally divided between slower (5 Hz) and faster (20 Hz) frequency treatment. Antidepressant response was defined as greater than a 50% improvement in the baseline HRSD.

and in Ref. 38:

Forty patients with MDD referred for ECT were randomly assigned to either ECT or rTMS.

and in Ref. 42:

We compared the antidepressant effects of rTMS and ECT in 32 patients suffering major depressive episode (MDE) who had failed to respond to at least one course of medication.

Neurobiologists have a different definition of large. To quote Sheldon Cooper, PhD:

Notify the editors of the Oxford English Dictionary: the word "plenty" has been redefined to mean "two".

and:

Oh, great. Now I’m worse than a fraud. I’m practically a biologist.

In Which Direction?

According to a psychological study, students (in a sample of 93) were classified as high working-memory or low working-memory. They were then given a math test and the HWM students did better. After that they were given a test in what was termed a “higher, real world pressure situation” and the HWM scores dropped to the LWM levels. The conclusion:

Since working memory is known to predict many higher-level brain functions, the research calls into question the ability of high-pressure tests such as the SAT, GRE, LSAT, and MCAT to accurately gauge who will succeed in future academic endeavors.

In which direction? Are they claiming that those tests measure things other than working memory and are thus unfair or are they claiming that they only test working memory and are thus unrealistic? Or is this an attempt by reporters to justify flunking algebra?

Notable and Quotable

There's a political activist who's more numerate than many social scientists. Technology Review reports:

“You’re making significant resource decisions based on 160 people?” asks Mitch Stewart, director of the Democratic campaign group Organizing for America. “Isn’t that nuts? And people have been doing that for decades!”

Sample Sizes 41 and 67

According to a recent study:

Researchers from the University of California quizzed 41 women in long-term relationships about how attractive they found their husbands or boyfriends.

And, twice over the course of a month, they were asked how committed they were to their relationship and how close they felt to their partner.

In a second, similar experiment, 67 women were questioned twice about how likely they were to find fault with their other half.

The results showed that the women’s attitudes changed depending on how fertile they were and how attractive their man was.

A propensity to rush into print based on minimal evidence in the field of gender relations is not limited to feminists.

Rosin v. Hymowitz

Hanna Rosin's article on the “hookup culture” and how it empowers women disagees with Kay S. Hymowitz, who found that the least empowered women were least likely to marry. The only part of the study that came close to looking at control groups had a sample size of 53:

The researchers found their first day of interviewing so enlightening that they decided to ask the administration if they could stay on campus for four years and track the 53 women’s romantic lives.

I won't more than mention that the study might have indicated the opposite of Ms. Rosin's claim.

At Long Last …

There's some actual data to support one of Freud's theories, based on a study with a sample size of 11:

Researchers took 11 people with anxiety disorders who all received psychoanalysis as part of the study, to see what underlying unconscious conflict might be causing the anxiety.

Words capturing the nature of the unconscious conflict were then selected from the interviews, as well as words that related to each patient's experience of anxiety disorder symptoms.

A control category with words that had no relationship to the unconscious conflict or anxiety symptoms were presented to three patients wearing scalp electrodes to measure brain responses.

Researchers found a new measurement of the brain's alpha wave frequency, linked to communication, was made when the unconscious conflict words were presented directly before the conscious words.

Results showed highly significant correlations, suggesting an inhibitory effect, were obtained when the amount of alpha generated by the unconscious words were grouped with the conscious words.

Wow. Is that the best they can do?

The Conclusion Makes Sense but …

According to a recent study, organic food can turn people into obnoxious snobs:

To find out, Eskine and his team divided 60 people into three groups. One group was shown pictures of clearly labeled organic food, like apples and spinach. Another group was shown comfort foods such as brownies and cookies. And a third group -- the controls -- were shown non-organic, non-comfort foods like rice, mustard and oatmeal. After viewing the pictures, each person was then asked to read a series of vignettes describing moral transgressions.

………

"We found that the organic people judged much harder compared to the control or comfort food groups," says Eskine. "On a scale of 1 to 7, the organic people were like 5.5 while the controls were about a 5 and the comfort food people were like a 4.89."

When it came to helping out a needy stranger, the organic people also proved to be more selfish, volunteering only 13 minutes as compared to 19 minutes (for controls) and 24 minutes (for comfort food folks).

I don't think a study with a sample size of 60 should be taken seriously even if its conclusion (that “organic people” act like jerks) is plausible.

I won't more than mention that it was the sight of the organic food that had the effect, which might have been due to us anti-environmentalists getting p___ed off.

A Rival Blog

There's another blog on roughly the same topic but somewhat more active.

Four Small Studies

According to four studies (sample sizes 85, 38, 36, and 34), people give more allegedly right-wing answers to survey questions when they are less able to think. The only one that even came close to a decent size was also the one most subject to selection effects. (It found that drunker people in bars gave more right-wing answers. The lesson we might learn is that high-IQ conservatives don't go to bars.)

Maybe they should come back when they have real data.

Is There a Slant to Small Samples?

Maybe I should start checking to see if studies with a sample size of 17 are likely to bolster ridiculous feminist cliches. I suspect I can find at least 17 of them.

Are Drivers of Expensive Cars @ssholes?

On the one hand, it seems intuitively obvious and there's even some empirical data to back up this conclusion. On the other hand, the data for car classes 2–5 are well within random variation, which means the conclusion is ultimately drawn from a sample of the five cars in class 1. (I also suspect that the lines between the classifications were drawn in order to derive a predetermined conclusion.)

Besides, the intuitively-obvious result would have class 5 be the outlier instead of class 1.

20 Subjects vs. 428 Subjects

Wired is reporting that a study with 428 subjects indicated that being sleepy might aid creativity and that a study with 20 subjects (I checked the ScienceNews report) purportedly indicated that being drunk can aid creativity. I have a question: Why is a study with 20 subjects reported as though it were as sound as a study with 428 subjects? It looks like there is some actual evidence that being sleepy can aid creativity. The claim that being drunk can aid creativity is still as anecdotal as ever.

An Unforgettable Small Sample

The most unforgettable scientific lecture ever was based on a sample size of 1. Professor G. S. Brindley tested an experimental treatment for erectile dysfunction on himself and, to prove it worked, dropped his pants during a presentation of the paper at a scientific conference. After that it got weird…

More Brains!

Studying to be a London taxi driver can increase the brain size of at least 39 people.

A few years ago, I would have taken a small study like this more seriously. Now it is, at best, an indication that a real study might be worthwhile.

This Would Be Creepier …

… if it were less dubious.

According to a study with 10 subjects (seen via io9)

"The most surprising thing in this study is that mere inductions of neural activation patterns...led to visual performance improvement...without presenting the feature or subjects' awareness of what was to be learned," said lead researcher Takeo Watanabe. He continues:

We found that subjects were not aware of what was to be learned while behavioral data obtained before and after the neurofeedback training showed that subjects' visual performance improved specifically for the target orientation, which was used in the neurofeedback training.

Is this research mind-blowing and exciting? Absolutely. I mean come on — automated learning? Yes. Sign me up. But according to research co-author Mitsuo Kawato, the neurofeedback mechanism could just as soon be used for purposes of hypnosis or covert mind control. And that... I'm not so keen on.

"We have to be careful," he explains, "so that this method is not used in an unethical way."

I think taking a study with a sample size of 10 seriously is unethical enough.

Besides, earlier mind-control research went nowhere … or at least that's what THEY want us to think …

Don't Tell the Tobacco Companies

According to a study with 36 subjects, carbon monoxide makes people better able to tolerate noise pollution.

On the other hand, maybe it makes them deaf.

They're Starting to Take Care of It

I'm sure many of my fellow nerds have seen an article purporting to claim that observers can see evidence of a possible ”love gene” in brief video clips (original article here). This far-reaching conclusion was based on a sample size of 23. The good news is that experts are starting to react:

But the paper has drawn harsh criticism from geneticists because of its small sample size. Daniel MacArthur, who blogs at Genetic Future, said, “[A simple size of] 23 for genetics means the paper might as well not exist. It carries no useful info. Without a larger sample and independent replication, it’s safest to simply assume these results are false.” Joe Pickrell from Harvard Medical School, agreed: “If the sample size is 23… there’s no way that’s a real association.”And Chris Gunter from the HudsonAlpha Institute for Biotechnology added, “The literature is full of behavioural genetics studies which don’t replicate, with similarly small numbers.”

Thin Evidence about Thin People Believing in Thin Ectoplasm

There is some evidence that skinny people are afraid of ghosts:

There are indications that subjects with schizotypal personality have a lower Body Mass Index. Also schizotypal personality is linked to a higher incidence of paranormal belief. In this study we examined whether low Body Mass Index is also linked to paranormal belief. In a pilot study 48 students of psychology (85.4% women) between the ages of 20 and 27 years were administered a questionnaire assessing weight, height, and paranormal belief. Analysis suggested an association between belief in paranormal phenomena and low Body Mass Index.

On the other hand, the evidence disappeared once they did a real study:

In a follow-up study with 300 subjects and equal sex distribution, the relationship was examined under control of schizotypy. The results for Body Mass Index could not be confirmed; however, paranormal belief was heavily associated with the cognitive-perceptual component of schizotypy.

Aren't ghosts also supposed to vanish in the light?

It's Been Done

I'm not the first to point out the absurdly-small sample sizes of social-science research. Tversky and Kahneman (seen via The Wall Street Journal) beat me to it by several decades.

On the other hand, their sample size was only 84. On the gripping hand, that might be a dividing line between social-science research that should be taken seriously and social-science research that should be regarded as tentative.

One more point: My usual reaction to reports of research on cognitive bias is to wonder why we don't see more research into the cognitive biases of psychologists. It turns out they've done such research, at least as far as the cognitive bias of believing in small samples is concerned. The problem is that it isn't as widely reported as first-order cognitive-bias research.

Has there been any research done on the cognitive biases of science reporters? (I wouldn't be surprised to find there has been such research and the reporters don't mention it … for obvious reasons.)

Life, the Universe, and Everything

The great question of Life, the Universe, and Everything turns out to be “What was the sample size for the paper ‘Subcortical processing of speech regularities underlies reading and music aptitude in children’?”

Digression: The problem with small-sample research isn't that it is necessarily invalid. A small-sample paper might be about something real. One problem is that if it were about a real phenomenon we should expect it to be backed up by large samples and frequently it isn't. Another problem is that it is usually reported as something definitive and small-sample research is always tentative.

Do Neuroscientists Have This Fold?

According to an article in Discover, there's a fold in the brain that helps keep reality and imagination separate. After seeing the sample size involved:

Of the 53 people selected for the study, some had this fold on both sides of their brain, some had it on one side, and some had no fold.

I wondered what part of the brain is responsible for taking studies with small sample sizes seriously.

I Reject Information That's Based on Small Samples

According to scientists studying optimism:

In a study published October 9 in Nature Neuroscience, researchers at the Wellcome Trust Centre for Neuroimaging at UCL (University College London) show that people who are very optimistic about the outcome of events tend to learn only from information that reinforces their rose-tinted view of the world. This is related to 'faulty' function of their frontal lobes.

………

Nineteen volunteers were presented with a series of negative life events, such as car theft or Parkinson's disease, whilst lying in a functional magnetic resonance imaging (fMRI) scanner, which measures activity in the brain. They were asked to estimate the probability that this event would happen to them in the future. After a short pause, the volunteers were told the average probability of this event to occur. In total, the participants saw eighty such events.

This was mentioned in an article cited by John Derbyshire.

We Know Free Will Doesn't Exist …

… because of experiments on five subjects:

The recordable cerebral activity (readiness-potential, RP) that precedes a freely voluntary, fully endogenous motor act was directly compared with the reportable time (W) for appearance of the subjective experience of 'wanting' or intending to act. The onset of cerebral activity clearly preceded by at least several hundred milliseconds the reported time of conscious intention to act. This relationship held even for those series (with 'type II' RPs) in which subjects reported that all of the 40 self-initiated movements in the series appeared 'spontaneously' and capriciously. Data were obtained in at least 6 different experimental sessions with each of 5 subjects.

There's better data, though. Another study had 14 subjects:

Haynes and his colleagues imaged the brains of 14 volunteers while they performed a decision-making task. The volunteers were asked to press one of two buttons when they felt the urge to. Each button was operated by a different hand. At the same time, a stream of letters were presented on a screen at half-second intervals, and the volunteers had to remember which letter was showing when they decided to press their button.

I think this was the study I discussed here.

Transcranial Magnetic Stimulation

After noticing that two of the articles mentioned were about Transcranial Magnetic Stimulation, I started getting suspicious. So looked at a few more on Pubmed.

According to Excitatory repetitive transcranial magnetic stimulation induces improvements in chronic post-stroke aphasia.:

We enrolled 8 patients with moderate or severe aphasia >1 year after LMCA stroke.

Sample size 8.

According to Accelerated repetitive transcranial magnetic stimulation (aTMS) for treatment-resistant depression

Patients included 9 men, 5 women with a median age of 51 years (range 20–74 years).

Sample size 14.

According to Revisiting the Backward Masking Deficit in Schizophrenia: Individual Differences in Performance and Modeling With Transcranial Magnetic Stimulation:

Seventeen patients (5 female) were recruited from the New York State Psychiatric Institute (NYSPI) Schizophrenia Research Unit.

Sample size 17.

According to Ten sessions of adjunctive left prefrontal rTMS significantly reduces fibromyalgia pain: A randomized, controlled pilot study.:

We recruited 20 patients with fibromyalgia, defined by American College of Rheumatology criteria, and randomized them to receive 4000 pulses at 10Hz TMS (n=10), or sham TMS (n=10) treatment for 10 sessions over 2weeks along with their standard medications, which were fixed and stable for at least 4weeks before starting sessions.

Sample size 20.

According to Worry Facilitates Corticospinal Motor Response to Transcranial Magnetic Stimulation

The 21 participants were self-identified right-handed males who ranged in age from 18 to 38 years (M=20.33, SD=4.22).

Sample size 21.

According to A bi-directional assessment of the human brain-anorectal axis.

Cortical evoked potentials was assessed following electrical stimulations of anus and rectum with bipolar electrodes in 26 healthy subjects.

Sample size 26.

Finally, according to Noninvasive brain stimulation with high-frequency and low-intensity repetitive transcranial magnetic stimulation treatment for posttraumatic stress disorder.:

In this double-blind, placebo-controlled phase II trial conducted between October 2005 and July 2008, 30 patients with DSM-IV-diagnosed PTSD were randomly assigned to receive 1 of the following treatments: active 20 Hz rTMS of the right DLPFC, active 20 Hz rTMS of the left DLPFC, or sham rTMS.

Sample size 30.

I've looked at nine of these papers. By small-sample standards, I could publish something. By the way, are there any large studies on Transcranial Magnetic Stimulation?

Analgesia for Freshmen

According to NCBI ROFL (I don't think they take this seriously either):

Two studies assessed whether playing video games would significantly distract participants from painful stimulation via a cold pressor test. In Study 1, participants (8 men, 22 women, M age = 18.5 yr., SD = 1.3) in an action-oriented game condition tolerated pain for a longer time period and reported lower pain intensity ratings than those in a nonaction-oriented game or a nongame control condition. No differences were found on scores of aggressiveness, competitiveness, or prior video game experience, suggesting that these factors play little role. In Study 2, participants (14 men, 13 women, M age = 19.7 yr., SD = 1.3) engaged in six video game conditions (action, fighting, puzzle, sports, arcade, and boxing) and a nongame control condition. Video game play produced an increase in pulse, which was greatest during the action, fighting, sports, and boxing games. Pain tolerance was greatest during the sports and fighting games. Thus, certain games produce greater distraction, which may have implications for the medical field as an adjunct to pain management.

Maybe this only applies if you're also given extra credit for a psychology class.

More Mesmerism

Magnets are again being used in a small sample study (seen via futurisms):

In the study, the researchers asked 49 female volunteers to take part in a study where they were asked to rate the attractiveness of 220 photographs of female faces, but they were allowed to change their ratings after seeing what others in the study had scored.

When Transcranial Electromagnetic Stimulation (TMS) was used to inhibit the activity of the neurons in the posterior medial frontal cortex, the participants did not change their ratings of the photographs so they were more in line with the rest of the group.

Even if this does pan out, I doubt if it will have much of an effect on society. After all, the people most likely to want to use this are the people who need it the least.

Didn't Mesmer Pioneer This?

According to a recent study:

Here, we show that when subjects can name the colour of presented objects correctly or incorrectly at their free will, the tendency to stick to truthful answers can be manipulated by stimulation targeted at dorsolateral prefrontal cortex.

According to the Guardian's summary:

Inga Karton and Talis Bachmann of the University of Tartu adopted a different and novel approach, by examining the natural propensity to lie spontaneously during situations in which deception has no consequences. They recruited 16 volunteers, and showed them red and blue discs, which were presented randomly on a computer screen. The participants were asked to name the colour of each disc, and that they could do so correctly or incorrectly at their free will.

How do we know the experimenters weren't influenced by magnets?

They Can Make You Take This Nonsense Seriously

The latest news from psychedelic research (an apparent repeat from forty years ago) is that that they can make you more “open”:

In the test, 30 people had a mystical experience, as established by a set of psychological scales. On tests of major personality traits, their openness scores rose, suggesting a greater interest in imagination, aesthetics, feelings, ideas and values. The 22 patients who didn't have a mystical experience showed no change.

Worthy of an Ignobel Prize

One of the latest Ignobel Prizes was awarded for a study on the effects of a need to urinate:

The studies on pee pressure were conducted by two groups of researchers who found that the need to urinate affected decision-making by their experimental subjects. One group found that moderate stress seemed to focus attention on the tasks at hand, but the other group concluded that an extreme need to urinate reduced attention span and the ability to make decisions.

"When people reach a point when they are in so much pain they just can't stand it anymore, it was like being drunk," Peter Snyder, a professor of neurology at Brown University, told AP. "The ability to hold information was really impaired."

When I looked up the paper in question (located via Scientific American), I found (I added the bold):

Eight healthy young adults consumed 250 ml of water every 15 min until they could no longer inhibit voiding. Performance on standardized measures of cognitive function was measured at hourly intervals which were classified as baseline, when individuals reported an increase in the urge to void, a strong increase in the urge to void, an extreme increase in the urge to void and postmicturition.

Small Sample Watch No. 1

The first example (also on my main blog):

According to a new study at the University of St. Gallen seen by SPIEGEL, one contributing factor may be that stockbrokers' behavior is more reckless and manipulative than that of psychopaths. Researchers at the Swiss research university measured the readiness to cooperate and the egotism of 28 professional traders who took part in computer simulations and intelligence tests. The results, compared with the behavior of psychopaths, exceeded the expectations of the study's co-authors, forensic expert Pascal Scherrer, and Thomas Noll, a lead administrator at the Pöschwies prison north of Zürich.

28???

Introducing Small Sample Watch

I have often been annoyed by breathless reports of studies by alleged “scientists” using pathetically-small samples that mainly serve to confirm standard prejudices. I am starting a blog dedicated to such studies. When I see a news report of one of these, I'll post a link and sometimes a quote.