Tag Archive for 'bad statistics'

This guest post from Daniel Howrigan, Benjamin Neale, Elise Robinson, Patrick Sullivan, Peter Visscher, Naomi Wray and Jian Yang (see biographies at end of post) describes their recent rebuttal of a paper claiming to have developed a new approach to genetic prediction of autism. This story has also been covered by Ed Yong and Emily Willingham. Genomes Unzipped authors Luke Jostins, Jeff Barrett and Daniel MacArthur were also involved in the rebuttal.

Last year, in a paper published in Molecular Psychiatry, Stan Skafidas and colleagues made a remarkable claim: a simple genetic test could be used to predict autism risk from birth. The degree of genetic predictive power suggested by the paper was unprecedented for a common disease, let alone for a disease as complex and poorly understood as autism. However, instead of representing a revolution in autism research, many scientists felt that the paper illustrated the pitfalls of pursuing genetic risk prediction. After nearly a year of study, two papers have shown how the Skafidas et al. study demonstrates the dangers of poor experimental design and biases due to important confounders.

The story in a nutshell: the Skafidas paper proposes a method for generating a genetic risk score for autism spectrum disorder (ASD) based on a small number of SNPs. The method is fairly straightforward – analyze genetic data from ASD case samples and from publicly available controls to develop, test, and validate a prediction algorithm for ASD. The stated result – Skafidas et al. claim successful prediction of ASD based on a subset of 237 SNPs. For the downstream consumer, the application is simple – have your doctor take a saliva sample from your newborn baby, send in the sample to get genotyped, and get a probability of your child developing ASD. It would be easy to test fetuses and for prospective parents to consider abortions if the algorithm suggested high risk of ASD.

The apparent simplicity is refreshing and, from the lay perspective, the result will resonate above all the technical jargon of multiple-testing correction, linkage disequilibrium (LD), or population stratification that dominates our field. This is what makes this paper all the more dangerous, because lurking beneath the appealing results is flawed methodology and design as we describe below.

We begin our critique with the abstract from Skafidas et al. (emphasis added):
Continue reading ‘Guest post: the perils of genetic risk prediction in autism’