Revisiting the Flawed 1% Estimate of Adult Autism in the UK

If you remember, a survey results that was first published in 2009 and again in 2010 found that an estimated 1% of the adult population in the UK had some form of autism.  This result was a big deal at the time because it was one of the first results that demonstrated that autism is as common in adults as it is in children.

Unfortunately, it is now very likely that this result is also wrong.  A new study was quietly published this month  that seems to blow the earlier result out of the water.  The problem is something of a technical one but, to put it simply, it turns out that the screening test used in the survey, a modified version of the autism quotient called the AQ-20, is a very poor test for autism.

The basic idea of surveys like this is to give a quick and easy screening test to a large population to identify the people most likely to have a condition.  You then pick a sample out of the screened population and give them a more comprehensive evaluation to determine how many of them actually have the condition.  You can then use that number to estimate how many people would have been found to have the condition if you had evaluated them all.

The survey in the UK did just this - it used the AQ-20 to screen the population and then picked a subset of the population and gave them the ADOS.  It found that out of the an approximate 618 people evaluated using the ADOS that 19 of them had a form of autism.  That 19 was then extrapolated back into the entire population using the AQ-20 score to arrive at the the estimate that there would have 72 total cases of autism found, or about 1% of the population.

This final step is the problem as it turns out that the AQ-20 has two glaring flaws -

First, it is a very poor screen for autism, at least in this population.  The sensitivity was found to be 0.73 and the specificity was 0.62.  That means that it would miss almost 1 in 3 people who had autism while saying that almost 40% of the people who didn't have autism did.

Second, and much more importantly, the actual AQ scores are not a good predictor of whether a person had autism or not.  In the words of the authors in the latest validation study -

The AQ-20 self-report screening questionnaire score was found to have a low correlation (0.24 ; p<;0.0001) with the continuous ADOS-4 total score and unsatisfactory sensitivity and specificity with the ADOS 10+ threshold. It was not possible to predict confidently which of the phase 1 respondents with AQ-20 scores of >= 5 had ASD unless they had been assessed on the ADOS-4 in phase 2.

Pay very close attention to that last sentence - having an AQ-20 score of greater than five could not be used to accurately predict which of the people had autism.  Yet that is exactly what the original survey did to arrive at the 1% estimate.

I think it is safe to say that it is likely that the one percent estimate is wrong.