Tuesday, August 9, 2011

Revisiting the Flawed 1% Estimate of Adult Autism in the UK

If you remember, a survey results that was first published in 2009 and again in 2010 found that an estimated 1% of the adult population in the UK had some form of autism.  This result was a big deal at the time because it was one of the first results that demonstrated that autism is as common in adults as it is in children.

Unfortunately, it is now very likely that this result is also wrong.  A new study was quietly published this month  that seems to blow the earlier result out of the water.  The problem is something of a technical one but, to put it simply, it turns out that the screening test used in the survey, a modified version of the autism quotient called the AQ-20, is a very poor test for autism.

The basic idea of surveys like this is to give a quick and easy screening test to a large population to identify the people most likely to have a condition.  You then pick a sample out of the screened population and give them a more comprehensive evaluation to determine how many of them actually have the condition.  You can then use that number to estimate how many people would have been found to have the condition if you had evaluated them all.

The survey in the UK did just this - it used the AQ-20 to screen the population and then picked a subset of the population and gave them the ADOS.  It found that out of the an approximate 618 people evaluated using the ADOS that 19 of them had a form of autism.  That 19 was then extrapolated back into the entire population using the AQ-20 score to arrive at the the estimate that there would have 72 total cases of autism found, or about 1% of the population.

This final step is the problem as it turns out that the AQ-20 has two glaring flaws -

First, it is a very poor screen for autism, at least in this population.  The sensitivity was found to be 0.73 and the specificity was 0.62.  That means that it would miss almost 1 in 3 people who had autism while saying that almost 40% of the people who didn't have autism did.

Second, and much more importantly, the actual AQ scores are not a good predictor of whether a person had autism or not.  In the words of the authors in the latest validation study -
The AQ-20 self-report screening questionnaire score was found to have a low correlation (0.24 ; p<;0.0001) with the continuous ADOS-4 total score and unsatisfactory sensitivity and specificity with the ADOS 10+ threshold. It was not possible to predict confidently which of the phase 1 respondents with AQ-20 scores of >= 5 had ASD unless they had been assessed on the ADOS-4 in phase 2.
Pay very close attention to that last sentence - having an AQ-20 score of greater than five could not be used to accurately predict which of the people had autism.  Yet that is exactly what the original survey did to arrive at the 1% estimate.

I think it is safe to say that it is likely that the one percent estimate is wrong.

5 comments:

  1. Hi MJ -

    Thanks for pointing this out. Very nicely done.

    - pD

    ReplyDelete
  2. It does not mean the 1% estimate is necessarily wrong, only that this study and methodology were flawed and failed to show it. It is still unclear what the prevalence of autism is among adults and this is something that would be very difficult to find for a variety of reasons. Like the old saw, "looking for a needle in a haystack".

    What was really wrong was that Joseph and "the autistic bitch from hell" and other members of the neurodiversity movement used this is a propaganda tool to claim there was nothing wrong with autistic people, that they were half as likely to marry as nonhandicapped persons and there was no unemployment problem among autistics and they were employed in the same manner as nonautistics.

    ReplyDelete
  3. I would imagine the researchers were working to the figure that they had in their heads anyway.

    More interestingly the Scottish Prison study's survey of the prevalence of ASC's in prisoners seems to have gone AWOL.

    3 years overdue now.

    The Joy of Autism is currently preventing me from doing anything except live in fear of losing my SSI in the face of our Conservative Government's plan to rid the world of Erbkranke like me, so perhaps someone else would like to email the lead author (a woman I once described as the Haggis Queen of Edinburgh) and find out what's going on.

    I suspect, very strongly they've got some very disturbing and embarrassing results.

    ReplyDelete
    Replies
    1. There was some movement on the Prison study front, they tried the AQ to see if they could use it to screen for autism and found it not to work well.

      http://www.ncbi.nlm.nih.gov/pubmed/22662113.1

      Delete