Lies, damned lies, and statistics

As I wrote about last time, a new survey has been released in the UK that seems to show that the rate of autism in adults is the same as in children - about 1 percent. These results are being pushed as some sort of final confirmation that autism is as common in adults as in children.

Yet, if you take the time to look at what the survey says, you will see that this result of 1 percent is rather flimsy and based in large part on statical models. I am going to walk through the numbers and how this survey has put together to show what I am talking about.

Here is the document that I will working from -

Autism Spectrum Disorders in adults living in households throughout England

A group of 13,171 households were selected from what was designed to be a representative population of people living in private households in England. The researchers contacted each of these households and 7,461 of them agreed to participate. From each household one person between the age of 16 and 74 was selected to be interviewed for the survey. This is referred to as phase I as of the survey.

These interviews took place between October 2006 and December 2007, which means that the latest birth year included in the survey should be 1990. As a side note for those people who think this survey disproves any relation between MMR and autism - keep in mind that the MMR was introduced in the UK in 1988. So the youngest people included in the survey most likely had the MMR.

Each person interviewed completed a series of short tests designed to determine the probability of Psychosis, ASD, Borderline personality disorder, or Antisocial personality disorder. Those that scored the highest in any of the four area were selected to be screened in phase II.

The autism section of this questionnaire has derived from the Autism Quotient screening test. But, the standard 50 question version of the test that was not used. The researchers thought that version was too long so they created their own version. They cut down the number of questions to 20 and changed the wording of some of the questions.

This custom test is one of the problems with this survey. Since this test has not been validated by other researchers (or even used anywhere else) it is difficult to know how well it finds individuals with autism. This becomes important later.

The exact method used to select for phase II is somewhat confusing. The simple way of looking at it is that the researchers converted each score in the four areas to a probability that this person would be selected for phase II. They then picked the highest single probability and used this as the chance that the person would be selected. Anyone who had a non-zero chance and agreed to participate in the second phase was included. This resulted in a set of 4,050 who were "eligible" for phase II.

The scales used to set the probabilities were very non-linear. For example, in the autism portion of the test the scores were grouped from 0-4 up to 12+. If your score was 0-4 you had no chance of being included while if your score was 9 you had 2.9 percent change, 10 was a 25 percent chance, 11 a 61 percent change, and 12+ was a 100 percent chance. There were similar scales for the other tests, and in general they worked the same way.

From this group of 4,050 the researchers selected those with the highest probabilities of being selected. This resulted in a group of 849 individuals that, out of the entire population, were MOST likely to have autism. Out of this group 630 agree to participate and were tested in a few ways, the most important being the ADOS. This test is considered to be the current "gold standard" for diagnosing autism.

In this group of 630 the ADOS ultimately identified 19 individuals with autism. So far, so good. The only real problem is that custom selection test.

If you look at the numbers at this point, the second phase of the survey covered the 8 percent of the population most likely to have autism and found about 3 percent of this group had autism. When put this back into terms of the entire population (19 in 7,461) you find that the rate would be close to 0.25 percent (about 25 in 10,000). This result is in line with past estimates of autism in adult populations.

This is where the survey gets strange. The researchers assert that, if they had examined the entire population of eligible for phase II, they would have found an additional 53 cases of autism. This would mean there would be 72 cases in all and the rate would jump to just under 1 percent. It is this assertion that is being widely quoted in the media.

Yet this figure is based on a complex statistical extrapolation that makes a lot of assumptions about the data. One of the main assumptions here is that anyone who had a score about 4 on the custom autism screening test had a non-zero chance of having autism. The problem is that we have no idea how good of a job this test does at finding people with autism. Is a score of 4, 7, 9, or highest actually associated with autism? This question has no answer because this test has not been validated. I am unsure that you can successfully extrapolate from a smaller group to a larger one when the properties of both groups are unknown.

I also find it hard to believe that, even though the segment most likely to have autism was examined, there are supposed to be more than double the the number of cases found in the remaining population.

If I gave a group of children a known screening test, such as the MCHAT, and used the results to select the 8 percent most likely to have autism, I would expect that this subset would contain most, if not all, of the children with autism. I would not think that there would be more than double the number of children in the other 92 percent of the population.

The other major problem with this survey is that we are not given any data about the age of the individuals with autism. All that we are told is that there were not enough individuals found to group into "narrow bands of 10 years", but, when combined into three broad bands of 20-30 years, there was a "slight downward" trend. The end result is that we have no real information about the ages of these individuals so it is entirely possible that they are mostly found at younger ages. But what we cannot say from what we know is that autism is evenly distributed across adults of all ages - we lack the data. But again, this is being claimed in the media.

The bottom line here is that you have to be careful when looking at the results of surveys or studies. It is way too easy to get the results that you want using statistics.

(P.S. If you made it all the way down to the bottom of this post you deserve an award)

The Hidden Horde of Adults with Autism Found?

Perhaps the most important question about autism that remains unanswered is whether we are currently in the midst of an autism epidemic. Is autism more common today than it was ten, twenty, or fifty years ago, or are we just better at recognizing it?

The most recent data available suggests that the current prevalence of autism in the United States is 1 in 100 while in the UK it is thought to be 1 in 84. This stands in stark contrast to the prevalence of 4 in 10,000 that was the going rate for decades before.

The skeptics who don't believe that the autism is more common today than yesterday will tell you that there are a variety of reasons for this contrast. They will tell you that the definition of what autism has become broader, that people with autism used to be given different diagnoses, that the greater awareness of autism leads to greater numbers, and that parents are jumping at the chance to have their children labeled as autistic to take advantage of all of the great services available.

All of the reasons are true, to a point. But I don't think that they can, even if combined, take account for more than half of the rise. Just for the sake of argument, assume for a minute that all of the reasons above are completely plausible and that one percent of the population has always had autism. This leaves us with one rather large, unanswered question.

Where are all of the adults with autism?

If the estimate of 1 percent is correct, then there should be over 2 million adults living in the US with autism while in the UK there should be over four hundred thousand. You would think that it would be easy to find a group that makes up one percent of the population, yet all attempts to locate this hidden horde have failed.

That is until now. Earlier this week the results of the Adult Psychiatric Morbidity Survey were released and they purport to show that one percent of adults in the UK do in fact have autism.

The BBC has an article on the survey that says, among other things, that

...the report suggests that, despite popular perceptions, rates of autism are not increasing, with prevalence among adults in line with that among children.

It also suggests that, among adults, rates of autism remain broadly constant across age groups.

The findings do not support suggestions of a link between the MMR vaccine and the development of this condition.

So there you have it, this survey finally shows that the rate of autism is constant across age groups and that ... wait, what does this have to do with the MMR?

That doesn't sound, right. Lets look at what the survey actually found, shall we?

Hmm, they randomly selected households in England and came up with a group of 7,461 adults who agreed to take part in the survey. These adults were screened using a modified version of the Autism Quotient, and 5,329 people where selected by the AQ to move onto the second phase. Of this second set 630 people were selected to be given more comprehensive tests including the current gold standard, the ADOS.

Of this last group 19 were found to have an autism spectrum disorder.

The survey estimates that had all of the 5,329 cases been tested using the ADOS that 77 cases would have been found, but that is based on a lot of statistical extrapolations that is hard to make head or tails of. Actually, if you read the survey it is filled with a lot of talk about the methods and the statistics and how these numbers were estimated and how those numbers were made up, I mean estimated.

So based on the 19 cases, the survey forecasts that a full one percent of the adult population of England has autism. And, regardless of what is in the BBC, there is no mention of the MMR nor is there any data concerning it.

The survey found 19 members of the hidden horde in the UK, that means there are only 399,981 more members to find. How hard can that be?

The Autism Vaccine Wars

For those of you who missed the opening battle in the autism vaccine wars and wanted a cliff notes version there is a recent paper in Neurotoxicity Research that gives a good summary. This paper is freely available, so I encourage everyone who is interested to read it for themselves.

Are Neuropathological Conditions Relevant to Ethylmercury Exposure?

As most of you know there are two forms of mercury at issue here. The first is called methylmercury and is the type that makes up the majority of our exposure to mercury. We are exposed to this type from the environment and the food that we eat. Science has a good idea of how the body responds to this form of mercury and what its affects are.

Then you have the second type of mercury, known as ethylmercury. Vaccines contained a substance called thimerosal that is metabolized by the body into ethylmercury. We do not know as much about how this form of mercury acts in the body and therein lies the problem.

Around 1999 someone in the US looked at the vaccine schedule, did some basic math, and came to the conclusion that children were receiving far more ethylmercury than was realized. Since there are no specific safety guidelines for ethylmercury, they used the next thing, the guidelines for methylmercury. And according to these guidelines, the combined does that children were receiving was well over the safety limit.

As would be expected, panic ensued. Two years later it was recommended that thimerosal be removed from vaccinations, and today most vaccines are thimerosal free.

This was the first major battle in the autism-vaccine wars which continue to this day. With that in mind, lets consider this paper.

The authors start out by giving a brief history of mercury and its use in vaccinations. If you don't know the history then it is worth reading this paper for that alone. Just keep in mind where the authors are coming from and that they are glossing over some inconvenient details.

The authors make the the point that the properties of ethylmercury were not understood in 2001 when the decision to remove it from vaccinations was made. It was assumed that this form of mercury acted the same way that methlymercury. However, subsequent research has shown that this assumption is not valid and that ethylmercury acts differently in the body than methylmercury. As a result, our initial concerns were most likely unfounded and thimerosal in vaccines does not cause developmental disorders.

The authors give three specific ways that ethylmercury differs from methlymercury :

1. Ethylmercury clears from the body much faster than methylmercury.
2. Ethylmercury does not deposit as much mercury in the brain as methylmercury.
3. Ethylmercury decomposes much faster than methylmercury.

The authors go into great detail for each reason, citing multiple studies and building a careful, rational argument for each of their points. I am not going to talk about the underlying reasons as I think they are mostly correct - anyone who is interested can read the paper for themselves.

I believe it likely that the paper is correct, ethylmercury acts differently in the body and is safer than methylmercury. The available evidence does not support the idea that thimerosal alone can cause autism.

However, at the same time, I don't think that the case is completely closed. There are still unanswered questions and concerns that have not yet been addressed.

First, even though it was recommended in 2001 that thimerosal be removed from vaccinations, the existing stock of vaccinations were not recalled so were likely still used until exhausted. This means that the exact removal date isn't know and that some children could have been still receiving these vaccinations in 2002 and maybe even early 2003.

Second, thimerosal is still used in some vaccinations, such as certain versions of the flu shot, so it has not been completely removed. If you consider recent developments such as the swine flu shot, you will see it is still possible for a young child to get a sizable dose thimerosal (100 mcg over 30 days).

Third, it is not enough to say that thimerosal is not as bad as another form of mercury. It is still a form of mercury that has never had adequate safety testing (thimerosal vs placebo). It should be completely removed from vaccines as it is still a toxic substance and there are safer alternatives available.

Fourth, we don't yet have accurate data to tell us what is going on with the rate of autism in children born after thimerosal was removed. The normal figures the CDC publishes looks at the prevalence of autism in 8 year old children. This age was selected because it is thought that most children will be identified by this age and that the diagnosis is stable at this point.

The latest data available from the CDC is for children who were 8 in 2002. So to be able to do an apples to apples comparison we are going to have to look at children who are 8 and have never received thimerosal. If you look back to my first point, the first year we will be able to do this is 2009 (children born in 2001) although to be completely sure we will have to wait until 2011 or 2012 (children born in 2003 and later). At the rate the CDC is publishing data we won't have enough information for another 6 years.

Lastly, and most importantly, this research is talking about only one of the potential issues with vaccinations - thimerosal. Even if you accept the fact that is isn't likely that thimerosal alone causes autism that does not automatically say that vaccines in general are safe or that there aren't other potential problems.

The bottom line is that there is still a lot of research to be done and until that research is done the autism vaccine wars will continue.

Swine Flu Vaccine

Photo by samantha celera
(flickr)

There has been a lot of chatter recently on the new H1N1 (Swine) flu vaccines concerning how well they will work, what side effects they might have, and what testing (or lack of) has been done. Much of what I have run across up until now has been speculative and devoid of any real information that would help a parent decide whether to vaccinate their child.

So I thought that I would point out this excellent article by Dr. Sears about the new vaccines -

Four Swine Flu Vaccines Approved by FDA

If you are interested in the topic I highly suggest reading this. Mainstream medicine does a lot of lip service to the concept of informed consent but they tend to provide a one sided version of the facts and the only "informed" decision that is acceptable is the one that they agree with. Dr. Sears provides real information on which to base a decision.

To paraphrase what Dr. Sears is saying, the vaccine manufacturers basically took their existing flu shots and replaced the part that targets the conventional flu virus with one that targets H1N1. So the pros and cons of the components of the shots themselves are the same as the other flu shots.

The main concern with these vaccines is that they have received very little testing so it is not known whether they will be effective in preventing swine flu. Also, if you following the CDC's recommendations and give your child both the regular flu shot and the H1N1 shot they could be receiving up to four shots over a month. Giving a child this many shots flu shots in a short period of time has not much safety testing.

Some types of these shots still contain up to 25 mcg of mercury. If you need two shots of each type and do them at the same time, this could add up to 100 mcg in a 30 day period and, well, haven't we all been down this road before? All I will say is that if you are following the recommendations for your child ask for the versions of the shots that do not contain mercury.

Study : Omega-3 fatty acid for treating autism

In the August issue of Journal of Child and Adolescent Psychopharmacology is a brief report on the omega-3 and autism -

Omega 3 fatty acids for treating autism

In this study the researchers gave 1 gram of omega-3 to a group of ten children for twelve weeks. Of the nine children who completed the study, eight of them showed modest improvements in the core symptoms of autism using several measures.

That's the good news.

The bad news is that like most of the studies of this type this one is relatively weak because the study population was very small and the participants knew what they were taking. However, unlike other studies, the evaluations on the children were done by trained professionals and did not rely on parental ratings.

None of the children in the study showed any side effects nor did any children show a worsening of their autism. This study adds to the growing list of studies that shows that omega-3 supplementation is safe and potentially helpful for treating autism.

Now, if the US Patent Office would only grant a patent on omega-3 to a pharmaceutical company then maybe we would see larger scale testing to prove the effectiveness.

Prenatal testing for Autism is here

Picture from NIH

One of the more popular arguments against searching for a cure for autism is equating the search for a cure with the desire to kill unborn children.

The reasoning goes that since autism is genetic and immutable the only way to "cure" it is to prevent it from happening in the first place. So when anyone talks about a cure what they really are talking about is developing a prenatal test to detect autism so that these children can be aborted.

You know, like happens with Down syndrome. It is estimated that, right or wrong, up to 90 percent of fetuses with Down's are aborted - although I believe this number is starting to fall as the prognosis for these individuals is improving.

This line of reasoning is also used to argue against doing research into the genetics of autism. Because you know, if we can find the genes responsible we can create a test and, well, I think you get the idea.

There is only one problem with this idea - OK, I lie, there are many problems with it - but one in particular that I want to point out.

Prenatal tests already exist for a form of autism called Rett's syndrome.

Rett's syndrome is one of the five conditions that make up what is called the autism spectrum. It is the only one that has a known genetic cause. And, according to the National Institute of Health there is a prenatal test available -

Rett syndrome affects one in every 10,000 to 15,000 live female births. It occurs in all racial and ethnic groups worldwide. Prenatal testing is available for families with an affected daughter who has an identified MECP2 mutation

So here you have the ideal battleground for the Neurodiversity movement - a form of autism that has a proven genetic cause and a prenatal test that can be used to abort the fetus that have this condition. Get ready for the outcry over this because this is going to be big.

...

Wait, is that the sound of crickets that I hear?

Why hasn't the ND movement picked this up as a talking point? This test has been around for long enough for people to notice so why are there no outraged protests over this form of prenatal testing? These people are their fellow "aspies", why won't they stick up for them? They claim to be dedicated to preventing this from happening, so why don't they try to stop this?

Perhaps this group that claims to speak for all of autism actually doesn't have many with Rett's in their ranks.

Or maybe it is because Rett's is a very serious condition that can have any number of "side" effects such as gastrointestinal disorders, inability to walk, spontaneous brainstem dysfunction, heart problems, seizures, and a shortened life span? Could it be because if you are male and have Rett's you are unlikely to survive for more than a few months?

In all seriousness, this really isn't a joking matter.

Rett's and Down's are serious condition and I don't know for certain what I would do if I were placed in a situation where I had to make a choice to abort a child. I would not give up any of children for the world nor do I think I could choose abortion except under the most extreme circumstances. And at the same time I can understand where people come from when they make that choice.

But, and this is the important part, trying to cure autism is definitely not the same as wanting to abort unborn children.

EDITED : Reposting this because it for some reason when I initially posted it it appeared as of a month ago.

An inspirational story

It isn't often that I run across a realistic story of someone succeeding in spite of autism that doesn't have a "see its just a difference" twist. So when I ran across this one I thought I would point it out.

Autism didn’t stifle his ambition

By ALEXA GARCIA-DITTA | Special to the Observer | The News & Observer

For the first 12 years of his life, Chris Fitzmaurice couldn’t sound out the letters of the alphabet.

He spoke in high-pitched screeches, didn’t make eye contact with others and wouldn’t hug his parents. Diagnosed with mild-to-moderate autism at age 2, Fitzmaurice’s communication was so limited that his doctor thought he would have a difficult time learning new things.

He proved everyone wrong.

Last month, the 23-year-old started graduate school at UNC-Charlotte.

“I made it a personal commitment of mine,” the Asheville native said. “A lot of people have college degrees; I want to do something big for everyone else.”

Read the rest at The Autism News