Sunday, August 23, 2009

How not to compare data


People throw around a lot of figures and charts when talking about autism in an attempt to prove their point. Some of the time the information is valid and other times the comparison is as valid as comparing apples and ducks.

Take for example a recent post written by Sullivan at Left Brain Right Brain -


The basic point of his post was to compare this chart from Thoughtful House (TH)


to one that was his own creation that is similar to the one below and based on data from the National Survey of Children's Health (I didn't want to rip off his chart so I made one of my own, feel free to compare it to the one on Sullivan's original post).


He then talked about how the charts don't look anything, that the magnitudes of the numbers are completely different, and furthermore that the NSCH data is "basically flat" and shows no evidence of an epidemic of autism.

Well, I have to agree with him, the charts don't look anything alike. But there is a good reason for that - the data on the charts aren't directly comparable.

Lets first start with some definitions.

The data on the TH chart is labeled a "crude" incidence. It is the basically the yearly change in the number of children with a label of autism served under the Individuals with Disabilities Education Act (IDEA). This number is then assumed to represent the number of new cases and all of those cases are assumed to be in the youngest age. There are number of problems with this approach, but like the title says, it is very crude. There are some comments on the LBRB post here and here that describe the data in more detail.

The data on the NSCH is the number of children found during the survey in 2007 who currently have autism or another ASD. This number is broken out by the age of the child and then normalized to be a number of children per 10,000 children based on the number of children in the age group. This is basically a prevalence figure.

The CDC tells us the difference between "incidence" and "prevalence" is
Incidence is the number of new cases of disease in a defined group of people over a specific time. Prevalence is the number of existing disease cases in a defined group of people during a specific time period. Public health professionals use prevalence measures to track a condition over time and to plan responses at local, regional, and national levels. Incidence is very difficult to establish because the exact time a person develops an ASD is not known.
And what is an epidemic?
In epidemiology, an epidemic occurs when new cases of a certain disease, in a given human population, and during a given period, substantially exceed what is "expected," based on recent experience
So, what are the problems with the comparisons that Sullivan did?

First and foremost, the data being compared don't even represent the same thing. The figures from the TH are meant to represent incidence while the ones from NSCH are prevalence. You can't directly compare them to each other as they don't represent the same thing.

The second problem is that the group being represented isn't the same. The TH data represents children who are being served under the IDEA act and is based on the classification of a child in a public school system. A particular child with autism maybe classified as having autism, having some other problem, not having a problem, or not even included in the data if they don't attend public school. The NSCH data is a survey meant to be a representative sample of the entire population of children. So the data on the two charts does not even represent the same underlying population.

The third problem is that the time periods that are being compared aren't at all equivalent. The numbers from the TH graph were derived from historical IDEA prevalence figures published over a number of years. The numbers of the NSCH are a snapshot of what the current prevalence was in 2007.

The last problem is that an epidemic isn't defined by numbers on a chart, it is defined by something being substantially more common than was expected. The NSCH data from 2007 indicates that the current prevalence is about 1 in 100 (although it does vary by age and substantially by state). Prior to this data being released the accepted figure from the CDC was 1 in 150, and that estimate was from 2002.

If the data from NSCH is accurate and autism is 33% more common now than 5 years ago I think that would count as a substantial increase. The CDC is expected to publish similar findings in the near future and, if they confirm that number, they are going to have to either call an epidemic or provide some other explanation for the increase. Or they might just weasel out of it again, who knows. (My money is on the weasel).

But the point is that you don't look at a chart of prevalence figures and declare there is no epidemic without comparing it to what the expected prevalence is. The CDC is "the" authority in the US for this sort of thing so it is there estimate that counts.

So, how could the comparison have been done better?

Well, it would have helped to be able to compare like to like, but given the data available from IDEA this isn't possible. But, I am going to try to do the next best thing. I am going to use the data from the 2007 and 2003 NSCH surveys and attempt to back into a "crude" incidence.

I will freely admit that this isn't the most scientific approach and does suffer from some of the same flaws that I mentioned above. However, I think the results are closer to how the chart from TH is being generated and will demonstrate what I am saying about comparing apples to apples (well, maybe apples to oranges in this case, but certainly not apples to ducks).

To produce the following chart I took the data from the 2007 and 2003 NSCH data sets and broke out the rough prevalence grouped by the child's age. I then shifted the 2003 data so that the ages of the children between the two data sets would line up properly (ie a child in 2003 that was 5 would be 9 in 2007) and be directly comparable. The result is the following.


There are a few things of note here.

First, while there has a been a push in recent years to diagnosis autism at a younger age, the figures from the youngest age are very likely understated. So the first two ages (2 and 3) for 2007 is likely low as is the 2003 data shown at ages 6 and 7.

Second, there is something off at age 12 in the 2007 data - the number is substantially higher than its surrounding points. If you look at the original 2003 data there was also something off at the unadjusted age 12 - it was too low compared to the the surrounding ages. I don't know what is going on there but it is strange.

If you notice on the chart above there is a a very clear upwards trend as you go from older to younger children. This says to me that there are more younger children being diagnosed than older ones and that the number is growing. Unfortunately, this chart cannot tell you why this is happening.

As a final comparison I took the difference between the prevalence on the above chart to show a (very) crude incidence between 2003 and 2007. You should disregard the points at age 6 and 7 but the overall increasing trend is clear.


Doesn't this look more like the chart from Thoughtful House? It is amazing what happens when you try to compare data properly .

Don't eat the fish, they might give you autism

The U.S. Geological Survey released the results of a new study this past week that looked at the mercury contamination in the environment. The study looked at 291 streams from around the country and found that mercury contamination is present in all of them.

The bad news in all of this is that no fish is safe - every single fish that was sampled had a detectable level of mercury. One in every four fish sampled had a level of mercury that exceeded the EPA's safety limit. Two thirds of the fish exceeded the EPA's level of concern for fish eating mammals.

The good news? The remaining fish would not be that bad to eat - as long as you didn't eat too many at once and only ate them every once and a while. This finding is also expected to help grow the fish population because more fishermen will participant in the catch and release program. Those who don't choose to participate and actually eat their fish? Well, they won't last that long so nothing to worry about there.

So where does all of this mercury come from? According to the study it comes from atmospheric sources and, to a lesser extent, other source like mining operations. The main culprit seems to be coal burning power plants - they emit a form mercury into the air which then ends up in the water supply and works its way up the food chain. By the time the fish end up on our tables they are full of methylmercury goodness.

OK, so there really is no good news here and nothing to be joking about. Methylmercury in the environment is no laughing matter and can cause serious health problems. It is a neurotoxin that can seriously mess you up if you are exposed to too much of it or happen to belong to a vulnerable population.

This is why the EPA suggests limiting fish intake for young children and women who are or might become pregnant or are nursing. According to the EPA, exposure to methylmercury in the womb and shortly after can result in impaired neurological development, damage to the nervous system, and problems with cognitive thinking, memory, attention, language, and fine motor and visual spatial skills.

So, if you have kids, what you really should be watching out for is the fish. I mean just look at that list of things it can cause - it almost sounds as bad as autism. It is a good thing that we know that there is absolutely no relation between mercury exposure and autism otherwise I might start to worry about eating fish.

Friday, August 21, 2009

Autism Rising?

One of the burning questions about autism is whether it is more common today than in the past. This question is more than idle curiosity and the answer has the potential to reshape how we think about autism.

Over the past twenty years we have been treated to an ever rising statistic for how common autism is. Twenty years ago it was something like 4 in 10,000, ten years ago it was 1 in 500, and in 2002 it was 1 in 150. How common is it today? Good question, the CDC is supposed to tell us any day now....

No one will dispute the fact that we seem to be able to find more children with autism than we could in the past and that an ever increasing number of children are being diagnosed each year. Yet there is no agreement of what these rising numbers mean.

So, are we in the midst of an autism epidemic or has the medical community finally put on their collective glasses and seen what has been staring them in the face these past several thousand years of human existence (Gee, the kid can't talk, won't look me in the eye and keeps doing funny things with their hands, maybe something isn't right there) ?

As Harold Doherty recently pointed out researchers have been noodling over this ever increasing figure for a number of years. However, to date, the response has been something along the lines of - "Huh, that's strange, I wonder what is causing that. Maybe we should look at that closer". I am sure that they will get around to looking closely at the issue any day now.

Earlier this month we were handled another tidbit of information in the form of the second National Survey of Children's Health (NSCH). This is a nationwide survey of children's health issues that was first done in 2003 and then again in 2007. The results of the 2007 survey were released earlier this month and they had something interesting to say about autism -

The rate of autism now seems to be at 1 in 100.

That rate does keep moving up, maybe looking at why would be a good idea?

Since the data for 2003 and 2007 are publicly available I thought I would make a pretty picture of the differences in rates between the years. Just as a note, this comparison is not perfect because the wording of the question changed between the years.

In 2003 the question was whether the child ever had autism while in 2007 it was whether the child currently has autism or another asd.

I think the ever vs current would over estimate the 2003 numbers and the autism vs "autism or another asd" has the potential to inflate the 2007 numbers. Although in all honesty, I don't think the exact wording of autism vs "autism or another asd" would make a huge difference because most parents whose child had pdd-nos or aspergers would have said "yes" to the autism alone question in 2003.

But, without further ado, here is a pretty picture of the rates between 2003 and 2007.


Those number are definitely going up, I just have to wonder what it means.

Wednesday, August 12, 2009

Our results with the GFCF diet

I think the Gluten-free Casein-free (GFCF) diet can work for some children with autism. My opinion is not based on a preponderance of scientific evidence because science has not yet caught up with the realities of why people use the diet. Rather my opinion is based on the specific symptoms that my children had and what results that I have seen in my children as a result of the diet.

If you missed the first part where I went into the reasons that we tried the diet for my twin daughters, go back and read it. Done reading it? Good.

Before I go into our experiences let me just say that your experiences can and will vary. If you are thinking of trying this diet with your children do a lot of reading about it first - there are several good sites and books - and if at all possible consult with an appropriate medical person to help deal with any issues that come up.

So as I wrote last time we decided to try the GFCF diet for the twins - or technically in their case we were going GFCF and SF (soy-free). We started off first by slowly removing all sources of dairy (casein) from their diets over a four week period followed by wheat (gluten) over the next four weeks followed by a quick removal of soy. You have to take it slowly because if there is a real food intolerance there, as was the case with the twins, they can go through a form of withdrawal as you remove the food. (Hint, autism + withdrawal = very bad)

Just to mention it, this is one of the problems with the studies done so far on the diet - they are too short. The only double blind study on the diet lasted only 6 weeks. At 6 weeks we were still removing food from the diet. And it takes a while for the effects from the intolerance to fade, it doesn't necessary happen overnight - it will happen over the course of several months to a year.

What we saw from the removal of dairy was impressive. First, there was an improvement in some of the core problems of autism - both girls showed an increase in eye contact, an increase in attention, and they started paying attention to sounds.

The last part is a little bit strange, so let me explain it. After the twins turned a year old they slowly seemed to loose their hearing. It wasn't as if they couldn't hear at all - we did see them respond to some sounds some of the time. But for the most part the simply did not respond to sounds such as talking or noise. What was extremely unusual was they would not startle or react to sudden, loud noises. You could sneak up behind then and bang two pot lids together and they wouldn't flinch, blink, or show any sort of reaction.

Being good parents we took them for a hearing test. They both failed. We took them to a pediatric ENT who examined them, found no obvious physical problems, and sent them for another hearing test. They failed. Finally, he ordered a test called an ABR (auditory brainstem response). This test involved sedating the child and directly measuring the electric signals generated in the brain as a result of sounds. They "passed" this test and established that, physically, they could hear perfectly well.

What is unusual here is the amount that the twins were able to tune out all sounds. I had read that you might wonder if your child with autism was deaf because they wouldn't respond when you talked to them but I have never seen anyone else talk about their child being able to tune out all sound so completely that they wouldn't even startle.

So eliminating dairy seems to have started them down the path of responding to sounds again.

The speech and development therapists who were working with the twins at the time also noted that they started doing much better with their therapy. They would pay attention and become more interactive. They started learning and progressing through the tasks faster than they had before.

The other symptoms that I mentioned last time also started a slow fade and over the course of the following year disappeared. Their stools became normal, the bloating in their stomachs went down, they could sleep on their backs, and their eczema finally went away.

A year later when we repeated the food intolerance tests the readings for all three types of intolerance had dropped to just under 10 - for the casein that is a drop from over 100 to 10 which is a significant drop.

So, in short, for the twins the diet was a huge success. We saw the majority of the changes from removing dairy from their diet - they do not seem to be anywhere near as sensitive to wheat and soy. But, to be on the safe side, since there was some intolerance shown by their blood work, we keep it out of their diet.

We did not have the instant talking that you will hear about other places, and if fact we had to wait two more years before they would start talking to us. Nor did they become non-autistic and loose the autism diagnosis but even still, the changes were significant.

Tuesday, August 11, 2009

Why we tried the GFCF diet

In my previous post I talked about the negative publicity that the Gluten-free Casein-free (GFCF) diet was getting and the need for more research into whether an elimination diet like the GFCF can work. I also said that I am of opinion that it does work - even though there is not sufficient evidence to say that from a scientific or medical point of view.

I thought I would take this opportunity to explain why I think the diet can work. I will warn you in advance, my reasons are based on my experiences and not on any scientific study. If you are the sort of person who thinks that there can be absolutely no value in such statements, you would want to stop reading now. Having said that, I think my experiences are less common than most and I do have some objective data behind my statements.

For those of you who don't know, I have three daughters who have autism. My older two daughters are identical twins so they are among the 60-90% of identical twins who both develop autism. As a fun fact, the change of having identical twins with autism happen to you are relatively small, something like 1 in 50,000. Lucky us.

The twins have been on the the GFCF diet for over three years now and have very good results from it. My younger daughter, while her diet is somewhat being limited by living in the same house, is not on the diet and does not seem to need it. So the following applies only to the twins, my youngest did not have have any of the issues listed below.

This is going to be a little long, so I am going to break this post up into a few parts. This part is going to address the reasons why we tried the GFCF diet in the first place and later parts will go into what effects the diet had and some of the arguments against the diet.

So, the reason that the twins are on the GFCF diet is simple - a developmental pediatrician recommended it. He recommended it for the several reasons -

First, both girls had a history of having of frequent loose stools. We did not experience the explosions that are referred to other other sites but, let me put it this way, things did not have the shape and consistency that they should (think applesauce). We also had problems with constant diaper rashes and redness.

Second, both girls constantly had bloated stomachs. They were not badly distended to the point were their regular pediatrician would have taken steps to address the problem, but it was noticeable.

Third, up until the twins were 16 months old, neither girl was able to sleep for any length of time on their backs. They had to sleep in a bouncer chair, a car seat, or something of the sort to sleep for more than 30 minutes. Once they could turn over they started sleeping on their stomachs and would sleep for longer but if they ended up on their backs they would quickly wake up crying.

Forth, both girls craved milk and other dairy products such as cheese. To put this into context, both girls were somewhat picky eaters and we couldn't get them to even try solid food until 16 months or so - they just weren't interested in what we were eating and refused to try what we gave them. But, they would happily drink bottles and bottles of milk, and once they did start on solids, would happily eat as much cheese as we would give them. They also loved eating tofu - how many infants do you know crave tofu?

Firth, the twins had problems with eczema. Twin A constantly had it all over her body while twin B would get large patches of it every couple of weeks. This is one of the very few areas where they were different, and we still don't know why.

As an aside, I think it is worth noting that we had talked about all of the above with the the twins' pediatrician and most of the time received only non-committal responses. We did get some advice on the eczema - put skin lotion on it. I really doubt that any of the above is in either girls medical charts, even though we mentioned each of the concerns several times. This is the reason that I am skeptical of the Mayo Clinic study that relied on the children's medical charts - my children had some of the symptoms yet I don't think it would be on their charts. Our pediatrician is actually very familiar with autism and its related problems. They are the one who first picked up on the it in the twins and referred us for an evaluation at 15 months. So if a pediatrician who is on top of all of the issues and can recognize autism that early wouldn't chart the non-specific GI complaints, who would?

The last and most probably most important reason for the diet was food intolerance tests that the development pediatrician ordered. As part of his initial consultation for the twins he went over their history, examined them, and then ordered blood tests to check for food intolerance to gluten, casein, and soy.

I am not going to go into detail about these tests but they are fairly standard things - they look for antibodies in the blood that would indicate an intolerance to a specific type of food. The tests gave back a numerical score in the following sensitivity groups - less than 10, undetectable or very low; 10 to 15, low; 15-20, moderate; 20 to 50, high; 50-100 very high; over 100, extremely high. If you scored in the bottom group, as most people would, you had nothing to worry about. As you went up the scale the the intolerance became worse and worse.

Both girls had similar results and the results were interesting. For gluten and soy both girls showed a very mild intolerance (low group) but for casein , well, let me put it this way, you know the saying "off the chart"? That is exactly what their scores were, off the chart, coming in at over 100 in the extremely high group.

Based on all of the above the we took them off of casein, then gluten, and soy.

In the next part I will talk about the results of the diet.

Saturday, August 8, 2009

The Gluten-free Casein-free Diet

The gluten-free casein-free (GFCF) diet has taking a little bit of a pounding in the last few weeks, mainly due to the new study put out by the Mayo Clinic. This study compared gastrointestinal symptoms in children with autism to typical children and found that, for the most part, there were no differences.

Of course the results have been publicized far and wide as proof that the GFCF diet isn't needed and doesn't do anything. There are stories such as the one on Pediatric Supersite entitled "Dietary restrictions may do more harm than good for children with autism" all over the place. (In fairness, if you read the actual story it is a more balanced report than the title indicates; but then again, I think most people just read titles and skim the stories)

Then you have what is, so far, my favorite bastardization of the facts in "Doctor rejects gluten casein free diet for autism" -
Experts at the Mayo Clinic found no potential in the gluten or casein free repertoire to heal the symptoms of autism.
The study says no such thing or even anything close for that matter. There was no data in the study at all about usage of the GFCF diet or symptom relief from the diet so how could it say that the diet doesn't work? The study looked at one set of potential symptoms that can lead to trying the diet, not at whether the diet itself actually did anything.

Of course there are also reports from the agenda driven sites such as Left-Brain Right-Brain which ran this one. In it Sullivan has cherry-picked two studies, the Mayo Clinic study and this one, and concludes that "there is no evidence that children with autism have GI problems at a greater rate than the general public", ignoring the numerous studies that have found the opposite. But what do you expect, if autism is just a "difference" instead of a disorder, as LBRB advocates, it would hardly do to define part of the difference to be perpetual diarrhea. But I digress.

One thing is certain, there is clearly a need for more research into GI issues with autism and how elimination diets might help the symptoms of autism. As it stands today there is a insufficient evidence to say that the diet is effective from a scientific or evidence based medicine point of view.

For me though, the question of whether the GFCF diet can work has already been answered.

It can.

It won't work for every child and it doesn't seem to be a "cure" for autism. But, when it is needed and done properly, it can make a major difference in the symptoms of autism.

Thursday, August 6, 2009

Michelle Dawson


Photo by Leo Reynolds (flickr)

Michelle Dawson has a major problem with using ABA to treat autism. She likes to claim that ABA is unethical, unproven, and bad for "autistics". I am not sure who these "autistics" are or how they differ from the group of adults and children who have a diagnosis on the autism spectrum, maybe they are the same.

But, according to Ms Dawson, ABA is bad for them. Oh, and unethical too, although I can never seem to find the her exact ethical complaints with ABA spelled out precisely, just the assertion that it is, in fact, unethical.

But this isn't anything new and anyone who was familiar with her writings would already know this.

Ms Dawson also has a bad habit of discouraging parents from using ABA to help their children - I wrote about an example of this earlier. She seems especially keen to reach the parents of newly diagnosed children and convince them that ABA is bad for their children. Now, if you ask her about this she will tell you that this is not what she is doing, that is trying to provide accurate information from primary sources, which is good for autistics.

Did I mention that she thinks that ABA is unethical, unproven, and bad for autistics? Good, I didn't want to forget mentioning that.

Now, here is the thing. There is almost universal agreement that ABA is one of the most effective treatments for children with autism that is available today. No one is saying that it is a cure or that it will work for every person on the spectrum - as the saying goes, if you have seen one person with autism then you have seen one person with autism (this holds true even for identical twins).

A huge body of literature and evidence exists that supports the notion that ABA, in general and specifically for autism, is an effective teaching technique. Every major organization dealing with autism agrees on this point, from Autism Speaks to the Autism Science Foundation to the CDC to the American Academy of Pediatrics to the groups that think autism is caused by vaccinations. Heck, even some of the Neurodiversity and ASAN folks accept that it as a valid treatment and they are normally against any treatment for autism.

Then you have Michelle Dawson arguing that ABA is unethical, unproven, and bad for autistics as she did, yet again, just a few weeks ago.

I am all for listening to peoples ideas and considering all possible points of view, even ones that are completely at odds with conventional wisdom, but I don't understand Ms Dawson's obsession with trash talking ABA.

At first I thought that Ms Dawson underwent ABA as a child or young adult and had a bad experience. But I have not been able to find any references to that occurring and everything that I did find seems to indicate that she never had any ABA style therapies, so that can't be it.

Then I thought that Ms Dawson might have some sort of agenda that she is pushing. But I like to give people the benefit of the doubt and I can't see what agenda she would be pursuing or any possible benefit to pursuing it. This view is certainly not helping other people with autism and is especially harmful to young children with autism.

So what I am left with is that Ms Dawson really believes that ABA is unethical, unproven, and bad in spite of the overwhelming evidence against her position. But she seems to be an intelligent person, so I can't see how she can sustain her belief - it isn't rational. And it isn't like she is in an isolated environment where it is easier for irrational ideas to take root.

But then I remembered. Ms Dawson has autism and part of autism is getting fixated or obsessed with items or possibly ideas. Could that be the reason?

I have no idea.

Wednesday, August 5, 2009

Study Watch : Incidence of Gastrointestinal Symptoms in Children With Autism


Photo by EyalNow (flickr)

In the August issues of Pediatrics there is a study that looks at how often gastrointestinal symptoms are seen in children with autism.


The authors looked at the medical records of children under 21 who had been patients of the Mayo Clinic between 1976 and 1997 -apparently this group covers 95% of the residents of Olmsed County, Minnesota - and identified the children who met the criteria DSM IV for autism.

There were a total of 124 children identified, 121 of which agreed to be included in this study. Each of these children were then matched with two control subjects (242) who were matched on gender, age, as well as other factors.

The charts of these groups were examined to find gastrointestinal symptoms and diagnoses in five categories :
  1. Constipation
  2. Diarrhea
  3. Abdominal bloating, discomfort or irritability
  4. Gastroesphageal reflux or vomiting
  5. Feeding issues or food selectivity
The data was then analyzed to determine if children with autism were more likely to have these symptoms or diagnoses. Out of the 121 children with autism, 87 had any reported problem on their charts while 167 of the 242 in the control group had issues.

The results show that children with autism were more likely to have constipation and feeding issues than typical children. There were not more likely to have diarrhea, abdominal bloating, reflux or vomiting and overall had the same incidence of symptoms.

The authors speculate that the constipation and feeding issues in the autism group are more likely to be related behavioral issues rather than any underlying biological reason. So the overall conclusion of the study is that children with autism are not more likely to have gastrointestinal symptoms.

This is a good study, as far as it goes. There is a lot of speculation and guessing in the study that is not supported by the data and there are some areas that I wish had been addressed better.

For example, the authors assert in the discussion section that children with autism are commonly treated with risperidone and that could potentially be responsible for constipation and selective eating. Yet I have to wonder why this wasn't included in the analysis or the data - they had access to the medical records of the children so they should have been able to determine if this was a factor or not.

Then there is the assumption that any gastrointestinal problem would be included in the medical records. The authors note this as a possible limitation of the study but dismiss this concern because children with autism are "subjected to increased medical scrutiny". I find this of line of reasoning very hard to shallow for a number of reasons but mostly because of my experiences with my own children.

My twin daughters had diarrhea, bloating and irritability, and feeding issues when they were infants before we placed them on a restricted diet. After being put on the diet they have food selectivity. We did talk to our children's doctor about these issues at the time yet our concerns were largely dismissed. I would be very surprised if there was any mention in their charts of these symptoms.

Mostly I wish that the authors had collected data on which of the children with autism are on restricted diets and whether that impacted the results. For example, were GI symptoms more common in children on a restricted diet or off?

But, as I said before, this is a decent study, if a bit limited. I certainly think that more research is needed in this area.