Thursday, August 11, 2005

Questions for David Kirby-- Dataphobes Beware

David Kirby’s response regarding the California autism data is interesting and perplexing on a number of levels. He notes that we aren’t certain about the extent to which thimerosal remained in vaccines between 1999 and 2003, and that it may be too early to tell whether reduction in thimerosal has had an impact on autism caseload in California. I agree. I wouldn’t myself have trumpeted the California DDS numbers as proof of anything. It is Kirby, in his Huffington Post article, who holds up the California data as reflecting the "gold standard of autism epidemiology" and who claims that "if the numbers in California and elsewhere continue to drop – and that still is a big if -- the implication of thimerosal in the autism epidemic will be practically undeniable."

Even after reading Kirby’s response, I can’t tell whether he still thinks that the numbers in California show a "drop." Gratifyingly, Kirby says that Citizen Cain is "absolutely correct," though it isn’t clear what I am "absolutely correct" about. I contend that the California Department of Developmental Services data show a steadily increasing caseload of autistic children aged 3-5, from at least the third quarter of 2002 through the second quarter of 2005. Kirby responds with some unfamiliar numbers on "new entries" into the California DDS system since the third quarter of 2003, which show a decline for two quarters, then start climbing up. He also reiterates his original position that the "California numbers" are "dropping," but that he "will certainly point out this new, and perhaps confounding, development."

I admit to being confused about what this means. But let’s try to work through it. The truth is out there! We may never find it, but we should be able to get closer to it.

Kirby and I seem to have very different approaches to looking at the numbers. Mine is pretty straight-forward– look at the caseload of 3-5 year-olds with autism. Is it decreasing? No. Is it increasing? Yes. Every quarter, for a total of 38 percent since the third quarter of 2002.

Kirby’s approach is more complicated and, to my eyes, confused. He provides data on "new entries" to the California DDS autism caseload, and then says that these "entries" are more correctly called a "net gain" in cases. Then he continues to refer to "entries." The two things are very different: "new entries" would tell us how many new cases of autism among a given age cohort have been registered in a given quarter. To the best of my knowledge, California DDS does not report such data. The data that I’m aware of just shows caseload by age. If California DDS also reports new entries, I would appreciate being informed of where to find this information.

But I don’t think that new entries really are reported, because Kirby says that "entries" really means "net gain." A net gain would be an increase, quarter over quarter, in the number of persons within a given age cohort that are part of the California DDS autism case load. Therefore, a "net gain" within an age cohort is a very different thing than "new entries." Why? Consider 3-5 year olds. If there were a steady state (zero net gain) in the autism caseload, and no deaths or drop-outs from the system, there would be new 3-5 year old entries into the system every quarter, and these new entries would be matched by an equal number of children that would no longer be in the 3-5 year old category for the simple reason that they turned 6 years old. In a steady state, there would be no net gain, but continual new entries. If there were 5000 cases, and during the next quarter 1000 of them turned 6 years old, and there were 1000 new entries, there would be zero net gain.

Therefore, a positive net gain of any size means that autism incidence is growing among 3-5 year old Californians (if we assume, just for the purposes of this discussion, that the California DDS data really reflect autism incidence). For the numbers to be "dropping," we would need to see actual decreases in caseload and a negative net change. Kirby seems to think, if I understand him correctly (a big if) that if the rate of increase in caseload slows then autism incidence is decreasing. A steady state, for Kirby, seems to involve a steady rate of increase in the number of 3-5 year olds with autism. Wrong. Autism caseload among 3-5 year olds is increasing every quarter. The amount by which it increases varies slightly, but in no sense can a reduction in the increase be considered a decrease in incidence.

But what the heck are these numbers that Kirby provides as either "entries" or "net gain." Kirby says that he got them from Rick Rollens; could Rick Rollens or Kirby let us know where these data have been published? Because the numbers don’t make sense as either "new entries" or as "net gain." The caseload numbers that I compiled XXX show a net gain of 139 3-5 year old cases in the second quarter of 2005, compared with Rollens/Kirby’s 449. For every quarter, the Rollens/Kirby numbers are higher by far than the actual net gain in California DDS caseload. So the Rollens/Kirby numbers are too high to represent net gain.

Hmm. Could the Rollens/Kirby numbers actually represent "entries" rather than "net gain." No, because the Rollens/Kirby numbers are much too low to represent entries. There is no way, with new entries averaging 397 cases per quarter over the last 8 quarters (as the Rollens/Kirby data portray) that the number of cases among 3-5 year olds could be 5446 and growing (as the California DDS data indicate). Since children don’t enter the system until they are 3 years old, and since there are 12 quarters in the three year age spread, there would have to have been an average of at least 454 new 3-5 year old entries per quarter to sustain a caseload of 5446. In reality, the number of new cases would have to be substantially larger than 454 cases per quarter, given that some children do not enter the caseload until they are 5 years old (or older), and therefore the distribution of 3-5 year old cases will be skewed towards the older end of the range. As a result, more than 1/12 of the caseload would be turning 6 years old every quarter, and an equivalent number of new entries would be needed just to maintain a steady state.

So– some questions for Mr. Kirby– where do these numbers come from? What do they represent– new entries? Net gain? Something else? Do you concede that the numbers are not, in fact, "dropping?" Do you concede that the relevant numbers (caseload among 3-5 year olds) are increasing every quarter, with some fluctuations in the rate of increase?

Let me repeat my thanks to Mr. Kirby for responding so graciously. I hope the conversation will continue.