That a medical student may have doctored data is both a crime and a terrible pun.
But this is probably what happened when a student formerly from the University of Michigan Medical School presented a paper at the American Academy of Pediatrics convention, and the media apishly copied headlines about the paper’s claims concerning the intersection of guns and kids.
Since reporters are generally not well educated [should I end the sentence there] on criminology, statistics, robust data sources and the minutia of gun policy, they accepted the paper without scrutiny. So I’ll have to roll-up my sleeves and dig into this one for the reporters who wouldn’t.
The paper (United States Childhood Gun-Violence – Disturbing Trends) took child hospitalization discharge data along with survey responses concerning gun ownership, and applied a weighted analysis to determine that more handguns causes more child deaths and hospitalizations.
This came as a surprise to criminologists, and other people with cranial blood supplies, who have watched the number of handguns in circulation rise steadily over the past three decades, while simultaneously watching child firearm deaths decline. Using criminology/epidemiology data gold standards – Center for Disease Control (CDC) mortality databases and Bureau of Alcohol, Tobacco and Firearms Commerce Reports – we can chart these with ease. It shows that while gun availability has risen, fewer kids die.
Surprising then that the aspiring doctor told the world that “Policies designed to reduce the number of household firearms, especially handguns, may reduce childhood [gunshot wounds].”
To be clear, from 1981 through 2009, the number of child firearm fatalities has fallen 48%. But because the population kept growing, the rate of child firearm fatalities has fallen 56%. This reduction occurred in a period where the number of handguns went up 216%.
So how can a young doctor be so wrong, and would you want him removing your appendix? With only his abstract to review (the paper and raw data were not immediately locatable and a request sent to the American Academy of Pediatrics was not answered) we can only illustrate the obvious deficiencies in United States Childhood Gun-Violence – Disturbing Trends.
When is a database debased?
In every field of science, there are several sources of raw data. Some are the standard, some are not, and some simply stink. This paper failed to use the most common data sources available in the fields of criminology and epidemiology, namely the CDC’s WISQARS database and the Bureau of Justice Statistics crime databases (both online for everyone to use). Instead the doctor-in-training selected a highly variable database of hospital discharge records and a single-point-in-time government survey of gun ownership.
None of this is pretty.
The Kids’ Inpatient Database (KID) lists the discharge status of young patients. However, in terms of mortality and even injury, it is incomplete. Not every dead child enters a hospital, not every injured child needs hospitalization, and the descriptions of the “accidents” have no validation. Data quality is vague, and for such small numbers as those represented by fatal firearm deaths for children, they are statistically suspect.
But this gets much worse. Participation in KID is completely voluntary, and the number of contributing states and hospitals has risen since its inception. This means the number of children being recorded rose as well. Whereas the CDC shows a steady drop in child firearm deaths since 1993, the KID database wasn’t even launched until 1997, when it had a mere 22 participating states. Between then and 2009, the number of states adding data to KID rose