The only thing worse than slipshod research is having it misreported by the media.
U.S. News and World Report (or World Distort, depending on your degree of cynicism) recently noted a study concerning guns and violence. The reporter’s headline summed up the study by stating “Mathematicians say that statistical data shows that gun control is the best option to save lives.”
Working criminologists reading the headline soaked their computer monitors when they spit out their morning coffee.
In all fairness, we can’t blame the U.S. News reporter for misinterpreting the results of the study. Heavy statically analysis of criminological theoretical math is not her profession. Nor can we fault the researchers … employed at the University of California Irvine’s Department of Ecology and Evolutionary Biology. In the ongoing and tragically comic saga of unqualified biological sciences professionals inappropriately drawing conclusions about gun control and crime, this may well be the most unlikely pairing yet. It is difficult to determine how the previous publications from these researchers – effects of multiple infection of cells, multiploid inheritance of HIV-1 and a weighted coenotic index as applied to soil animal assemblages – relate to gun control and criminology.
Then again, ever since Barack Obama’s Joyce Foundation started shoveling money to medical schools to study crime, not much has made sense in these intersections of politics and “science.”
But let’s not make false assumptions, like so many in the pseudo-sciences of epidemiology-criminology caste have done before. Let’s have a look at this study’s methodologies, assumptions and other comical elements.
The fundamentally flawed assumptions
No research is devoid of assumptions, but there are limits to every absurdity.
This and other such suspect studies begin with the assumption that guns-as-viruses are “bad,” and that there exists a direct correlation between gun availability and gun violence. Even the authors admit “Model predictions are a direct consequence of the underlying assumptions.” And the authors are candidly blunt about fatal gun deaths being their measurement criteria. Using this criterion amuses working criminologists who, knowing criminals on a deeply personal level, tend instead to use violent crime as the standard of measure. They do so especially when discussing gun control, because their research shows that guns are used to deter criminal activity (usually without a gun death), upwards of six times more often than to commit crimes (with or without a gun death). A woman pointing her pink-gripped revolver at a rapist with his clothesline noose will instantly prevent a fatal crime of violence that did not involve a gun.
Much like the routinely derided research of Arthur Kellerman, who also only looked at gun deaths, the fatality measure is meaningless. None the less, that was the unit of measure in this study. GIGO (garbage in, garbage out).
Unfortunately the authors did not redeem themselves. Instead, their own prose buries their credibility even deeper. “The most important assumption for our calculations is that increased legal gun availability leads to more attacks by criminals.” This comes as a surprise to most Americans. In the twenty year span from 1990 through 2010, the per capita handgun supply (legally purchased guns) has increased 36% while the violent crime rate has fallen 45%. Prima facia, a key assumption made by the theoretical biologist (the study’s lead author’s field) is incorrect. Odd it is then that he makes this assumption since the data to prove or disprove his claim is freely available to the public on government web sites.
The author’s assumption gets more bizarre as the paper devolves. “We assume that a certain small fraction of the population is violent (this assumption is relaxed later on by assuming that the population of offenders is not different from the population of victims).” Though there are a couple of possible interpretations of this line, it appears the authors admit that violence tends to be a subcultural phenomenon, but that they are nonetheless going to apply it to the entire population. Stated more simply, you are the same as any random member of the MS-13 street gang. So the odds of being a drive-by victim are just as likely in East Montague Estates as it is in East L.A. Really?
Another assumption was stated in a way that obscures the author’s ignorance. “The extent to which gun possession protects against death when attacked is an important parameter that determines the outcome of the model.” (emphasis mine) But throughout the construction of his mathematical model, the author skirts around is the deterrence affect. Criminologists (not biologists) love mentally molesting inmates, interviewing incarcerated felons about their crimes and determining their attitudes toward their victims. With very few exceptions, criminals state with perfect clarity that if they are unsure their victim is armed, they will not commit the crime. The author bases his models on the false assumption that only armed defense is an issue, which negates both the deterrence and uncertainty effects (“She might have a gun, so I’ll find somebody else to rape and murder”).
As a side note, this highlights the obvious counter observation. Since gun ownership has been rising for the past twenty years and violent crime has been falling, the deterrence effect may be statistically demonstrated. Working economists (people who often delve into criminological investigations and are better equipped than biologists to do so) have concluded that private gun ownership leads to crime reduction. At the very least we can say without contradiction that the historical increase in gun ownership has not caused violent crime to increase.
This is where so much of this alleged study falls apart, much less proving that “gun control is the best option to save lives.” Aside from a catalog of erroneous or at least unprovable assumptions, the authors ignore decades of deep criminological research, most of which is much more statistically robust than the models perpetrated in this study. And it is the model that is really damning. Instead of running multi-variant calculations based on hard numbers collected by the FBI’s Uniform Crime Statistics (like real criminologists do), the authors attempt to concoct a mathematical model using hand-picked supporting studies and thin-air assumptions.
This isn’t science. It might not even be good math.
Bad supporting actors
Since the authors admit to attempting construction of an abstract model, they had to substantiate some (not all) of their assumptions. This is where a lack of patience prevents most people from seeing congenital research defects.
The simplest examples are two cited studies about the number of gun owning households. The authors note that “Approximately 30% of all adult Americans households own a gun,” and list two citations in endnotes.
One of the two citations is to a recent Gallup poll wherein 43% of respondents claim they have a gun in their house (and some criminologists believe such polls under-report ownership rates due to respondent privacy concerns). To make a bold statement that is off by more than 25% (30/43) is a sign of poor workmanship or unexplained data diving.
A bit more esoteric is their statement based on a second citation that “These results demonstrate that the predisposition to engage in crimes remains the same after gun ownership was denied [NICS checks], and that the prevalence of gun-mediated attack depends significantly on the ability of potentially violent people to obtain guns legally.” In simpler language – which is unknown to research scientists – criminals are as likely to commit crimes if they get a gun, even if they get one after being turned away at a gun store. Though unproven even by the study they cite (which involves only selected California modes of criminal gun acquisition) this claim ignores a steady and painful reality, namely that criminals don’t get their guns at gun stores.
The Bureau of Justice Statistics in their recent Firearm Violence, 1993-2011 report again notes that 40% of crime guns come from illegal street dealers. These guns, typically recycled between criminals, are completely off the legal market as evidenced by the serial numbers being ground off. Another 40% come from other non-retail channels, which include thugs buying from thugs, convicts having their girlfriends buy guns for them and other strictly illegal pursuits. Hence this study’s author looks down the wrong end of his microscope. The predisposition is neither created nor enhanced by the acquisition of a gun. Criminals acquire whatever tools they need however they can get them. They are predisposed to violate whatever law is in their way. Deny them a gun at a gun store and they will buy one off the street. This rather negates the study’s primary assumption that “The number of offenders that own firearms is a function of the gun control policy.”
Sadly, it gets worse. “The inverse relative protection that gun ownership provides during an attack has also been statistically investigated. This is best done in a setting where a large fraction of the general population carries firearms, such as in the USA , and this study has been performed in Philadelphia.”
Philly!?!?! Seriously? Have the authors not been reading the newspapers? Despite the rest of the nation reducing their violence, Philadelphia’s violence has been unchanged over the most recent twenty years, except for a few when it spiked (like in 2006 when it was 7% above their 1990 level). No investigation of Philly’s fatalism fails to show that street gangs are perpetrating nearly all of it. Yet the author concludes, incorrectly, that because a large fraction (i.e., gang members) of the general population carries guns, this applies to the rest of the country. Better not tell that to a Dallas resident who bears no resemblance to a Philadelphian.
Indeed, the authors proceed to claim the cited study correlates carrying a gun with the likelihood of gun violence (which is true if you are a Crip and participating in a turf war). Yet a quick scan of the study shows alcohol and drugs to be a much more likely correlating, and the original study doesn’t even list gang affiliation as a tested variable.
Researchers should keep cherry picking in the orchard.
“In order to examine the fraction of offenders that cannot legally obtain a gun but own one illegally, we have to turn to a country with tough gun control laws.” The biologists decided to compare the U.S. with the U.K.. This begs the question “Why the U.K.?” Mexico has tougher gun control laws and is awash in blood (a 2002 Canadian cross-national study listed Mexico having a firearm homicide rate double that of the U.S.). Why not Honduras where gun control is a Brady Campaign vision of perfection and the gun homicide rate is 23 times that of the United States (according to the 2007 Small Arms Survey).
The reason they chose the U.K. is that people in England and Wales have historically had low homicide rates regardless of the tools available or used. Going back to the beginning of their modern data collection (1918, which mirrors that in the United States), even during periods of nearly no gun control at all, the British propensity toward murder was admirably low. The authors, apparently ignorant of this fundamental piece of criminological history, assumed (yes, another assumption) that only the U.K. gun control laws determined their homicide rates. Had the study’s authors bothered to look at U.K. time series data, they would have learned that gun deaths in the U.K. steadily increased in the years after their near-total ban on private firearm ownership.
As part of their odd methodology, the study’s authors equated gun laws to market saturation. In other words, they modeled their math assuming that the more liberal gun laws are the more people who would own them. Yet here in the United States, arguably the freest in terms of gun laws, there is nowhere near complete ownership. Criminological surveys and those conducted by the likes of Gallup Poll show the household saturation rate to be between 40-50%. The authors also fail to define their understanding of “the market.” Since they have intermixed criminal and non-criminal participation, through legal and illegal acquisition, they present no definition of a “market” and thus no basis for analysis.
Their continued citations raise even more concerns. Sprinkled among the endnotes, they reference papers dealing with military conflict. If the research was attempting to correlate planned violence, be it on the battle field or inner city drug battles, this might have some place. But contrasting warfare with rape prevention is beyond unreasonable. They also cite studies that employ game theory, which commonly involves random event simulations. Though interesting from a homogenized prediction of reactive behaviors, it has nothing to do with criminal violence. Violent predators have their own psychology and motivations that defy game theory. So do armed victims.
Perhaps the most disturbing denial of evidence came when the authors said “In the field of gun violence … compared to other fields of biology or epidemiology, only a limited amount of data is available to inform the construction of mathematical models.” We might accept perhaps that they meant in the intersection of epidemiology and guns there is little useable data. But the statement admits an ignorance of decades of study published at the Bureau of Justice Statistics, the deep research conducted and summarized by criminologists like Gary Kleck in Targeting Guns, and extensive time series studies from economists such as John Lott. Ignorance is forgivable. Willful ignorance isn’t.
End game, but not end of game
The real danger is not this study. My analysis will be just the beginning of a rapid evisceration of the paper if not the intellectual evisceration of the authors. The real problems are in the future.
I mentioned Author Kellerman, who wrote the fatally flawed and much ridiculed paper Protection or Peril. That study earned a record number of entries at JunkScience.com and has been used as an example of lousy research methodology. Yet it is still citied by gun control advocates near thirty years later and gets echoed by today’s media, despite its bottom line conclusion being thoroughly debunked.
The future is the real danger, as the authors of this new study foreshadow. “The aim is to steer the debate towards arguing about assumptions and statistics … [to] serve as a guide for future statistical, epidemiological, and modeling studies … our paper paves the way to further studies which will refine this approach and eventually provide a detailed understanding of how gun availability influences the amount of gun-related and other violence in human populations … the models highlight crucial parameters that can determine how gun availability affects the level of firearm-induced homicides.”
Ponder this for a moment. The authors, who openly admit their goal is to show firearm availability to be a Bad Thing, created this methodological monster to feed future criminological illiterates. Like some recently debased climate scientists, they seek to arrive at a conclusion through mathematical modeling and not analysis of substantial existing data. Worse still, they want to create a test bed that is not criminologically accurate but which, based on assumption, predicts a non-existent future. They seek to pervert future policy by perverting science.
The media is their ally. The study, in a rare moment of full disclosure, said “This model suggestion cannot be interpreted as a solid result that can recommend specific policies.” Yet U.S. News and World Report stated with blunt certainty “Mathematicians say that statistical data shows that gun control is the best option to save lives.” Maybe truth, spread liberally, can save everyone though these researchers and U.S. News reporters are beyond help.