Today's WSJ discussed an econometric debate, which is always fun (see here).They state:
The "Freakonomics" chapter on abortion grew out of statistical studies Mr. Levitt and a co-author, Yale Law School Prof. John Donohue, conducted on the subject. The theory: Unwanted children are more likely to become troubled adolescents, prone to crime and drug use, than are wanted children. When abortion was legalized in the 1970s, a whole generation of unwanted births were averted, leading to a drop in crime nearly two decades later when this phantom generation would have come of age.Levitt's response is on his website (see here) where he notes
The Boston Fed's Mr. Foote says he spotted a missing formula in the programming of Mr. Levitt's original research. He argues the programming oversight made it difficult to pick up other factors that might have influenced crime rates during the 1980s and 1990s, like the crack wave that waxed and waned during that period. He also argues that in producing the research, Mr. Levitt should have counted arrests on a per-capita basis. Instead, he counted overall arrests. After he adjusted for both factors, Mr. Foote says, the abortion effect disappeared.
The part of the paper that Foote and Goetz focus on is one that is incredibly demanding of the data. For those of you who are technically minded, our results survive if you include state*age interactions, year*age interactions, and state*year interactions.3 interaction variables are necessary to get the right sign and significance? I think that is incredibly demanding. In my experience, interaction variables are kitchen sink type regressors that induce severe multicollinearity and give spurious results. It's like an economist saying his results only appear after doing 3-stage least squares. I have to think something's not really there if you can't normalize the data somehow and show in a simple graph that the pattern is there (in this case, say, by showing the change in arrest rates for abortion and non-abortion states for the relevant age cohort).
I'm partial to the opposite theory, that abortion would, if anything, increase the proportion of evil-doers: abortion is more common among forward-thinking moms who would be good moms, less common among bad moms who view life as a series of random events that happen to them.
HedgeFundGuy - am 2005-11-29 00:42
Steve Sailer (anonymous) meinte am 29. Nov, 01:18:
Right, Levitt left out American studies contradicting his argument
Excellent analysis. The reason Levitt cites European studies claiming that women who have abortions would make worse mothers than the ones who have their children in his study of _American_ crime trends is because the American studies of the impact of abortion came to the opposite conclusion.
Trent and Griner's research, along with other studies undermining Levitt's central argument, was pointed out to Levitt by CCNY economist Ted Joyce in his response to Levitt & Donohue in the Journal of Human Resources, which was entitled "Did Legalized Abortion Lower Crime?" Joyce summed up two reason why Levitt's theory didn't work. The second was:
"Second, analysts, I being one, have tended to overestimate the selection effects associated with abortion. A careful examination of studies of pregnancy resolution reveals that women who abort are at lower risk of having children with criminal propensities than women of similar age, race and marital status who instead carried to term. For instance, in an early study of teens in Ventura County, California between 1972 and 1974, researchers demonstrated that pregnant teens with better grades, more completed schooling, and not on public assistance were much more likely to abort than their poorer, less academically oriented counterparts (Leibowitz, Eisen, and Chow 1986).
"Studies based on data from the National Health and Social Life Survey (NHSLS) and the National Longitudinal Survey of Youth (NLSY) make the same point (Michael 2000; Hotz, McElroy, and Sanders 1999). Indeed, Hotz, McElroy, and Sanders (1999) found that teens who abort are similar along observed characteristics to teens that were never pregnant, both of whom differ significantly from pregnant teens that spontaneously abort or carry to term.
"Nor is favorable selection limited to teens. Unmarried women that abort have more completed schooling and higher AFQT [the military's IQ test for applicants for enlistment] scores than their counterparts that carry the pregnancy to term (Powell-Griner and Trent 1987; Currie, Nixon, and Cole 1995).
"In sum, legalized abortion has improved the lives of many women by allowing them to avoid an unwanted birth. I found little evidence to suggest, however, that the legalization of abortion had an appreciable effect on the criminality of subsequent cohorts."
Andrew Gelman (anonymous) antwortete am 29. Nov, 01:40:
Interactions are important
I can't comment on Levitt's paper (having read neither the book, the paper, the critique, or the response) but I wanted to briefly comment on two general points you made.First, I agree that if the effect is there, you should be able to show it with a graph (although quite a bit of pre-processing might be needed for the graph to make much sense). It's always mystified me that social scientists seem to think tables are more definitive than graphs.
Second, I disagree about your disparagement of interactions. More and more in my work, I find that interactions are not only important; they're often the most interesting part of the story. For some general thoughts on this, see here:
http://www.stat.columbia.edu/~cook/movabletype/archives/2005/08/interactions_ar.html
For a specific example, see here:
http://www.stat.columbia.edu/~cook/movabletype/archives/2005/11/income_matters.html
Acad Ronin (anonymous) meinte am 29. Nov, 02:24:
Minor qualification
I agree with you (pl.) that ideally data should pass the "Tukey intra-ocular trauma test", i.e., it should hit you between the eyes; I am suspicious when I see GMM FIML 3-stage least squares with robust errors (I think I made that up but God knows it could exist); I always worry that the researcher followed an optimal stopping rule that says, keep improving your methods until the results come out right, and then stop.That said, as I read the Levitt response you cite, he is saying that introduced the interaction effects in response to criticism, and that even after he did so, the main results survived. That is, he got his results despite the interaction variables and not because of them.
HedgeFundGuy antwortete am 29. Nov, 03:30:
Acad: I read it again, and Levitt seems to be saying that after Foot and Goetz make adjustments, his results only survive if the interaction variables are included.Andrew: I looked at that first reference, and found "page" 79 to be a rather uncompelling. If any of the main coefficients went to of from insignificance from Model 1 to Model 4, would you really believe it?
Kaiser (anonymous) antwortete am 29. Nov, 07:25:
Is he serious?
Like Andrew, I preface my comments by disclosing that I haven't read the book, the articles or the Foot and Goetz critique. I'm only commenting on Levitt's comment, which is the only thing I read.He seems to be saying that even after including 2-way interactions, the main effects are still significant, which he implies is a good thing ("the results survive"). Is he serious? Whenever interaction effects are significant (which would be why one would keep them in the model), main effects cannot be interpreted in isolation so I don't know what he means by "the results survive".
A totally different line of thinking, which I think he is tangentially referring to, is the fact that his sample size is small so it may be "demanding" of the model to estimate interaction effects (takes away some degrees of freedom). I have no idea how big his sample size is so I can't really comment on whether this is reasonable or not.
Andrew Gelman (anonymous) antwortete am 29. Nov, 12:52:
Interactions are important even if hard to estimate from a given dataset
Hedgefundguy,I agree that the estimates on page 79 of that presentation are uncompelling. The N of that study is just too small to accurately estimate all the interactions that we care about there. The point of that slide to show an exapmle where interactions are important, even though we don't have enough data to estimate them well. The model on the previous slide, fit without interactions, has statistical significance but is not scientifically plausible (I don't really believe that giving a low-value postpaid gift will reduce response rates at all, let alone by 6.9 percentage points).
The actual analysis we did for this problem was far from perfect. But we needed to look at interactions for the purpose of our goal of estimating the effect per dollar of survey incentive. The full analysis (in the Journal of Business and Economic Statistics) is here:
http://www.stat.columbia.edu/~gelman/research/published/jbes01m045r3.pdf
RhetoricalQuestionGuy (anonymous) meinte am 30. Nov, 06:45:
That WSJ dot picture is considerably less nerdy-looking than pictures of Levitt.On the serious side, doesn't this analysis only apply to one particular argument in the paper? If this piece of evidence were effectively nullified, how much would it affect the conclusions?
Steve Sailer (anonymous) antwortete am 30. Nov, 23:42:
His other lines of argument are weak
No, the other lines of argument that Levitt makes are quite weak and have been disputed. For example, he claims that European studies show that women who have abortion would make worse mothers than those who don't. The reason he uses European studies is because American studies came to opposite conclusions, and his theory is about American crime rates. His appeals to Australian and Canadian studies also suffer from the problem that Australians and Canadians aren't Americans (specifically, the racial issues that make up such a large part of the study of crime in America are largely irrelevant there). Furthermore, I've seen a summary of one of the Australian studies, and I'd call it inconclusive, rather than supportive, although that is a grey area judgment.
No, the prestige of his theory has rested mostly on his assertions about what the state-level data showed because it was so much work to try to reproduce it that many felt they had to take his word for it. People were intimidated by that. Now that somebody has tried to reproduce his model, and they've found two important errors, one of which Levitt has so far admitted to.
My position since I debated Levitt in 1999 is that he hasn't met the burden of proof. Large assertions require large evidence, and he hasn't come close. As a Nobel-winning economist told me, Levitt is a master self-promoter.
The cult of personality he has developed around himself might be good for the social sciences in that they attract public attention, but they also diminish critical responses to logic and data. Levitt's oracular approach in Freakonomics induces in the media a hero-worshipping attitude that a Really Smart Young Guy has Figured It All Out, so we shouldn't worry our pretty little heads about it. That's fine for sumo wrestling and other topics Levitt deals with, but for significant public policy issues like abortion and crime, that's a deleterious attitude.
