by Mark Thoma, Economist’s View
This article was originally published by Economist’s View on May 01, 2013
Publication in our best journals is based on factors other than merit, “author prestige also comes into play“:
Everyone has the same chance at the AER, right?, by John Whitehead: Wrong:
We spoke with Virginia economics professor William R. Johnson, who edited the edition of the Review in which the [Reinhart and Rogoff] paper first appeared.
This annual edition, “Papers and Proceedings,” differs from all others in that the papers come out of presentations made at the yearly meeting of the American Economic Association, he said.
The papers are personally selected by the AEA’s president-elect, in consultation with a committee.
As a result of these unusual circumstances, Johnson said, the editing of “Proceeding” papers is less rigorous.
“Normal peer review doesn’t happen for these papers in the way of other issues of the AER.” …
But author prestige also comes into play, Johnson said, adding that that was true for all AER papers, not just the ones that appear in “Proceedings.”
We all knew this but it is a little surprisingly to see its admission. [via www.businessinsider.com]
It was awhile ago, but I once discovered an error in a paper in the AER (the author used the level of the price level rather than the log). When I pointed out the error, the author – who is very well known (and now the Fed chair) – wrote a letter to the editor of the AER arguing that it didn’t materially affect the results, and subsequently our note pointing out the error was rejected (much like Reinhart and Rogoff argue that their results are not materially affected by their error, but I think the error mattered as it weakens the case in the Bernanke and Blinder paper, but even if it doesn’t wrong results should be corrected – the results cannot be replicated based on the information in the paper). The prestigious author won out over lowly me, and to this day there is a wrong result in a table in a influential paper in the debate over how to conduct monetary policy. The only place I know of where the error is noted is in a footnote to a (relatively obscure) paper of mine (along with Jo Anna Gray). The footnote says:
Table 1 of Bernanke and Blinder [1992 ] reports marginal significance levels for the federal funds rate that are dramatically higher than those for M2. There are numerous differences between our studies, most of which are inconsequential. However, this discrepancy in results is due to a computational error in the Bernanke and Blinder study. While the error significantly affects some of the F-statistics reported by Bernanke and Blinder, it has little effect on the corresponding variance decompositions reported in the paper. We thank Ben Bernanke for providing assistance that allowed us to confirm and correct the error.
Again, whether or not is materially affects the results is debatable. The F-statistics changed quite a bit, the IRFs less so, but in any case results containing computational errors should be corrected, especially in papers as influential as this one turned out to be. (Update: This also shows that the profession cares very little about making it easy to replicate results.)
We will never be a science so long as this crap persists.
