On the Misuse of Data and Confirmation Bias
October 1st, 2015
by Philip Pilkington
When I was doing my undergraduate in journalism I became interested in economic reporting and commentary. One of the things that struck me was the inability of respected opinion-makers to handle data; a disease that Dean Baker documents daily on his Beat the Press blog.
Much to my surprise when I began to study economics I found that the situation was not much better among academics and working economists. Often whole debates would disintegrate on a basic data-point.
A good recent example of this is the case of Jonathan Finegold Catalán, an Austrian economist and Mises.org contributor. (The Austrians, I should add, are serial offenders in the misuse of data…). Catalán was debating the blogger Lord Keynes on a rather obscure point of the equally obscure Austrian Business Cycle Theory. Without getting into the weeds of what is a rather unproductive and interminable debate, Catalán provided the following graph to prove that the price of finished capital equipment fluctuated in line with supply and demand:
Looks good, right? Wrong. Catalán committed the cardinal sin of data usage: he forgot to inquire whether the measure was nominal or real. If it is nominal the price fluctuations can be explained by changes in inflation, while if it is real Catalán’s point that there is something resembling a free-market for capital equipment might stand up. Is this hard to check? Nope. If the measure doesn’t say “real” or “inflation-adjusted” then it is likely nominal. We can further check this by mapping the CPI next to the data-points being used. If they approximately match up then most of the price fluctuations are just an illusion built into the data.
Low and behold, the effects of inflation seem to be almost completely dominant! This should not be surprising at all to anyone familiar with the structure of the data produced by the Federal Reserve. Nor should it be surprising to someone who works with data often. (And I should say before the Austrians pile into the comments section: no, much of the inflation shown in the above time-series is not the result of demand-pull forces).
It is extremely pleasant when data fits nicely with our conception of the world.
I would like to put such misuse of data down to poor education. And indeed, part of the blame lies there. There is, however, another reason for such misuse: confirmation bias. It is extremely pleasant when data fits nicely with our conception of the world; it gives us an ego-boost and reinforces our theories. But we must be extremely careful because oftentimes this desire to prove oneself right leads us astray. Better to deflate our egos and our figures at the same time then get caught up in errors.
It is for this reason that we should always let the data speak rather than our theories. It is the latter that should change in response to the former; not the other way around. The only way to get away from confirmation bias is to fully give oneself over to the data: the data is sacred; the theory is not. It seems to me unlikely, however, that certain types of economists can accomplish this; because, as I have argued elsewhere, their theories are not an attempt to understand the world but rather an active attempt to shape it in their phantasmatic image.
... we should always let the data speak rather than our theories.
Update: Catalán has responded in the comments section saying that I have misrepresented his argument. He says that his point “doesn’t have anything to do whether the prices of capital goods have to do with supply and demand”. Let’s be very clear here so that we avoid confusion. He wrote:
I’m not saying that prices aren’t “administered,” because all prices, to some extent or another, are set by the firm (the real world isn’t perfectly competitive, where everyone is necessarily a price taker). I am saying that the idea that producers’ good prices aren’t flexible enough to change with flows of profit is wrong.
Now to me that does have to do with whether the price of capital goods has to do with supply and demand. What Catalán is arguing is that the price of capital equipment fluctuates with the change in the flow of profits (that is, the demand for the goods and services being produced by said capital equipment).
I do not, however, wish to argue semantics because I have no desire to handle the slippery fish that is Austrian capital theory. So, let us just say for the sake of argument that I have misrepresented his argument. Well, we are quite clear then that if he is saying anything at all it is that “producers’ good prices” are “flexible enough to change with flows of profit”. If this is the case then we would expect there to be some correlation between the change in the price of capital equipment and the flow of profits. Do we see such a correlation? Nope. See for yourself:
That graph is a bit pesky to read. I had to graph the two variables on different axes because the fluctuation of profits is so much greater than the fluctuation of the price of capital equipment. Nevertheless, if you can take the time to read it properly you will see that there is no correlation. Indeed, in some periods such as 1974-75 and 1979-1980 there is a strong negative correlation. (There is, however, some correlation in the late-1940s, but in order to say that there is a relationship between these two variables we would need a longer term correlation; this graph is, as can clearly be seen, all over the place).
The only way to get away from confirmation bias is to fully give oneself over to the data: the data is sacred; the theory is not.
Whatever way you cut it, Catalán’s argument doesn’t add up. What he was really seeing when he was looking at the price of capital equipment was inflation plain and simple. And inflation has nothing to do with the flow of profits in a capitalist economy as can be seen clearly above.