by Dirk Ehnts, Econoblog101
Thanks to Stefan A. for pointing out the Wikipedia entry “All models are wrong”:
“All models are wrong” is a common aphorism in statistics. It is generally attributed to the statistician George Box.
The first record of Box saying “all models are wrong” is in a 1976 paper published in the Journal of the American Statistical Association.[1]The paragraph containing the aphorism is below.
Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.
Box repeated the aphorism in a paper that was published in the proceedings of a 1978 statistics workshop.[2] The paper contains a section entitled “All models are wrong but some are useful”. The section is copied below.
Now it would be very remarkable if any system existing in the real world could be exactly represented by any simple model. However, cunningly chosen parsimonious models often do provide remarkably useful approximations. For example, the law PV = RT relating pressure P, volume V and temperature T of an “ideal” gas via a constant R is not exactly true for any real gas, but it frequently provides a useful approximation and furthermore its structure is informative since it springs from a physical view of the behavior of gas molecules.
For such a model there is no need to ask the question “Is the model true?”. If “truth” is to be the “whole truth” the answer must be “No”. The only question of interest is “Is the model illuminating and useful?”.
Box repeated the aphorism twice more in his 1987 book, Empirical Model-Building and Response Surfaces (which was co-authored with Norman Draper).[3] The first repetition is on p.74: “Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.” The second repetition is on p.424: “Essentially, all models are wrong, but some are useful”.
The entry has some more comments, quotes and discussion of the subject, among them a short story – or rather, short paragraph – by Jorge Luis Borges about a map the size of the territory. The Wikipedia entry somehow does not contain any critical doubts concerning the definition of “wrong”.
Let me very briefly discuss this point. If you say “All models are wrong” then the most important issue is to define the words. “All” is quite clear, “are” also is without much doubt. So, we are left with “models” and “wrong”. A model usually refers to some mathematical variables linked in a system of equations. A model cannot be an identity, because they are right by definition.
The more interesting discussion is the one about the definition of truth. The philosopher Bertrand Russell has written something on truth about a century ago in his book The Problems of Philosophy (ch. XII). Here is the beginning of the chapter:
Our knowledge of truths, unlike our knowledge of things, has an opposite, namely error. So far as things are concerned, we may know them or not know them, but there is no positive state of mind which can be described as erroneous knowledge of things, so long, at any rate, as we confine ourselves to knowledge by acquaintance. Whatever we are acquainted with must be something; we may draw wrong inferences from our acquaintance, but the acquaintance itself cannot be deceptive. Thus there is no dualism as regards acquaintance. But as regards knowledge of truths, there is a dualism. We may believe what is false as well as what is true. We know that on very many subjects different people hold different and incompatible opinions: hence some beliefs must be erroneous. Since erroneous beliefs are often held just as strongly as true beliefs, it becomes a difficult question how they are to be distinguished from true beliefs. How are we to know, in a given case, that our belief is not erroneous? This is a question of the very greatest difficulty, to which no completely satisfactory answer is possible. There is, however, a preliminary question which is rather less difficult, and that is: What do we mean by truth and falsehood? It is this preliminary question which is to be considered in this chapter. In this chapter we are not asking how we can know whether a belief is true or false: we are asking what is meant by the question whether a belief is true or false. It is to be hoped that a clear answer to this question may help us to obtain an answer to the question what beliefs are true, but for the present we ask only ‘What is truth?’ and ‘What is falsehood?’ not ‘What beliefs are true?’ and ‘What beliefs are false?’ It is very important to keep these different questions entirely separate, since any confusion between them is sure to produce an answer which is not really applicable to either.
The chapter ends with a reformulation of the results:
We may restate our theory as follows: If we take such a belief as ‘Othello believes that Desdemona loves Cassio’, we will call Desdemona and Cassio the object-terms, and loving theobject-relation. If there is a complex unity ‘Desdemona’s love for Cassio’, consisting of the object-terms related by the object-relation in the same order as they have in the belief, then this complex unity is called the fact corresponding to the belief. Thus a belief is true when there is a corresponding fact, and is false when there is no corresponding fact.
It will be seen that minds do not create truth or falsehood. They create beliefs, but when once the beliefs are created, the mind cannot make them true or false, except in the special case where they concern future things which are within the power of the person believing, such as catching trains. What makes a belief true is a fact, and this fact does not (except in exceptional cases) in any way involve the mind of the person who has the belief.
Truth, Russell says, is correspondence with facts. If minds do not create truth or falsehood, but only beliefs, then I would argue that models are beliefs. They can be true when they correspond to the facts. So, there is hope! Models can be right after all! I’m happy with the answer, because I have developed a model based on the BoP identity and some simple behavioral equations (consumption, imports, taxes all depend on income). A model is an abstraction, but as such it can be right. Of course, there is no proof that a model that has been right today will be right tomorrow, but that only makes economics an art. What one cannot know perhaps one should state clearly, to paraphrase Wittgenstein badly and tongue in cheek. An article in the NYT by Jamie Holmes says the same thing:
The study of ignorance – or agnotology, a term popularized by Robert N. Proctor, a historian of science at Stanford – is in its infancy. This emerging field of inquiry is fragmented because of its relative novelty and cross-disciplinary nature (as illustrated by a new book, “Routledge International Handbook of Ignorance Studies”). But giving due emphasis to unknowns, highlighting case studies that illustrate the fertile interplay between questions and answers, and exploring the psychology of ambiguity are essential. Educators should also devote time to the relationship between ignorance and creativity and the strategic manufacturing of uncertainty.
So much food for thought, but it’s only Monday!
