Lars P. Syll — What is a statistical model?

Best explanation or best guess?

As Bertrand Russell put it at the end of his long life devoted to philosophy, “Roughly speaking, what we know is science and what we don’t know is philosophy.” In the scientific context, but perhaps not in the applied area, I fear statistical modeling today belongs to the realm of philosophy. — Rudolf Kalman

This is important. Since the scientific revolution the progress of knowledge has been moving information from best guess using reasoning (philosophy) to best explanation based on testing hypotheses of a theory against data obtained from observations (science).

Science is true explanation where truth is based on criteria and comparison with criteria. 

Philosophy asks why, that is, for reasons. Science asks how, that is, for mechanisms. 

Philosophical accounts are based chiefly on reasoning from assumptions and it is therefore speculative. 

Scientific explanations are based on how things stand and move in terms of observable relationships, ideally able to be expressed formally. Therefore, science is, well, scientific.

See Richard Feynman, Cargo Cult Science. Engineering and Science, Volume 37:7, June 1974. (pdf) (This is a seminal article in philosophy of science, and as a bonus for reading it, it's funny, too.)

A lot of economic, social science and political science fits Feynman's description of cargo cult science. 

A lot of putative scientific knowledge is actually philosophy, that is, speculation, because it rests on speculation based on assumptions rather than explanation based on rigorous observation. However, some it is even flawed formally because the reasoning process is invalid.

Doing good philosophy is difficult because it is easy to fool oneself by falling into logical traps, as Ludwig Wittgenstein spent the later par to his live exploring. 

Philosophy is a battle against the bewitchment of our intelligence by means of our language. — Philosophical Investigations, § 109

Thus, philosophy in Wittgenstein's sense is a prerequisite to critical thinking.

Doing real science is harder because it requires careful attention not only to logical but also to data collection and processing data into information, keeping the signal to noise ratio within tight boundaries. As Feynman points out, a lot of putative science is mostly just noise.

The first principle is that you must not fool yourself--and you are the easiest person to fool. So you have to be very careful about that. After you've not fooled yourself, it's easy not to fool other scientists. You just have to be honest in a conventional way after that.
I would like to add something that's not essential to the science, but something I kind of believe, which is that you should not fool the layman when you're talking as a scientist. I am not trying to tell you what to do about cheating on your wife, or fooling your girlfriend, or something like that, when you're not trying to be a scientist, but just trying to be an ordinary human being. We'll leave those problems up to you and your rabbi. I'm talking about a specific, extra type of integrity that is not lying, but bending over backwards to show how you are maybe wrong, that you ought to have when acting as a scientist. And this is our responsibility as scientists, certainly to other scientists, and I think to laymen. Feynman, "Cargo Cult Science," cited above.

Lars P. Syll’s Blog
What is a statistical model?
Lars P. Syll | Professor, Malmo University