by Henry Oliner

One of the most common barriers to logical thinking is the confusion of correlation with causation.  Many errors can be traced to this fallacy.

This does not mean, however, that logical correlations should be dismissed. In the social sciences we usually lack proof of causation and correlation is all we have, but all correlation is not alike. The larger the data pool the greater the chance of over rating correlation and confusing it for causation.

The blackjack player who wins 12 consecutive hands has not defied the odds of the game. The more times he plays the greater the chance of such a hot hand.  Our tendency to underestimate luck and overestimate skill is the subject of Fooled by Randomness by Nassim Taleb. Daniel Kahneman in Thinking Fast and Slow also noted the difficulty of even the most intelligent to comprehend probability.

Some correlations are stronger than others.  Strong correlations that rely on pure statistical outcomes are suspect. A logical explanation of the correlation increases its validity. We can find correlations that led to bad policy, but that does not mean all correlations should be dismissed.

In social sciences causation in a world of numerous inputs can be very difficult to prove, but the combination of logic with multiple correlations can greatly increase the odds of good analysis if we are conscious of avoiding other obstacles to clear thinking.

The problem in political policy is that once a decision is reached, a policy enacted, and a bureaucracy created we get married to the solution and refuse to correct previous analysis and assumptions. Tools that are successful in analyzing are often not so successful in predicting. The problem with government analysis is not their imperfection but their resistance to correction.