Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Simpson's Paradox is scary if you've not seen it before, but you should not stop here and instead proceed immediately on to omitted variable bias and the conversation here (http://normaldeviate.wordpress.com/2013/06/20/simpsons-parad...).

In short, Simpson's paradox occurs because probability distributions and causal claims are distinct things which behave differently. It's nothing more than a particular, insidious example of correlation not implying causation. It's perfectly possible for a probability distribution to have a "contradictory" shape, but perfectly impossible for logical statements about the world to be contradictory.

The resolution is that you shouldn't let your probability distributions turn into logical statements without analyzing your causal assumptions. This will lead you to whether or not excluding a variable is omitted variable bias (and whether including an improper one will lead to included variable bias, which is rarely recognized).



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: