Welcome to my new website, still very much under development.

To have something to put in a first post, I’ll link to this very interesting Nature Reviews Neuroscience paper by Button et al.  (2013); thanks to Neil Berthier for bringing it to my attention.  The paper discusses a statistical concept I hadn’t previously encountered, “Positive Predictive Value” (or PPV), which is the conditional probability that given a rejection of the null hypothesis, the null is actually false.  It points out that PPV depends very strongly on statistical power:  For low powered studies, it can be the case (given some assumptions about the base rate of true vs. false null hypotheses) that most rejections of the null actually arise when the null is true!  The paper is primarily concerned with the problem this presents for the interpretation of human neuroscience studies, but it’s clearly a problem for other branches of psychology as well.  The moral here is that the power of your study not only influences the probability of rejecting the null, if the null is false; it also influences the probability that a rejection of the null, if you do reject, corresponds to a real effect.  This result provides some justification for the intuition that I (and maybe most people) have that results meeting conventional tests for statistical significance (i.e., p < .05) are still to be taken less seriously when the study is very small.