In “A Study in Pink”, Episode One of the new Sherlock, the cab driver has been giving people a chance to pick one of two pills. The driver has been given the non-poisonous one four times in a row, and now it’s Sherlock’s chance.
The cabbie maintains “Four people in a row? It’s not just chance”
Sherlock says it’s “luck”.
The internet is full of people who are amazed at the cabbie getting it right four times in a row and Sherlock himself seems finally to be impressed by the cabbie’s skill. But should we be impressed?
If chance is really operating, the probability of getting one guess right is 1/2. If you have two events, this is 1/2 x 1/2 = 1/4. So, by the time you guess a 50:50 situation right four times, it drops to 1/2 x 1/2 x 1/2 x 1/2 = 1/16 (or .0625). Now, the conventional level where statisticians say that there’s more than chance operating is 1/20 (or 5%). In other words, the cabbie really hasn’t done anything significant so far; his luck (.0625) is not rarer than the conventional levels statisticians assume via chance (.05).
However, had the cabbie killed five people with the same method, his luck would have been rare enough (.03125) that we, like Sherlock and the internet, might be impressed and think the cabbie really did have a trick up his sleeve. Unfortunately, Watson shot him before we could find out.