Arguments from Probability

As with any logical proposition, one contradiction disproves the proposed rule. If each of 38 counterexamples has merely a 10% chance of being valid — an underestimate — then the probability that the Earth is billions of years old is less than 3.8%. In other words, the Earth must be young with a likelihood of greater than 96%.

(From the Conservapedia.)

It is a statistical certainty (p < 10^-11) that there are innocent people being held at Guantanamo Bay.

(Sig on Slashdot.) These are a couple of cases I've seen recently of probability theory misused. The first one is easier to deal with, so let's start with that. The claim is that each counterexample has "merely a 10% chance of being valid — an underestimate". This claim is already problematic. Take this counterexample:

The Bible makes references to the dinosaurs. There is no explanation for this if dinosaurs supposedly lived hundreds of millions of years ago.

How would we calculate the odds that this claim is correct at greater or less than 10%? For the record: Biblical evidence for dinosaurs is razor-thin. It mostly rests upon Job 40, which reads (in the King James Version):

Behold now behemoth, which I made with thee; he eateth grass as an ox. Lo now, his strength is in his loins, and his force is in the navel of his belly. He moveth his tail like a cedar: the sinews of his stones are wrapped together. His bones are as strong pieces of brass; his bones are like bars of iron.

I simply ask the reader: do you think there is a greater or less than 10% chance this describes a dinosaur? Follow-up question: if dinosaurs — huge beasts larger than anything else that ever walked this earth — lived in Biblical times, do you think it's likely there would be one single reference?

It strikes me that the probability that this is true is less than 10% — much less, in my judgment. But ultimately the point is that this is a judgment call.

But there is a deeper problem with the Conservapedia's argument. It makes the implicit assumption that each counterexample is independent. Here's a thought experiment: suppose we roll a fair 6-sided die and try to determine which face is on the table (that is, the one we cannot observe). We observe each of the five other faces, and find they are 1, 2, 3, 4 and 5. For each one we say: we did not observe a 6, which means the odds of the bottom face being a 6 is 5/6. In total, then, the odds of the bottom face being a 6 is (5/6)^6 or about 33.5%. But this is obviously absurd, since we know that the bottom face must be a 6: we've seen all the other faces. The reason the argument fails is that our observations are not independent.

The same holds with Conservapedia's argument. Consider these two claims:

1. The Moon's orbit is a very strong counterexample: the moon is receding from the Earth at a rate[3] that would have placed it too close to the Earth merely four billion years ago, causing instability in its orbit, tidal catastrophes on Earth, and other problems that would have prevented the Earth and the Moon being as they are today. Additionally, the moon's orbit is becoming increasingly and unexpectedly eccentric, suggesting a lack of long-term stability,[4] which further disproves the theory of an Old Earth.

4. The planetary orbits in the Solar System — including Earth's — are unstable and unsustainable over the long periods claimed by Old Earth believers.

Let's ignore the question science behind these claims and take them at face value. The problem is that even if both are true, both may simply be indicative that orbits are unstable. If that claim has a 10% chance of being true, then both claims 1 and 4 would be true, but would not make the basic fact any more or less true. If we observed planets in other star systems that had unstable orbits, for example, we could toss in additional claims about them, but they would not be independent of the claims already made and thus would add no value to the overall question of whether the universe is old or young.

Very well. Let's consider the Guantanamo prisoner claim now. We can back into the assumptions made pretty easily. If the likelihood of an individual prisoner being guilty is g, then the likelihood that every prisoner is guilty is g^N where N is the number of prisoners. Currently N=171 if Wikipedia is reliable. The claim is that g^171 < 10^-11, so this means g < 0.86.

That strikes me as low. But perhaps the calculation was made when we had more detainees. The maximum number was 775, in which case g < 0.968. Eh, maybe. But as with the Old Earth argument, one problem is that we are asked to believe a specific probability about the individual case, and then extrapolate it to the overall case. The argument is extremely sensitive to these probabilities: suppose that actually g = 0.995. Then the odds of a single innocent detainee would be only 58%. We have no idea what the probability g really is, though, so we are left making guesses. And these guesses strongly influence the outcome.

Independence is less of an issue here, but it's important to mention another factor. Not each detainee may have the same probability of being innocent. This can make a big difference. Suppose that there are 10 detainees, 9 of which are definitely guilty and 1 of which has a 50% chance of being guilty. Alternately, suppose there are 10 detainees who each have a 95% chance of being guilty. In both cases, a randomly chosen detainee has a 95% chance of being guilty. But the probability of having at least one innocent detainee is 50% in the first case and 40% in the second. It is easy to craft other examples that can push these probabilities as far apart as you like.

One thing I've ignored covering here is the question of whether these numbers are actually relevant. That's a big topic that could cover large swathes of the philosophy of science, but suffice for now to point out that probability has a somewhat different meaning in the Old Earth question than in, say, gambling. Either the Earth was created about 4.5 billion years ago or it was not: this is a question fundamentally unlike asking whether the next dice roll will be boxcars. The similarity between the two is that both deal with unknowns. We will never know, with 100% certainty, whether the age of Earth is about 4.5 billion years, so the best we can do is assemble all the evidence and weigh it. That's how science works.

On the Guantanamo question, I have similarly questions of relevance. Suppose the claim was true that at least one detainee was innocent with statistical certainty. How would that affect our policy? Would it help us determine which one? Would we be willing to let terrorists go in order to decrease the probability of detaining innocents? Such questions cut to the heart of any judicial policy, and they can't be answered by facile calculation.