Wide Open Windows

- Overview messages -

Carl Sagan

The Baloney Detection Kit

The Carl Sagan 'Baloney Detection Kit' should be brought out as a matter of course whenever new ideas are offered for consideration. If the new idea survives scrutiny with the tools in our kit, it can be wholeheartedly, if hesitantly, accepted. If you don't want to buy nonsense, even if it is reassuring, there are precautions you can take; there is a tried and tested method, which is the following nine recommendations.

  1. Wherever possible, get independent confirmation of the "facts".

  2. Encourage substantive debate on the evidence by knowledgeable advocates of all points of view.

  3. Arguments from authority carry little weight - "authorities" have made mistakes in the past. They will do so again in the future. Perhaps it is better to say that in science there are no authorities; there are, at best, experts.

  4. Think of more than one hypothesis. If there is something to be explained, think of all the different ways it could be explained. Then think of tests that would systematically disprove each of the alternatives. Whatever survives, the hypothesis that withstands refutation in this Darwinian selection between "multiple working hypotheses" has a much better chance of being the right answer than if you had just dropped the first idea that caught your eye.

  5. Try not to get too attached to a hypothesis just because it is yours. It is only a way station in the search for knowledge. Ask yourself why you like the idea. Compare it honestly with the alternatives. See if you can find reasons to reject it. If you do not, others will.

  6. Quantify. If what you are explaining has a measure, a numerical quantity, you will be much better able to distinguish between competing hypotheses. What is vague and qualitative is open to many explanations. Of course, there are truths to be found in the many qualitative issues we face, but finding them is a greater challenge.

  7. If there is a chain of arguments, every link in the chain must work (including the premise) - not just the most.

  8. Occam's Razor. This handy rule of thumb encourages us to choose the simplest one when we have two hypotheses that explain the data equally well.

  9. Always ask whether the hypothesis, at least in principle, can be falsified. Theses that cannot be tested, that cannot be falsified, are not worth much. Consider the grand idea that our Universe and everything in it is just an elementary particle - an electron, say - in a much larger Cosmos. But if we can never acquire information from outside our universe, isn't the idea unprovable? You have to be able to verify claims. Skeptics should be allowed to follow your reasoning, duplicate your experiments and see if they get the same result.