Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why is Bayes' theorem such a magnet for cranks? Is it because you make your prior anything and pretend it's partially true?


I don't know, but it does seem to be sometimes evangelised with more fervor than I'm comfortable with. Us v Them is always a suspicious narrative.

Even in this article, stuff like:

Finally, in 1983 the US Air Force sponsored a review of NASA's estimates of the probability of shuttle failure. NASA's estimate was 1 in 100,000. The contractor used Bayes and estimated the odds of rocket booster failure at 1 in 35. In 1986, Challenger exploded.

makes me uneasy. The rhetoric is too clearly marked.


The "1 in 100,000" silliness was noticed by Feynman and has nothing to do with Bayes, it was just PHB-thinking.

http://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_o...


Yea, that has nothing to do with Bayesian vs frequentist, and everything with massaging numbers to get the results that where expedient. I'm sure that the people at NASA would have been plenty smart enough to work out how to get the 1 in 100,000 result using Bayesian analysis if they really wanted to.


Just curious, who are the cranks you refer to?

(Not trolling, I'm honestly curious. My mental association for Bayesians is people who want to tag documents, diagnose illnesses, or similar AI tasks.)


There seems to be some sort of variant of Goodwins law surrounding Bayesian analysis. Anytime a topic anywhere touches anything related to statistics or probability chances are someone, who's total understanding of the field is a couple of misunderstood wikipedia articles, will jump in and say "Pah, you're only saying that because you're all dirty Frequentists. If only you'd use Bayesian analysis is everything would be obvious".


This made me laugh, because I'm working on a blogpost about how one can use Bayes to argue that humanity is about to go extinct. Seriously.


'Seriously'?That seems plausible to me. It's not like nukes, asteroids, biowarfare, supervolcanoes, etc. have simply gone away.


Luckily my argument doesn't depend on the messy details of any of those things. But I'm getting ahead of myself -- I should just finish the post.


Anything to do with world population history?


I'd guess Doomsday Argument (sigh).


Look on the bright side: in the future the cranks might cite you as well


I knew a doctor at a noted research hospital who was using Bayes to fine tune cancer treatments. I still wonder if he was on the right track with his research or was he as you put it a "crank".


I think you're misunderstanding, there is nothing about using Bayesian analysis that makes you a crank. Using Bayesian analysis where it makes sense is good and sensible and standard practice among just about everybody in the field (although there are some argument about the size an shape of the set where Bayesian analysis makes sense). Cancer treatment would be a perfect example of an area where Bayesian analysis makes sense.

However, for some unknown reason, Bayesian analysis has also become a trendy buzzwords among huge number of crazy internet trolls who seem to think it's a magic formula that can solve all problems, and who have some paranoid delusions that "they" are trying to suppress the knowledge of Bayesian analysis.


On the other hand, there's not a lot of Bayesian analysis taught in high school, statistics 101, or any of the other places that non-stats-nerds are likely to be. It's useful for a lot of things, easy and intuitive (thus its popularity amongst statistical laymen). So why isn't it taught?

The costs of teaching and learning Bayesian Analysis are low (it's just not as hard as, say, the method of moments), and it does have benefits.

My old stats 201 book (Wackerly, Mendenhall and Scheaffer) covers Probability, Discrete Random Variables, Continuous Random Variable, Multivariate Distributions, Functions of RVs, The Central Limit Theorem, Estimation, Properties of Point estimators and Methods of Estimation, Hypothesis Testing, Linear Models / Least Squares, Designing Experiments, Categorical Data, and Nonparametric Statistics. 15 topics (including the introduction), and Bayesian analysis isn't mentioned. Bayes Law is (of course), but only as a theoretical tool, and for solving toy problems about pirates, beads, and rats in the second chapter.

You wouldn't take an engineering analysis book seriously if it didn't mention FEA, but statistics courses can hold their heads up while completely ignoring a useful and easy to teach tool.

Of course, good statisticians and mathematicians will learn about it (later, or on their own), but there's leagues of economists and engineers coming out who will never bother wrapping their heads around it.

Of course, it's entirely possible that it's not so much a conspiracy spearheaded by old-guard frequentists so much as introductory stats courses being focused on teaching a core of theory (LS and MoM), rather then teaching practical tools to people who will use them. You could also accuse introductory math courses of ignoring useful, fun, and easy stuff (scaling?), while focusing on an old, predefined, widely accepted body of theory.


I find that most of the Bayesian 'militants', if you will, aren't actually doing research using Bayesian methods, but are more recent converts writing blog posts about it. Most researchers I know who use Bayesian methods in research aren't even particularly dogmatic about it, and don't go around writing essays about the Evil Frequentists and Oppression of Bayesianism.

In fact, most (at least in my circle) use frequentist methods as well, as well as methods that don't fall comfortably into either camp. In ML it's particularly common for the same researcher to use any/all of kernel density estimation, Bayesian graphical models, SVMs, etc., depending on the problem.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: