Not Even Wrong — Bayesian Edition
I know it's a fool's errand to go around shouting, "Someone is wrong on the internet!" The author throws the usual anti-frequentist barbs that spike my blood pressure. And for some reason, I've become a crusader for the Frequentist cause1.
The basic setup of his post is this: suppose we want to develop a statistical algorithm for trading stocks2. This means we're assuming we can use something about the observed dynamics of stock market (the closing prices of the Dow Jones Industrial Average, for example) to predict how the market will behave in the future. Presumably we want to make money, so we're looking for (hidden) patterns in the market we can use to beat all those silly people investing in mutual funds. Or the much cleverer people using more sophisticated methods.
This is all well and good. Obviously a hard (and unsolved) problem. Also an interesting problem. But now is where we come to the 'not even wrong' portion of his post. Joseph sets down a simple scenario. Suppose our goal is to just predict the market behavior over the next few days. Over the next business week, in fact. Being the silliest sort of frequentist, all we have at our disposal is our estimate of the unconditional probability that the market will be up or down on any given day. (For some reason, Joseph presents this as a conditional probability.) Nevermind that this isn't how a frequentist would approach this problem. We have the whole literature of time series analysis at our disposal. After all, people have been interested in correlated random processes for quite some time.
But forgetting this misstep, let's get to the heart of the misunderstanding:
Frequentists are flummoxed by the probabilities of singular events, but Bayesians deal with fixed and non-repeatable events all the time.
[...]
A Bayesian happily considers the probability distribution for the speed of light or the probability Obama will win the 2012 election.
Nevermind the fact that Nate Silver is a Frequentist. Or that the speed of light (in any given medium) really does have fixed value3. This is just a straw man characterization of frequentist methods. As I said last time, Frequentists make claims about their methods, not any given use of their methods. An intelligent Frequentist would try to infer structure in the stock market time series4. They might even use a sophisticated filtering algorithm. They would certainly treat the data as resulting from a stochastic process, not a bunch of independent Bernoulli coin flips.
We can all agree with Joseph that Mr. Straw Man Frequentist would do pretty terribly as a trader. Unfortunately, as with all straw men, he isn't indicative of the actual methodology used by any statistician with a modicum of training.
I'll mention this now, and probably continue to mention it in the future: I have nothing against Bayesian methods. I'm an instrumentalist (in fact, I really like collecting tools), and my motto is 'if it works, then use it.' But frequentist methods do work, and yet a large group of people (from what I can tell, largely physicists and computer scientists) are growing up hearing a fairy tale along the lines of 'Fisher is dead, long live Bayes!' But like all good fairy tales, there's very little truth to it.↩
It probably won't surprise you that this is something that people do. And quite successfully, I might add.↩
Despite what some colorful characters may say.↩
As someone who has tried to do this with the 'up-down' type model that Joseph proposes, I can tell you: there isn't much structure! Nate Silver covers this topic in a section of The Signal and the Noise titled A Statistical Test of the Efficient-Market Hypothesis. I'll probably write a post on his coverage and my thoughts on it shortly.↩