Thanks for the comment. The third blog post in this series will discuss linear contextual bandits, for which I used Langford's VW.

It does a huge amount better.

Yeah, in general, the contextual bandit approach seems to be a better strategy.

This is blog post 1 out of 3. The second is all about Thompson Sampling, and should be coming out on Friday =)

We don't actually use Gaussian models for any of our Bayesian tests =). For continuous data, we use log-normal models, and the next blog post will be about how to make all that work.

And for intervals: yup, we just use sampling. But it's super fast because all you need are independent samples. For more details, see this first post:

http://engineering.richrelevance.com/bayesian-ab-tests/