Monthly Archives: December 2008

When training generative models, we usually optimize parameters to optimize joint likelihood.  However, this is in no way (or is it? let me know if you know better) a guarantee that you’ll do better on many real-world benchmarks such as … Continue reading →

In the last post, I presented a comparison of different ways of doing prediction. A natural follow-up question is whether or not there are even better functions?  I observed in the last post that a straight line performs better than … Continue reading →

In the previous post I argued that the second order approximation is useful for prediction.  Let’s apply that to a model with links and see what happens. The random variable over which we take the expectation is now and the … Continue reading →

I was a little bit sneaky in my previous post and the reason I say this will become apparent when I change the function being approximated.  Let’s break it down like this: there are three things we want to compare … Continue reading →

In the previous post, I posed two questions.   I’ll answer the second first. This question considers what would happen if the response function (any response function) were to depend only on a single latent variable.  To use the notation … Continue reading →