Solved – Bayes rule and gaussian PDF

@Ahmed – you are definitely correct in thinking that something is not quite right here.

Conditioning on "point values" which have probability/measure 0 can be "dangerous" and can lead to what is called a Borel and Kolmogorov Paradox. The lesson from this is that in order to be certain that we are defining a conditional probability unambiguously, it is necessary to:

1. Specify a definite limiting process towards the null point.
2. Ensure that the limit exists and is "well behaved."
3. Don't change limits halfway through your calculations!

This is a rather tedious and often unnecessary process, but if find yourself getting weird answers, this is the "insurance policy" so to speak. You can pretty much never get absurdities if you follow the above rules. But like all insurance, there is a price – the painful process of working out a limit.

I like Edwin Jaynes in this case:

there is no right choice or wrong choice because it is for us to say which limit we want to take; i.e., which problem we want to solve. This is Sermon #1 on mathematical limits; although it was given long ago by Kolmogorov, many who try to do probability calculations still fail to heed it and get themselves into trouble. The moral is that, unless they are defined in the statement of a problem, probabilities conditional on point values of a parameter, have no meaning until the specific limiting process is stated. More generally, probabilities conditional on any propositions of probability zero, are undefined.