As stated in the title, I'm working on a convolutional auto encoder with RGB images, and wondering whether the loss function should be MSE, binary cross entropy, or even custom ?
Thanks
Best Answer
I think people typically use binary cross entropy (BCE; aka the cost/loss function used in vanilla logistic regression) since images are typically scaled in a range from 0 to 1. (E.g., by dividing each pixel by 255.; that's the cheapest and most trivial way.)
However, I am not sure if that's the most intuitive loss function to use, since even if you feed identical images, you will get a mean BCE of ~0.5. That's because the typical implementation goes like this:
y * log(y_pred) - (1 - y) * log(1 - y_pred) In e.g., binary classification, either of the terms [y * log(y_pred)] or [(1 – y) * log(1 – y_pred)] will be zero. That's not the case when you compare two images, because you have continuous values.
I just plotted it for comparing 2 pixels out of curiosity:
In comparison, the squared loss:
In practice though, I never noticed any difference between BCE and MSE in terms of the results. So, I'd say either of them is fine.
Similar Posts:
- Solved – a binary loss, and should I use a binary loss or a softmax loss for classification
- Solved – a binary loss, and should I use a binary loss or a softmax loss for classification
- Solved – Cross-entropy for comparing images
- Solved – Understanding cross entropy in neural networks
- Solved – Validation loss fluctuating while training the neural network in tensorflow