Cross Entropy in PyTorch

In your example you are treating output [0, 0, 0, 1] as probabilities as required by the mathematical definition of cross entropy. But PyTorch treats them as outputs, that don’t need to sum to 1, and need to be first converted into probabilities for which it uses the softmax function. So H(p, q) becomes: Translating the output [0, 0, 0, 1] into … Read more

What is the use of train_on_batch() in keras?

For this question, it’s a simple answer from the primary author: With fit_generator, you can use a generator for the validation data as well. In general, I would recommend using fit_generator, but using train_on_batch works fine too. These methods only exist for the sake of convenience in different use cases, there is no “correct” method. train_on_batch allows you to expressly update … Read more