Cross Entropy in PyTorch

In your example you are treating output [0, 0, 0, 1] as probabilities as required by the mathematical definition of cross entropy. But PyTorch treats them as outputs, that don’t need to sum to 1, and need to be first converted into probabilities for which it uses the softmax function.

So H(p, q) becomes:

H(p, softmax(output))

Translating the output [0, 0, 0, 1] into probabilities:

softmax([0, 0, 0, 1]) = [0.1749, 0.1749, 0.1749, 0.4754]

whence:

-log(0.4754) = 0.7437

Related Posts:

  1. What’s the difference between torch.stack() and torch.cat() functions?
  2. How to initialize weights in PyTorch?
  3. How to install pytorch in Anaconda with conda or pip?
  4. Expected 2D array, got 1D array instead error
  5. Error in Python script “Expected 2D array, got 1D array instead:”?
  6. How to use torch.stack function
  7. How does the “view” method work in PyTorch?
  8. Embedding in pytorch
  9. What does “unsqueeze” do in Pytorch?
  10. Cannot convert list to array: ValueError: only one element tensors can be converted to Python scalars
  11. What is the difference between sparse_categorical_crossentropy and categorical_crossentropy?
  12. Pytorch tensor to numpy array
  13. How to fix this strange error: “RuntimeError: CUDA error: out of memory”
  14. Pytorch reshape tensor dimension
  15. What is the difference between np.mean and tf.reduce_mean?
  16. No module named “Torch”
  17. How to install pytorch in windows?
  18. What is the meaning of ‘for _ in range()
  19. ValueError: x and y must be the same size
  20. What’s the difference between scikit-learn and tensorflow? Is it possible to use them together?
  21. Scikit-learn GridSearch giving “ValueError: multiclass format is not supported” error
  22. How to implement the ReLU function in Numpy
  23. ValueError: multiclass format is not supported
  24. Find the current directory and file’s directory [duplicate]
  25. Finding the index of an item in a list
  26. How to plot a histogram using Matplotlib in Python with a list of data?
  27. Convert pandas dataframe to NumPy array
  28. Fastest way to check if a value exists in a list
  29. TypeError: can’t multiply sequence by non-int of type ‘str’
  30. Error:cannot convert float NaN to integer in pandas
  31. What does the % sign mean in coding?
  32. Replacing column values in a pandas DataFrame
  33. How can I count the occurrences of a list item?
  34. Unable to plot Double Bar, Bar plot using pyplot for ndarray
  35. matplotlib error – no module named tkinter
  36. AttributeError: module ‘urllib3’ has no attribute ‘urlopen’ in python
  37. How to download a file over HTTP?
  38. filedialog, tkinter and opening files
  39. How do I get the row count of a Pandas DataFrame?
  40. ‘list’ object has no attribute ‘shape’
  41. What does the ‘b’ character do in front of a string literal?
  42. How to add multiple values per key in python dictionary
  43. Pythonic way to combine FOR loop and IF statement
  44. Python: OverflowError: math range error
  45. Best way to replace multiple characters in a string?
  46. Convert base-2 binary number string to int
  47. setup script exited with error: command ‘x86_64-linux-gnu-gcc’ failed with exit status 1
  48. sublime text2 python error message /usr/bin/python: can’t find ‘__main__’ module in ”
  49. load csv into 2D matrix with numpy for plotting
  50. Python TypeError: cannot convert the series to when trying to do math on dataframe
  51. How do I do a case-insensitive string comparison?
  52. How to fix ‘ValueError: list.remove(x): x not in list’ error in Python
  53. Converting binary to decimal integer output
  54. Don’t understand this SyntaxError: illegal target for annotation
  55. No module named setuptools
  56. Using Queue in python
  57. Whats the difference between ‘for x in list:’ and ‘for x in list[:]:’
  58. Python: maximum recursion depth exceeded while calling a Python object
  59. Checking if a number is prime in Python
  60. Implementing an efficient queue in Python
  61. Modify the legend of pandas bar plot
  62. Python int to binary string?
  63. Full examples of using pySerial package
  64. Checking whether a variable is an integer or not
  65. Pandas DataFrame: replace all values in a column, based on condition
  66. How to create a trie in Python
  67. How to convert string to datetime format in pandas python?
  68. TypeError: expected a character buffer object
  69. “RuntimeError: Make sure the Graphviz executables are on your system’s path” after installing Graphviz 2.38
  70. Convert date to datetime in Python
  71. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  72. How to install PyQt5 on Windows?
  73. How do I get the day of week given a date?
  74. TypeError: ‘dict_keys’ object does not support indexing
  75. Add Variables to Tuple
  76. Python: Converting string into decimal number
  77. Segmentation fault: 11 in OS X
  78. Removing duplicates from a list of lists
  79. plotly: TypeError: cannot convert dictionary update sequence element #0 to a sequence
  80. Update python dictionary (add another value to existing key)
  81. scrapy run spider from script
  82. Using replace() method in python by index
  83. Python, how to handle the “ValueError: unsupported pickle protocol: 4” error?
  84. python error: TypeError: an integer is required
  85. Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
  86. Plot a histogram such that the total area of the histogram equals 1
  87. pip throws TypeError: parse() got an unexpected keyword argument ‘transport_encoding’ when trying to install new packages
  88. How to change legend size with matplotlib.pyplot
  89. An equivalent function to matplotlib.mlab.bivariate_normal
  90. Sorting arrays in NumPy by column
  91. How to open html file?
  92. In Python, is there an elegant way to print a list in a custom format without explicit looping?
  93. “Python version 2.7 required, which was not found in the registry” error when attempting to install netCDF4 on Windows 8
  94. Does tkinter have a table widget?
  95. Releasing memory in Python
  96. Text progress bar in terminal with block characters
  97. Extracting specific selected columns to new DataFrame as a copy
  98. How to read a CSV file from a URL with Python?
  99. How to print both strings in a dictionary in Python
  100. Python: OSError: [Errno 2] No such file or directory: ”

Leave a Comment