What’s the difference between UTF-8 and UTF-8 without BOM?

The UTF-8 BOM is a sequence of bytes at the start of a text stream (0xEF, 0xBB, 0xBF) that allows the reader to more reliably guess a file as being encoded in UTF-8.

Normally, the BOM is used to signal the endianness of an encoding, but since endianness is irrelevant to UTF-8, the BOM is unnecessary.

According to the Unicode standard, the BOM for UTF-8 files is not recommended:

2.6 Encoding Schemes

… Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature. See the “Byte Order Mark” subsection in Section 16.8, Specials, for more information.

Related Posts:

  1. u’\ufeff’ in Python string
  2. Using awk to remove the Byte-order mark
  3. python encoding utf-8
  4. What is the difference between UTF-8 and Unicode?
  5. Convert Unicode to ASCII without errors in Python
  6. What is the difference between UTF-8 and ISO-8859-1?
  7. Tallest Unicode character?
  8. How to remove \xa0 from string in Python?
  9. How to remove this \xa0 from a string in python?
  10. What’s the difference between ASCII and Unicode?
  11. What’s up with these Unicode combining characters and how can we filter them?
  12. Unicode range for Japanese
  13. Using unicode character u201c
  14. Getting ’ instead of an apostrophe(‘) in PHP
  15. What is a unicode string?
  16. What Unicode characters represent “time”?
  17. Javascript: Unicode string to hex
  18. Python decoding Unicode is not supported
  19. List of all unicode’s open/close brackets?
  20. Japanese ASCII Code
  21. HTML encoding issues – “” character showing up instead of ” “
  22. Best way to convert string to bytes in Python 3?
  23. Best way to convert string to bytes in Python 3?
  24. How to resolve TypeError: can only concatenate str (not “int”) to str [duplicate]
  25. How to correct TypeError: Unicode-objects must be encoded before hashing?
  26. What character encoding is >?
  27. How to replace � in a string
  28. Invisible characters – ASCII
  29. What’s the difference between ASCII and Unicode?
  30. What’s the difference between ASCII and Unicode?
  31. error UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xff in position 0: invalid start byte
  32. Is there an upside down caret character?
  33. Is “ ” a replacement of ” “?
  34. Python – Reading and writing csv files with utf-8 encoding
  35. What does the ‘b’ character do in front of a string literal?
  36. What does the ‘b’ character do in front of a string literal?
  37. Meaning of –
  38. (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape
  39. SSIS Convert Between Unicode and Non-Unicode Error
  40. Elegant way to search for UTF-8 files with BOM?
  41. UnicodeDecodeError: ‘charmap’ codec can’t decode byte X in position Y: character maps to
  42. “for line in…” results in UnicodeDecodeError: ‘utf-8’ codec can’t decode byte
  43. (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape
  44. What’s the complete range for Chinese characters in Unicode?
  45. “Unicode Error “unicodeescape” codec can’t decode bytes… Cannot open text files in Python 3
  46. “Unmappable character for encoding UTF-8” error
  47. Python – ‘ascii’ codec can’t decode byte
  48. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  49. Is it possible to force Excel recognize UTF-8 CSV files automatically?
  50. std::wstring VS std::string
  51. All inclusive Charset to avoid “java.nio.charset.MalformedInputException: Input length = 1”?
  52. Strange Characters in database text: Ã, Ã, ¢, â‚ €,
  53. Python string prints as [u’String’]
  54. NameError: global name ‘unicode’ is not defined – in Python 3
  55. Convert a Unicode string to a string in Python (containing extra symbols)python string unicode type-conversion
  56. Best way to reverse a string
  57. Read a file line by line with VB.NET
  58. UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0x80 in position 3131: invalid start byte
  59. get char value in java
  60. How well is Unicode supported in C++11?
  61. TypeError: coercing to Unicode: need string or buffer, int found
  62. How to open html file?
  63. UnicodeDecodeError, invalid continuation byte
  64. ERROR 1115 (42000): Unknown character set: ‘utf8mb4’
  65. What exactly do “u” and “r” string flags do, and what are raw string literals?
  66. What is a vertical tab?
  67. Differences between utf8 and latin1
  68. Byte and char conversion in Java
  69. How to write unicode strings into a file?
  70. What does _T stands for in a CString
  71. JSON and escaping characters
  72. is there any plugin for auto taging with unicode support for wordpress?

Leave a Comment

tech