Best way to replace multiple characters in a string?

Replacing two characters

I timed all the methods in the current answers along with one extra.

With an input string of abc&def#ghi and replacing & -> \& and # -> \#, the fastest way was to chain together the replacements like this: text.replace('&', '\&').replace('#', '\#').

Timings for each function:

  • a) 1000000 loops, best of 3: 1.47 μs per loop
  • b) 1000000 loops, best of 3: 1.51 μs per loop
  • c) 100000 loops, best of 3: 12.3 μs per loop
  • d) 100000 loops, best of 3: 12 μs per loop
  • e) 100000 loops, best of 3: 3.27 μs per loop
  • f) 1000000 loops, best of 3: 0.817 μs per loop
  • g) 100000 loops, best of 3: 3.64 μs per loop
  • h) 1000000 loops, best of 3: 0.927 μs per loop
  • i) 1000000 loops, best of 3: 0.814 μs per loop

Here are the functions:

def a(text):
    chars = "&#"
    for c in chars:
        text = text.replace(c, "\\" + c)


def b(text):
    for ch in ['&','#']:
        if ch in text:
            text = text.replace(ch,"\\"+ch)


import re
def c(text):
    rx = re.compile('([&#])')
    text = rx.sub(r'\\\1', text)


RX = re.compile('([&#])')
def d(text):
    text = RX.sub(r'\\\1', text)


def mk_esc(esc_chars):
    return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s])
esc = mk_esc('&#')
def e(text):
    esc(text)


def f(text):
    text = text.replace('&', '\&').replace('#', '\#')


def g(text):
    replacements = {"&": "\&", "#": "\#"}
    text = "".join([replacements.get(c, c) for c in text])


def h(text):
    text = text.replace('&', r'\&')
    text = text.replace('#', r'\#')


def i(text):
    text = text.replace('&', r'\&').replace('#', r'\#')

Timed like this:

python -mtimeit -s"import time_functions" "time_functions.a('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.b('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.c('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.d('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.e('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.f('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.g('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.h('abc&def#ghi')"
python -mtimeit -s"import time_functions" "time_functions.i('abc&def#ghi')"

Replacing 17 characters

Here’s similar code to do the same but with more characters to escape (\`*_{}>#+-.!$):

def a(text):
    chars = "\\`*_{}[]()>#+-.!$"
    for c in chars:
        text = text.replace(c, "\\" + c)


def b(text):
    for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:
        if ch in text:
            text = text.replace(ch,"\\"+ch)


import re
def c(text):
    rx = re.compile('([&#])')
    text = rx.sub(r'\\\1', text)


RX = re.compile('([\\`*_{}[]()>#+-.!$])')
def d(text):
    text = RX.sub(r'\\\1', text)


def mk_esc(esc_chars):
    return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s])
esc = mk_esc('\\`*_{}[]()>#+-.!$')
def e(text):
    esc(text)


def f(text):
    text = text.replace('\\', '\\\\').replace('`', '\`').replace('*', '\*').replace('_', '\_').replace('{', '\{').replace('}', '\}').replace('[', '\[').replace(']', '\]').replace('(', '\(').replace(')', '\)').replace('>', '\>').replace('#', '\#').replace('+', '\+').replace('-', '\-').replace('.', '\.').replace('!', '\!').replace('$', '\$')


def g(text):
    replacements = {
        "\\": "\\\\",
        "`": "\`",
        "*": "\*",
        "_": "\_",
        "{": "\{",
        "}": "\}",
        "[": "\[",
        "]": "\]",
        "(": "\(",
        ")": "\)",
        ">": "\>",
        "#": "\#",
        "+": "\+",
        "-": "\-",
        ".": "\.",
        "!": "\!",
        "$": "\$",
    }
    text = "".join([replacements.get(c, c) for c in text])


def h(text):
    text = text.replace('\\', r'\\')
    text = text.replace('`', r'\`')
    text = text.replace('*', r'\*')
    text = text.replace('_', r'\_')
    text = text.replace('{', r'\{')
    text = text.replace('}', r'\}')
    text = text.replace('[', r'\[')
    text = text.replace(']', r'\]')
    text = text.replace('(', r'\(')
    text = text.replace(')', r'\)')
    text = text.replace('>', r'\>')
    text = text.replace('#', r'\#')
    text = text.replace('+', r'\+')
    text = text.replace('-', r'\-')
    text = text.replace('.', r'\.')
    text = text.replace('!', r'\!')
    text = text.replace('$', r'\$')


def i(text):
    text = text.replace('\\', r'\\').replace('`', r'\`').replace('*', r'\*').replace('_', r'\_').replace('{', r'\{').replace('}', r'\}').replace('[', r'\[').replace(']', r'\]').replace('(', r'\(').replace(')', r'\)').replace('>', r'\>').replace('#', r'\#').replace('+', r'\+').replace('-', r'\-').replace('.', r'\.').replace('!', r'\!').replace('$', r'\$')

Here’s the results for the same input string abc&def#ghi:

  • a) 100000 loops, best of 3: 6.72 μs per loop
  • b) 100000 loops, best of 3: 2.64 μs per loop
  • c) 100000 loops, best of 3: 11.9 μs per loop
  • d) 100000 loops, best of 3: 4.92 μs per loop
  • e) 100000 loops, best of 3: 2.96 μs per loop
  • f) 100000 loops, best of 3: 4.29 μs per loop
  • g) 100000 loops, best of 3: 4.68 μs per loop
  • h) 100000 loops, best of 3: 4.73 μs per loop
  • i) 100000 loops, best of 3: 4.24 μs per loop

And with a longer input string (## *Something* and [another] thing in a longer sentence with {more} things to replace$):

  • a) 100000 loops, best of 3: 7.59 μs per loop
  • b) 100000 loops, best of 3: 6.54 μs per loop
  • c) 100000 loops, best of 3: 16.9 μs per loop
  • d) 100000 loops, best of 3: 7.29 μs per loop
  • e) 100000 loops, best of 3: 12.2 μs per loop
  • f) 100000 loops, best of 3: 5.38 μs per loop
  • g) 10000 loops, best of 3: 21.7 μs per loop
  • h) 100000 loops, best of 3: 5.7 μs per loop
  • i) 100000 loops, best of 3: 5.13 μs per loop

Adding a couple of variants:

def ab(text):
    for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:
        text = text.replace(ch,"\\"+ch)


def ba(text):
    chars = "\\`*_{}[]()>#+-.!$"
    for c in chars:
        if c in text:
            text = text.replace(c, "\\" + c)

With the shorter input:

  • ab) 100000 loops, best of 3: 7.05 μs per loop
  • ba) 100000 loops, best of 3: 2.4 μs per loop

With the longer input:

  • ab) 100000 loops, best of 3: 7.71 μs per loop
  • ba) 100000 loops, best of 3: 6.08 μs per loop

So I’m going to use ba for readability and speed.

Addendum

Prompted by haccks in the comments, one difference between ab and ba is the if c in text: check. Let’s test them against two more variants:

def ab_with_check(text):
    for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:
        if ch in text:
            text = text.replace(ch,"\\"+ch)

def ba_without_check(text):
    chars = "\\`*_{}[]()>#+-.!$"
    for c in chars:
        text = text.replace(c, "\\" + c)

Times in μs per loop on Python 2.7.14 and 3.6.3, and on a different machine from the earlier set, so cannot be compared directly.

╭────────────╥──────┬───────────────┬──────┬──────────────────╮
│ Py, input  ║  ab  │ ab_with_check │  ba  │ ba_without_check │
╞════════════╬══════╪═══════════════╪══════╪══════════════════╡
│ Py2, short ║ 8.81 │    4.22       │ 3.45 │    8.01          │
│ Py3, short ║ 5.54 │    1.34       │ 1.46 │    5.34          │
├────────────╫──────┼───────────────┼──────┼──────────────────┤
│ Py2, long  ║ 9.3  │    7.15       │ 6.85 │    8.55          │
│ Py3, long  ║ 7.43 │    4.38       │ 4.41 │    7.02          │
└────────────╨──────┴───────────────┴──────┴──────────────────┘

We can conclude that:

  • Those with the check are up to 4x faster than those without the check
  • ab_with_check is slightly in the lead on Python 3, but ba (with check) has a greater lead on Python 2
  • However, the biggest lesson here is Python 3 is up to 3x faster than Python 2! There’s not a huge difference between the slowest on Python 3 and fastest on Python 2!

Related Posts:

  1. Best way to replace multiple characters in a string?
  2. Remove specific characters from a string in Python
  3. How do I lowercase a string in Python?
  4. How do I get a substring of a string in Python?
  5. Renaming column names in Pandas
  6. How to read a file line-by-line into a list?
  7. strip(char) on a string
  8. How to convert list to string [duplicate]
  9. TypeError: a bytes-like object is required, not ‘str’ when writing to a file in Python3
  10. Replace ‘ with \’ in a string
  11. Print string to text file
  12. Best way to convert string to bytes in Python 3?
  13. Pythonic way to create a long multi-line string
  14. Best way to convert string to bytes in Python 3?
  15. Does Python have a string ‘contains’ substring method?
  16. TypeError: not all arguments converted during string formatting python
  17. TypeError: ‘float’ object is not subscriptable
  18. How do I append one string to another in Python?
  19. TypeError: ‘float’ object is not subscriptable
  20. How do I append one string to another in Python?
  21. Random string generation with upper case letters and digits
  22. How can I split and parse a string in Python?
  23. How to read a text file into a string variable and strip newlines?
  24. How to print like printf in Python3?
  25. Python convert tuple to string
  26. write() versus writelines() and concatenated strings
  27. Why is it string.join(list) instead of list.join(string)?
  28. Why is it string.join(list) instead of list.join(string)?
  29. Split a string into 2 in Python
  30. Best way to strip punctuation from a string
  31. Split string with multiple delimiters in Python [duplicate]
  32. Finding and replacing elements in a list
  33. How to check if the string is empty?
  34. How to find all occurrences of a substring?
  35. How do I convert a list of ascii values to a string in python?
  36. How to check whether a str(variable) is empty or not?
  37. Replace values in list using Python
  38. What does the ‘b’ character do in front of a string literal?
  39. How to add a string in a certain position?
  40. Check string “None” or “not” in Python 2.7
  41. Remove all special characters, punctuation and spaces from string
  42. This can be done without regex:
  43. Convert columns to string in Pandas
  44. How to delete a character from a string using Python
  45. Remove the first character of a string
  46. Convert hex to binary
  47. Quoting backslashes in Python string literals
  48. Python string class like StringBuilder in C#?
  49. AttributeError: ‘list’ object has no attribute ‘replace’ when trying to remove character
  50. How to check if a string is a substring of items in a list of strings?
  51. How do I remove a substring from the end of a string?
  52. How can I splice a string?
  53. Detect whether a Python string is a number or a letter
  54. Replace and overwrite instead of appending
  55. Finding all possible permutations of a given string in python
  56. How do I append one string to another in Python?
  57. AttributeError: ‘list’ object has no attribute ‘replace’ when trying to remove character
  58. AttributeError: ‘list’ object has no attribute ‘replace’ when trying to remove character
  59. How to convert string to binary?
  60. Converting from a string to boolean in Python?
  61. How to check if type of a variable is string?
  62. Count the number of occurrences of a character in a string
  63. How do I check if there are duplicates in a flat list?
  64. How can I print variable and string on same line in Python?
  65. Remove quotes from String in Python
  66. Convert hex string to int in Python
  67. How can I check if character in a string is a letter? (Python)
  68. How can I find the first occurrence of a sub-string in a python string?
  69. How to concatenate items in a list to a single string?
  70. Replacing a character from a certain index
  71. AttributeError: ‘list’ object has no attribute ‘lower’ gensim
  72. Find the nth occurrence of substring in a string
  73. How to convert ‘binary string’ to normal string in Python3?
  74. Properly formatted multiplication table
  75. How to get a function name as a string?
  76. Pandas error “Can only use .str accessor with string values”
  77. How to compare individual characters in two strings in Python 3
  78. Convert a Unicode string to a string in Python (containing extra symbols)python string unicode type-conversion
  79. Create nice column output in python
  80. How to replace multiple substrings of a string?
  81. How to repeat a string with spaces?
  82. TypeError: decoding str is not supported
  83. AttributeError: ‘str’ object has no attribute ‘items’
  84. How to pad zeroes to a string?
  85. How does str(list) work?
  86. What is the difference between a string and a byte string?
  87. How do I convert a string to a double in Python?
  88. How can I copy a Python string?
  89. How to strip a specific word from a string?
  90. Checking if a string can be converted to float in Python
  91. How to get ° character in a string in python?
  92. Python cant convert ‘list’ object to str error [closed]
  93. Using strip() to clean up a string
  94. Mutable strings in Python
  95. Python code to remove HTML tags from a string 
  96. Cannot concatenate ‘str’ and ‘float’ objects?
  97. How to repeat individual characters in strings in Python
  98. When to use %r instead of %s in Python? [duplicate]
  99. Python – difference between two strings
  100. TypeError: coercing to Unicode: need string or buffer

Leave a Comment