python BeautifulSoup parsing table

Here you go:

data = []
table = soup.find('table', attrs={'class':'lineItemsTable'})
table_body = table.find('tbody')

rows = table_body.find_all('tr')
for row in rows:
    cols = row.find_all('td')
    cols = [ele.text.strip() for ele in cols]
    data.append([ele for ele in cols if ele]) # Get rid of empty values

This gives you:

[ [u'1359711259', u'SRF', u'08/05/2013', u'5310 4 AVE', u'K', u'19', u'125.00', u'$'], 
  [u'7086775850', u'PAS', u'12/14/2013', u'3908 6th Ave', u'K', u'40', u'125.00', u'$'], 
  [u'7355010165', u'OMT', u'12/14/2013', u'3908 6th Ave', u'K', u'40', u'145.00', u'$'], 
  [u'4002488755', u'OMT', u'02/12/2014', u'NB 1ST AVE @ E 23RD ST', u'5', u'115.00', u'$'], 
  [u'7913806837', u'OMT', u'03/03/2014', u'5015 4th Ave', u'K', u'46', u'115.00', u'$'], 
  [u'5080015366', u'OMT', u'03/10/2014', u'EB 65TH ST @ 16TH AV E', u'7', u'50.00', u'$'], 
  [u'7208770670', u'OMT', u'04/08/2014', u'333 15th St', u'K', u'70', u'65.00', u'$'], 
  [u'$0.00\n\n\nPayment Amount:']
]

Couple of things to note:

  • The last row in the output above, the Payment Amount is not a part of the table but that is how the table is laid out. You can filter it out by checking if the length of the list is less than 7.
  • The last column of every row will have to be handled separately since it is an input text box.

Related Posts:

  1. Where is BeautifulSoup4 hiding?
  2. How to remove \xa0 from string in Python?
  3. What is the meaning of [:] in python [duplicate]
  4. Understand the Find() function in Beautiful Soup
  5. ImportError: No Module Named bs4 (BeautifulSoup)
  6. Difference between BeautifulSoup and Scrapy crawler?
  7. Using BeautifulSoup to search HTML for string
  8. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  9. BeautifulSoup getting href
  10. How to find elements by class
  11. How can I from bs4 import BeautifulSoup?
  12. BeautifulSoup and lxml.html – what to prefer?
  13. Get an attribute value based on the name attribute with BeautifulSoup
  14. UnicodeEncodeError: ‘charmap’ codec can’t encode characters
  15. can we use XPath with BeautifulSoup?
  16. What should I use to open a url instead of urlopen in urllib3
  17. Can I remove script tags with BeautifulSoup?
  18. ImportError: No module named BeautifulSoup
  19. Parse the JavaScript returned from BeautifulSoup
  20. Truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()
  21. ImportError: No module named pip
  22. TypeError: ‘int’ object is not callable
  23. What exactly does numpy.exp() do? [closed]
  24. TypeError: ‘int’ object is not callable
  25. Is a Python dictionary an example of a hash table?
  26. TypeError: unhashable type: ‘dict’, when dict used as a key for another dict
  27. Importing files from different folder
  28. ModuleNotFoundError: No module named ‘sklearn’
  29. Python arrays used as indices must be of integer (or boolean) type
  30. What does ** (double star/asterisk) and * (star/asterisk) do for parameters?
  31. Conda uninstall one package and one package only
  32. Cannot find module cv2 when using OpenCV
  33. Getting TypeError: __init__() missing 1 required positional argument: ‘on_delete’ when trying to add parent table after child table with entries
  34. TypeError: list indices must be integers, not str (boolean convertion actually)
  35. ImportError: DLL load failed: %1 is not a valid Win32 application. But the DLL’s are there
  36. Pandas “Can only compare identically-labeled DataFrame objects” error
  37. ImportError: DLL load failed: %1 is not a valid Win32 application. But the DLL’s are there
  38. SyntaxError: multiple statements found while compiling a single statement
  39. How can I remove a trailing newline?
  40. Convert base-2 binary number string to int
  41. setup script exited with error: command ‘x86_64-linux-gnu-gcc’ failed with exit status 1
  42. sublime text2 python error message /usr/bin/python: can’t find ‘__main__’ module in ”
  43. load csv into 2D matrix with numpy for plotting
  44. Python TypeError: cannot convert the series to when trying to do math on dataframe
  45. How do I do a case-insensitive string comparison?
  46. How to fix ‘ValueError: list.remove(x): x not in list’ error in Python
  47. How can I convert radians to degrees with Python?
  48. How to initialize a two-dimensional array in Python?
  49. Python: maximum recursion depth exceeded while calling a Python object
  50. Checking if a number is prime in Python
  51. Implementing an efficient queue in Python
  52. Modify the legend of pandas bar plot
  53. Python int to binary string?
  54. Full examples of using pySerial package
  55. Checking whether a variable is an integer or not
  56. Using headers with the Python requests library’s get method
  57. How to measure elapsed time in Python?
  58. Issues using Keras np_utils.to_categorical
  59. Flask ImportError: No Module Named Flask
  60. How to convert string to datetime format in pandas python?
  61. TypeError: expected a character buffer object
  62. “RuntimeError: Make sure the Graphviz executables are on your system’s path” after installing Graphviz 2.38
  63. Convert date to datetime in Python
  64. How to install PyQt5 on Windows?
  65. How do I get the day of week given a date?
  66. TypeError: ‘dict_keys’ object does not support indexing
  67. Add Variables to Tuple
  68. Removing duplicates from a list of lists
  69. plotly: TypeError: cannot convert dictionary update sequence element #0 to a sequence
  70. Update python dictionary (add another value to existing key)
  71. scrapy run spider from script
  72. Using replace() method in python by index
  73. Python, how to handle the “ValueError: unsupported pickle protocol: 4” error?
  74. python error: TypeError: an integer is required
  75. Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
  76. getting ValueError : “Can only tuple-index with a MultiIndex “
  77. ModuleNotFoundError: No module named ‘seaborn’ in Python IDE
  78. Getting the name of a variable as a string
  79. can’t multiply sequence by non-int of type ‘list’
  80. cannot convert the series to
  81. Converting a RGB color tuple to a six digit code
  82. mean, nanmean and warning: Mean of empty slice
  83. SystemError: new style getargs format but argument is not a tuple?
  84. Python Set Comprehension
  85. How to use cmp() in Python 3?
  86. Python loop counter in a for loop
  87. ImportError: No module named ‘pymongo’
  88. UnicodeDecodeError, invalid continuation byte
  89. Circular list iterator in Python
  90. creating sets of tuples in python
  91. Using the AND and NOT Operator in Python [duplicate]
  92. How can I make one python file run another? [duplicate]
  93. Error: No module named ‘fcntl’
  94. “Can’t convert ‘float’ object to str implicitly”
  95. How can I convert a datetime object to milliseconds since epoch (unix time) in Python?
  96. Find index of last occurrence of a substring in a string
  97. Multiplication function with recursion in Python
  98. How to change the Spyder editor background to dark?
  99. Python convert tuple to string
  100. Does Python have an immutable list?

Leave a Comment