Something like PhantomJS may be more robust, but here’s some basic Python code to extract it the full menu: After that, you’ll want to search through the menu for the date you’re interested in. EDIT: Some overkill on my part:
Try this from bs4 import BeautifulSoup This might be a problem with Beautiful Soup, version 4, and the beta days. I just read this from the homepage.
I wanted to write a piece of code like the following: But I found that I have to install urllib3 package now. Moreover, I couldn’t find any tutorial or example to understand how to rewrite the above code, for example, urllib3 does not have urlopen. Any explanation or example, please?! P/S: I’m using python 3.4.
Nope, BeautifulSoup, by itself, does not support XPath expressions. An alternative library, lxml, does support XPath 1.0. It has a BeautifulSoup compatible mode where it’ll try and parse broken HTML the way Soup does. However, the default lxml HTML parser does just as good a job of parsing broken HTML, and I believe is faster. … Read more
Here you go: This gives you: Couple of things to note: The last row in the output above, the Payment Amount is not a part of the table but that is how the table is laid out. You can filter it out by checking if the length of the list is less than 7. The … Read more
I fixed it by adding .encode(“utf-8”) to soup. That means that print(soup) becomes print(soup.encode(“utf-8”)).
It’s pretty simple, use the following – Leave a comment if anything is not clear.
The simple answer, imo, is that if you trust your source to be well-formed, go with the lxml solution. Otherwise, BeautifulSoup all the way. Edit: This answer is three years old now; it’s worth noting, as Jonathan Vanasco does in the comments, that BeautifulSoup4 now supports using lxml as the internal parser, so you can … Read more
You should be using pip to install, so you can simply do That will install the latest BS4, which is 4.3.1 as of 2013-08-15. It supports Python 3.