This is probably because of mod_security
or some similar server security feature which blocks known spider/bot user agents (urllib
uses something like python urllib/3.3.0
, it’s easily detected). Try setting a known browser user agent with:
from urllib.request import Request, urlopen req = Request('http://www.cmegroup.com/trading/products/#sortField=oi&sortAsc=false&venues=3&page=1&cleared=1&group=1', headers={'User-Agent': 'Mozilla/5.0'}) webpage = urlopen(req).read()
This works for me.
By the way, in your code you are missing the ()
after .read
in the urlopen
line, but I think that it’s a typo.
TIP: since this is exercise, choose a different, non restrictive site. Maybe they are blocking urllib
for some reason…
Related Posts:
- Problem HTTP error 403 in Python 3 Web Scraping
- Should a 502 HTTP status code be used if a proxy receives no response at all?
- Web scraping redoc web api
- What is the difference between a URI, a URL and a URN?
- HTTP Status 504
- What is the difference between POST and GET? [duplicate]
- Do I need Content-Type: application/octet-stream for file download?
- application/x-www-form-urlencoded or multipart/form-data?
- application/x-www-form-urlencoded or multipart/form-data?
- Is 418 “I’m a teapot” really an HTTP response code?
- How to download a file over HTTP?
- How to define the basic HTTP authentication using cURL correctly?
- How to download a file over HTTP?
- How to define the basic HTTP authentication using cURL correctly?
- “Cannot GET /” with Connect on Node.js
- What is the meaning of [:] in python [duplicate]
- “CAUTION: provisional headers are shown” in Chrome debugger
- What’s the difference between a POST and a PUT HTTP REQUEST?
- How do I send a POST request with PHP?
- Why is it said that “HTTP is a stateless protocol”?
- What’s the difference between using application/csv vs text/csv? [duplicate]
- What are all the possible values for HTTP “Content-Type” header?
- How to find elements by class
- What is the difference between PUT, POST and PATCH?
- urllib2.HTTPError: HTTP Error 403: Forbidden
- What is the quickest way to HTTP GET in Python?
- What’s the difference between “Request Payload” vs “Form Data” as seen in Chrome dev tools Network tab
- Exception in thread “main” java.net.NoRouteToHostException: No route to host
- ndroid 8: Cleartext HTTP traffic not permitted
- How to save an image locally using Python whose URL address I already know?
- can we use XPath with BeautifulSoup?
- Can PHP cURL retrieve response headers AND body in a single request?
- Setting Curl’s Timeout in PHP
- How are parameters sent in an HTTP POST request?
- What should I use to open a url instead of urlopen in urllib3
- Why am I suddenly getting a “Blocked loading mixed active content” issue in Firefox?
- wget: unable to resolve host address `http’
- Are HTTP headers case-sensitive?
- When looking at the differences between X-Auth-Token vs Authorization headers, which is preferred?
- WordPress HTTP parameter pollution
- Does WordPress send data about your blog to WordPress.org or Automattic?
- Hiding WordPress REST API v2 endpoints from public viewing
- Does WordPress only support HTTP 1.1?
- How do I troubleshoot responses with WP HTTP API?
- Is curl required?
- The resource was preloaded using link preload but not used within a few seconds
- using wp_remote_get to retrieve own url on local host
- Using wp-cron in backpress – problems with wp_remote_post, fsockopen error
- Running index.php from command line & load balancer health checks
- Enable CORS in wordpress
- Change port of wordpress
- How to get value of custom http header?
- Several times request to load plugins when sending one request
- why is $_REQUESt[‘redirect_to’] empty?
- Get “HTTP/1.1 406 Not Acceptable” when accesing my website with Delphi Indy Control
- WordPress HTTP 500 Error “page isn’t working”
- What’s the point in having “www” in a URL?
- For what is the “.well-known”-folder?
- Human readable format for http headers with tcpdump
- How to make wireshark filter POST-requests only?
- Image file urls still point to http instead of https
- How can I reverse a list in Python?
- How to use filter, map, and reduce in Python 3
- Updating a dictionary in python
- anaconda update all possible packages?
- How to replace NaN values by Zeroes in a column of a Pandas Dataframe?
- TypeError: cannot perform reduce with flexible type
- Python regex AttributeError: ‘NoneType’ object has no attribute ‘group’
- syntaxerror: “unexpected character after line continuation character in python” math
- ValueError : I/O operation on closed file
- How can I remove a key from a Python dictionary?
- Matplotlib automatic legend outside plot
- How to compare two dates?
- Python circular importing?
- Is there a difference between `continue` and `pass` in a for loop in python?
- Create own colormap using matplotlib and plot color scale
- n-grams in python, four, five, six grams?
- The difference between ‘+=’ and ‘=+’?
- Python TypeError must be str not int
- “Unicode Error “unicodeescape” codec can’t decode bytes… Cannot open text files in Python 3
- If Python is interpreted, what are .pyc files?
- Pandas join issue: columns overlap but no suffix specified
- How to downgrade python from 3.7 to 3.6
- How to turn on line numbers in IDLE?
- OpenCV NoneType object has no attribute shape
- ImportError: libGL.so.1: cannot open shared object file: No such file or directory
- Not able to pip install pickle in python 3.6
- How to print Unicode character in Python?
- Coalesce values from 2 columns into a single column in a pandas dataframe
- Unknown format code ‘f’ for object of type ‘str’- Folium
- Pyinstaller setting icons don’t change
- No module named ‘tqdm’
- What is the purpose of the single underscore “_” variable in Python?
- Must have equal len keys and value when setting with an iterable
- print(__doc__) in Python 3 script
- Convert a 1D array to a 2D array in numpy
- How do I represent and work with n-bit vectors in Python?
- How to repeat individual characters in strings in Python
- Last-Modified header support doesnât speed up server processing – want more
- Does WordPress perform better with curl installed?