Requests, Timeout, and Odd Traceback

Post by Saul Shanabrook

I am working on a Python API for DC campaign finance records, and I started getting an odd error with Requests.

Traceback (most recent call last):  
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 493, in urlopen
    body=body, headers=headers)
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 291, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1066, in request
    self._send_request(method, url, body, headers)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1104, in _send_request
    self.endheaders(body)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1062, in endheaders
    self._send_output(message_body)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 907, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 842, in send
    self.connect()
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 106, in connect
    conn = self._new_conn()
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 90, in _new_conn
    (self.host, self.port), self.timeout, *extra_args)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 509, in create_connection
    raise err
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 500, in create_connection
    sock.connect(sa)
OSError: [Errno 22] Invalid argument

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/adapters.py", line 327, in send
    timeout=timeout
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 543, in urlopen
    raise MaxRetryError(self, url, e)
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='httpbin.org', port=80): Max retries exceeded with url: /get (Caused by <class 'OSError'>: [Errno 22] Invalid argument)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
  File "test_error.py", line 4, in <module>
    r = requests.get('http://httpbin.org/get', timeout=99**99)
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/Users/saul/.virtualenvs/finance-scraper3/lib/python3.4/site-packages/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='httpbin.org', port=80): Max retries exceeded with url: /get (Caused by <class 'OSError'>: [Errno 22] Invalid argument)  

I had might quite a few changes and I wasn't sure exactly what part was raising this error. I decided to raise this issue on the Requests Github issue tracker, and proceded to figure out what part of my code was actually raising the exception.

I realized that I could reproduce the exception with these two lines

import requests

r = requests.get('http://httpbin.org/get', timeout=99**99)  

I had set the timeout to a high value in my code, to try to avoid timeout errors. So if you get this exception, it might because you set a timeout too high.

The requests maintainers were very responsive and helpful, explaining that this error is raised in the socket python C Code and they just propogate up the exction.