Quickstart

Eager to get started? This page gives a good introduction in how to get started with Requests. This assumes you already have Requests installed. If you do not, head over to the Installation section.

First, make sure that:

Let’s get started with some simple examples.

Make a Request

Making a request with Requests is very simple.

Begin by importing the Requests module:

>>> import requests

Now, let’s try to get a webpage. For this example, let’s get GitHub’s public timeline

>>> r = requests.get('https://github.com/timeline.json')

Now, we have a Response object called r. We can get all the information we need from this object.

Requests’ simple API means that all forms of HTTP request are as obvious. For example, this is how you make an HTTP POST request:

>>> r = requests.post("http://httpbin.org/post")

Nice, right? What about the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:

>>> r = requests.put("http://httpbin.org/put")
>>> r = requests.delete("http://httpbin.org/delete")
>>> r = requests.head("http://httpbin.org/get")
>>> r = requests.options("http://httpbin.org/get")

That’s all well and good, but it’s also only the start of what Requests can do.

Passing Parameters In URLs

You often want to send some sort of data in the URL’s query string. If you were constructing the URL by hand, this data would be given as key/value pairs in the URL after a question mark, e.g. httpbin.org/get?key=val. Requests allows you to provide these arguments as a dictionary, using the params keyword argument. As an example, if you wanted to pass key1=value1 and key2=value2 to httpbin.org/get, you would use the following code:

>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get("http://httpbin.org/get", params=payload)

You can see that the URL has been correctly encoded by printing the URL:

>>> print r.url
u'http://httpbin.org/get?key2=value2&key1=value1'

Response Content

We can read the content of the server’s response. Consider the GitHub timeline again:

>>> import requests
>>> r = requests.get('https://github.com/timeline.json')
>>> r.text
'[{"repository":{"open_issues":0,"url":"https://github.com/...

Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded.

When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by Requests is used when you access r.text. You can find out what encoding Requests is using, and change it, using the r.encoding property:

>>> r.encoding
'utf-8'
>>> r.encoding = 'ISO-8859-1'

If you change the encoding, Requests will use the new value of r.encoding whenever you call r.text.

Requests will also use custom encodings in the event that you need them. If you have created your own encoding and registered it with the codecs module, you can simply use the codec name as the value of r.encoding and Requests will handle the decoding for you.

Binary Response Content

You can also access the response body as bytes, for non-text requests:

>>> r.content
b'[{"repository":{"open_issues":0,"url":"https://github.com/...

The gzip and deflate transfer-encodings are automatically decoded for you.

For example, to create an image from binary data returned by a request, you can use the following code:

>>> from PIL import Image
>>> from StringIO import StringIO
>>> i = Image.open(StringIO(r.content))

JSON Response Content

There’s also a builtin JSON decoder, in case you’re dealing with JSON data:

>>> import requests
>>> r = requests.get('https://github.com/timeline.json')
>>> r.json
[{u'repository': {u'open_issues': 0, u'url': 'https://github.com/...

In case the JSON decoding fails, r.json simply returns None.

Raw Response Content

In the rare case that you’d like to get the raw socket response from the server, you can access r.raw. If you want to do this, make sure you set stream=True in your initial request. Once you do, you can do this:

>>> r = requests.get('https:/github.com/timeline.json', stream=True)
>>> r.raw
<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'

Custom Headers

If you’d like to add HTTP headers to a request, simply pass in a dict to the headers parameter.

For example, we didn’t specify our content-type in the previous example:

>>> import json
>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}
>>> headers = {'content-type': 'application/json'}

>>> r = requests.post(url, data=json.dumps(payload), headers=headers)

More complicated POST requests

Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made:

>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("http://httpbin.org/post", data=payload)
>>> print r.text
{
  // ...snip... //
  "form": {
    "key2": "value2",
    "key1": "value1"
  },
  // ...snip... //
}

There are many times that you want to send data that is not form-encoded. If you pass in a string instead of a dict, that data will be posted directly.

For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data:

>>> import json
>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}

>>> r = requests.post(url, data=json.dumps(payload))

POST a Multipart-Encoded File

Requests makes it simple to upload Multipart-encoded files:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}

>>> r = requests.post(url, files=files)
>>> r.text
{
  // ...snip... //
  "files": {
    "file": "<censored...binary...data>"
  },
  // ...snip... //
}

You can set the filename explicitly:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'))}

>>> r = requests.post(url, files=files)
>>> r.text
{
  // ...snip... //
  "files": {
    "file": "<censored...binary...data>"
  },
  // ...snip... //
}

If you want, you can send strings to be received as files:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}

>>> r = requests.post(url, files=files)
>>> r.text
{
  // ...snip... //
  "files": {
    "file": "some,data,to,send\\nanother,row,to,send\\n"
  },
  // ...snip... //
}

Response Status Codes

We can check the response status code:

>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200

Requests also comes with a built-in status code lookup object for easy reference:

>>> r.status_code == requests.codes.ok
True

If we made a bad request (non-200 response), we can raise it with Response.raise_for_status():

>>> bad_r = requests.get('http://httpbin.org/status/404')
>>> bad_r.status_code
404

>>> bad_r.raise_for_status()
Traceback (most recent call last):
  File "requests/models.py", line 832, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error

But, since our status_code for r was 200, when we call raise_for_status() we get:

>>> r.raise_for_status()
None

All is well.

Response Headers

We can view the server’s response headers using a Python dictionary:

>>> r.headers
{
    'status': '200 OK',
    'content-encoding': 'gzip',
    'transfer-encoding': 'chunked',
    'connection': 'close',
    'server': 'nginx/1.0.4',
    'x-runtime': '148ms',
    'etag': '"e1ca502697e5c9317743dc078f67693f"',
    'content-type': 'application/json; charset=utf-8'
}

The dictionary is special, though: it’s made just for HTTP headers. According to RFC 2616, HTTP Headers are case-insensitive.

So, we can access the headers using any capitalization we want:

>>> r.headers['Content-Type']
'application/json; charset=utf-8'

>>> r.headers.get('content-type')
'application/json; charset=utf-8'

If a header doesn’t exist in the Response, its value defaults to None:

>>> r.headers['X-Random']
None

Cookies

If a response contains some Cookies, you can get quick access to them:

>>> url = 'http://httpbin.org/cookies/set/requests-is/awesome'
>>> r = requests.get(url)

>>> r.cookies['requests-is']
'awesome'

To send your own cookies to the server, you can use the cookies parameter:

>>> url = 'http://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')

>>> r = requests.get(url, cookies=cookies)
>>> r.text
'{"cookies": {"cookies_are": "working"}}'

Basic Authentication

Many web services require authentication. There are many different types of authentication, but the most common is HTTP Basic Auth.

Making requests with Basic Auth is extremely simple:

>>> from requests.auth import HTTPBasicAuth
>>> requests.get('https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
<Response [200]>

Due to the prevalence of HTTP Basic Auth, requests provides a shorthand for this authentication method:

>>> requests.get('https://api.github.com/user', auth=('user', 'pass'))
<Response [200]>

Providing the credentials as a tuple in this fashion is functionally equivalent to the HTTPBasicAuth example above.

Digest Authentication

Another popular form of web service protection is Digest Authentication:

>>> from requests.auth import HTTPDigestAuth
>>> url = 'http://httpbin.org/digest-auth/auth/user/pass'
>>> requests.get(url, auth=HTTPDigestAuth('user', 'pass'))
<Response [200]>

OAuth Authentication

Requests features robust, built-in OAuth support!

OAuth takes many forms, so let’s take a look at a few different forms:

import requests
from requests.auth import OAuth1

url = u'https://api.twitter.com/1/account/settings.json'

client_key = u'...'
client_secret = u'...'
resource_owner_key = u'...'
resource_owner_secret = u'...'

Query signing:

queryoauth = OAuth1(client_key, client_secret,
                    resource_owner_key, resource_owner_secret,
                    signature_type='query')
r = requests.get(url, auth=queryoauth)

Header signing:

headeroauth = OAuth1(client_key, client_secret,
                     resource_owner_key, resource_owner_secret,
                     signature_type='auth_header')
r = requests.get(url, auth=headeroauth)

Body signing:

bodyoauth = OAuth1(client_key, client_secret,
                   resource_owner_key, resource_owner_secret,
                   signature_type='body')

r = requests.post(url, auth=bodyoauth)

Redirection and History

Requests will automatically perform location redirection while using the GET and OPTIONS verbs.

GitHub redirects all HTTP requests to HTTPS. We can use the history method of the Response object to track redirection. Let’s see what Github does:

>>> r = requests.get('http://github.com')
>>> r.url
'https://github.com/'
>>> r.status_code
200
>>> r.history
[<Response [301]>]

The Response.history list contains a list of the Request objects that were created in order to complete the request. The list is sorted from the oldest to the most recent request.

If you’re using GET or OPTIONS, you can disable redirection handling with the allow_redirects parameter:

>>> r = requests.get('http://github.com', allow_redirects=False)
>>> r.status_code
301
>>> r.history
[]

If you’re using POST, PUT, PATCH, DELETE or HEAD, you can enable redirection as well:

>>> r = requests.post('http://github.com', allow_redirects=True)
>>> r.url
'https://github.com/'
>>> r.history
[<Response [301]>]

Timeouts

You can tell requests to stop waiting for a response after a given number of seconds with the timeout parameter:

>>> requests.get('http://github.com', timeout=0.001)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: Request timed out.

Note:

timeout only effects the connection process itself, not the downloading of the response body.

Errors and Exceptions

In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.

In the event of the rare invalid HTTP response, Requests will raise an HTTPError exception.

If a request times out, a Timeout exception is raised.

If a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised.

All exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException.

You can refer to Configuration API Docs for immediate raising of HTTPError exceptions via the danger_mode option or have Requests catch the majority of requests.exceptions.RequestException exceptions with the safe_mode option.


Ready for more? Check out the advanced section.

Requests is an elegant and simple HTTP library for Python, built for human beings. You are currently looking at the documentation of the development release.

Donate

If you love Requests, consider supporting the author on Gittip:

If your organization uses Requests, consider financial support:

Requests Pro

Table Of Contents

Related Topics

Fork me on GitHub