Python

Python Requests Module Tutorial

Posted on November 3, 2022November 3, 2022 by The Editor

This is a detailed tutorial on the Python Requests Module. Learn to send any kind of request (GET, PUT, POST, DELETE, etc.) and to get their response.

Table of Contents

Python Requests

In most applications these days, we need to get data from other sources by sending them different requests. The requests are also used for performing different actions on the servers and systems. In python, we have got a module named requests that can be used to send any kind of requests like GET, PUT, POST, DELETE, PATCH, HEAD, or any other custom request. The Python Requests module offers a complete solution to handle the request response as well. In other words, this module is a perfect solution to send any kind of HTTP request.

I have my hands-on different programming languages and I know in some of the programming languages, you need to do a lot of struggle to send the requests and process their response. But in python, the Python Requests library makes things simpler and better than ever. This module offers simple methods to send different kinds of requests and you can easily pass the request data using arguments and the response is provided as the returned data of these methods.

Syntax

The basic syntax for all the different methods that you can use under this library is given below.

import requests
response = requests.method("<REQUEST-URL>", params={key: value}, **kwargs)

Note. The keyword method in the above syntax has to be replaced with the actual method name that we’ll discuss further in this article. The response so returned will be stored in the variable named response and you can apply a number of different methods to fetch the response information appropriately.

The first parameter is always the request URL and it is also a mandatory argument. All other parameters are optional. In the second parameter, you can send data in the form of key-value pairs. You can also provide a Python Dictionary here as the second argument.

The third argument **kwargs is actually used to represent zero or more arguments. There are many more different arguments that can be specified within the different methods of the Python Requests Module. These additional arguments are can be used to specify additional data, files, timeout, cookies, proxies, streams, etc. We’ll find out more about the usage of these things in a later section of this article. It returns a response object. We can apply a number of properties and methods to this response object to get valuable data.

You can find out more about the *args and **kwargs notation at the following linked article.

Python *args and **kwargs for Variable Arguments

Request Methods

In this article, we’ll talk discuss seven different methods that you can use with the Python Requests library for the purpose of sending different requests and processing their responses.

GET

Mostly, we need to send GET requests to fetch data from some resource or URL. The python requests library offers get() a method to send a GET request and to receive its response.

Example. The following python program sends a GET request to a URL to print out the response code and the textual data so received.

import requests
response = requests.get("https://api.ratesapi.io/api/latest")
#Prints Status Code
print(response.status_code)
#Prints Textual Data
print(response.text)

The RatesAPI is a FREE API to fetch Foreign Exchange Rates and here we’re sending a GET request to the API URL using the Python Requests Library and then printing out the response status code and response text. Apart from the response code and text, there are a lot of different methods that can be applied to the response object to get more valuable information. We’ll also discuss these response object methods at the end of this article.

POST

The HTTP POST method is usually used to send requests that are intended to store or insert data into the database. In other words, the POST requests are used for submission purposes. In these requests, we send along with some data in different forms that are meant to be submitted or stored.

Example. The following python code sends a POST request to a URL. This URL is actually a dummy URL to test POST requests. Therefore, it also returns the arguments and data that we sent along with the POST request using the Python Requests Module post method.

import requests

#Data To Send
info = {"name": "Gurmeet Singh", "age": 21}

#Sending the POST Request
response = requests.post("https://postman-echo.com/post", info)

#Prints Status Code
print(response.status_code)

#Prints Textual Data
print(response.text)

We’ve stored the data to send in a dictionary stored in the variable info and now we’re sending this variable as the second argument to the post method. Observe carefully the request-response text shown in the following screenshot, the data that we sent is also returned.

PUT

The PUT method is usually used to send the information that is meant to be updated in the database. In the Python Requests module, the method put() can be used to send such requests.

Example. The following python code sends a PUT request to a dummy URL along with arguments and body data.

import requests

#Data To Send
info = {"id": 5, "name": "Gurmeet Singh", "age": 21}

#Sending the PUT Request
response = requests.put("https://postman-echo.com/put", info)

#Prints Status Code
print(response.status_code)

#Prints Textual Data
print(response.text)

DELETE

The delete method is used to send the server a request to delete some record(s) from the database. The method delete() can be used with the Python Requests module to simulate an HTTP DELETE Request.

Example. The following code snippet sends a DELETE request to a dummy Test URL.

import requests

#Data To Send, usually the ID of the record to be deleted
info = {"id": 5}

#Sending the DELETE Request
response = requests.delete("https://postman-echo.com/delete")

#Prints Status Code
print(response.status_code)

#Prints Textual Data
print(response.text)

In this DELETE request, we’ve sent the id of the database record to be deleted as the requested data.

HEAD

The method head() is used with the Python Requests module to simulate an HTTP HEAD request. Unlike the other methods like GET, POST, PUT and DELETE, this method does not take the second argument data. But it accepts the other **kwargs. The HEAD method is used to get the request-response status or code and not the content, therefore the data parameter is not required.

Example. The following code sends a head request to a URL just to check its status code.

import requests

#Sending HEAD Request
response = requests.head("https://api.ratesapi.io/api/latest")

#Prints Status Code
print(response.status_code)

#Prints Textual Data
print(response.text)

We’ve also written the line of code to print the textual data, but as it is a HEAD request, no textual data is returned.

PATCH

A PATCH request is also used to patch or update the server resources or data. The method patch() simulates PATCH requests in the Python Requests library.

Example. A PATCH request is being sent with some data in the following code snippet.

import requests

#Data To Send
info = {"id": 5, "name": "Gurmeet Singh", "age": 21}

#Sending patch Request
response = requests.patch("https://postman-echo.com/patch")

#Prints Status Code
print(response.status_code)

#Prints Textual Data
print(response.text)

CUSTOM REQUEST

In case you want to send a request with a CUSTOM method, you can use the request() method to send any kind of request. The first parameter of the method request() is a string defining the name of the method itself. This way you get the flexibility of sending requests to your own created methods.

Example. In the following code, we’re simulating the GET Method request using the method request() instead of using the get() method.

import requests

response = requests.request("GET","https://api.ratesapi.io/api/latest")

#Prints Status Code
print(response.status_code)

#Prints Textual Data
print(response.text)

Request Method Parameters

As denoted by the third syntax argument **kwargs that any of the above methods can take a number of variable parameters. This section describes those parameters and their purpose with the help of an example for each.

Parameter	Description	Example
`url`	The only required parameter defines the URL to which the request has to be sent.	requests.get(“https://postman-echo.com/get”)
`params`	Optional. You can provide key-value pair data here in the form of a Python Dictionary. A list of Tuples, and Query String bytes can also be provided. Default value is `None`.	info = {“id”: 5, “name”: “Gurmeet Singh”, “age”: 21} requests.post(“https://postman-echo.com/post”, info)
`allow_redirects`	Optional Boolean parameter. Specifies whether you want the request URL to be redirected to another Request URL or not. Default value is `True`.	requests.head(“https://postman-echo.com/get”, allow_redirects = False)
`auth`	Optional Tuple parameter. Used for HTTP authentication. Default value is `None`.	requests.get(url, auth = (‘username’, ‘password’)) or requests.get(url, auth = (‘<api-key>’))
`cert`	Optional String or Tuple Parameter. Specified a certificate or a key for the current request. Default value is `None`.	requests.put(“https://postman-echo.com/get”, cert = “some-location/certificate-file.cert”)
`cookies`	Optional Dictionary Parameter. Used to specify cookies to send along with the current request. Default value is `None`.	some_cookies = {“id”: 5, “name”: “Gurmeet Singh”, “age”: 21} requests.post(“https://postman-echo.com/post”, cookies = some_cookies)
`headers`	Optional Dictionary Parameter. Used to specify the different headers to be sent for the current request. Default value is `None`.	my_headers = {“Content-Type”: “application/json; charset=utf-8”} requests.delete(“https://postman-echo.com/post”, headers = my_headers)
`proxies`	Optional Dictionary Parameter. Used to specify the proxies in the key-value form for the current request. Default value is `None`.	requests.get(url, proxies = { “https” : “<some-IP-address”})
`stream`	Optional Boolean Parameter. Used to specify whether the response should be immediately downloaded into the response object or it has to be streamed. Useful for API requests that respond with streaming videos. False means Download and True means stream. Default value is `False`.	requests.get(“http://some-video-api.com/video?id=546”, stream=True)
`timeout`	Optional Numerical or Tuple Parameter. Used to specify the time a request should wait to get its response. The time has to be specified in seconds. 0.05 represents 50 milliseconds. Default value is `None` indicating the request has no defined response wait time and it will keep on continuing until the response is received.	requests.head(“https://postman-echo.com/get”, timeout = 0.05)
`verify`	Optional Boolean Parameter. Used to verify a Certificate. Specifying it as False will not verify the certificate. Default value is `True`.	requests.get(“https://postman-echo.com/get”, verify=’some-location/some-certificate’)

Response Object Properties and Methods

We know that all of the methods return a Response Object. The response object itself is not giving any information, but you need to make use of some properties and methods to get the right and required information contained in this response object. These different methods and properties are given below in tabular form. Instead of giving example for each of the response object properties and methods, I’ve given a full example code afterward that makes use of each of these.

Property or Method	Description
`apparent_encoding`	Property. Returns the apparent encoding for the response.
`close()`	Method. Used to close the current connection. Quite useful for streaming responses.
`content`	Property. Returns the content in bytes form.
`cookies`	Property. It returns the cookie Jar object that contains the cookies sent back by the server.
`elapsed`	Property. It returns a `timedelta` object that contains the time to process the request, i.e. the time interval between the sending instance and response received instance.
`encoding`	Property. It returns the encoding that is used to decode the textual data so returned.
`headers`	Property. The response headers are stored in a Python Dictionary and are returned by this property.
`history`	Property. This returns the history of request URL(s). In case the request is hoped for from one URL to another, the `history` property can be used to know the redirected URLs.
`is_permanent_redirect`	Property. It returns a Boolean value; True if the URL is permanently redirected and False if not.
`is_redirect`	Property. It will return True if the request URL redirects at least once, otherwise False.
`iter_content()`	Method. It can be used to loop through the content of the response.
`iter_lines()`	Method. The response can be iterated line by line using this method.
`json()`	Method. If and only if the response is in JSON form, it returns the response as a JSON object, otherwise, it raises an error.
`links`	Property. It returns the links to the header.
`next`	Property. If the request is meant for a redirection it returns an object that is already prepared to be sent in the next redirected request.
`ok`	Property. If the status code is 200 or less 200, it returns true, otherwise False.
`raise_for_status()`	Property. This returns an HTTPError object if an error occurs.
`reason`	Property. Returns the message associated with the response code.
`request`	Property. The request object is returned that is being used to get the response for the current request.
`status_code`	Property. It returns the response status code. For example, 200, 201, 404, etc.
`text`	Property. It returns the textual response in Unicode form.
`url`	Property. This returns the URL for the response.

Example. The following python program illustrates the use of all of the above-defined Python Request Module’s Response Object’s Properties and methods for a simple GET request.

#Importing Python requests Module
import requests
#Getting Response Object
response = requests.get("https://api.ratesapi.io/api/latest")

#Applying Different Properties & Methods on Response Object

#Apparent Encoding
print(response.apparent_encoding)

#Closes Connection
response.close()

#Content
print(response.content)

#Returned CookieJar Object
print(response.cookies)

#TimeDelta Object
print(response.elapsed)

#Encoding To Decode Response Text
print(response.encoding)

#Respone Headers
print(response.headers)

#URL History
print(response.history)

#Check if it is a permanenet redirect
print(response.is_permanent_redirect)

#Check if the request URL is being redirected or not
print(response.is_redirect)

#Get JSON Data if any
print(response.json())

#Links
print(response.links)

#Next Request Object
print(response.next)

#Checking if status OK
print(response.ok)

#Raises Error if any
print(response.raise_for_status())

#Response Status Message
print(response.reason)

#Request object
print(response.request)

#Status Code
print(response.status_code)

#Response Textual Content
print(response.text)

#Response URL
print(response.url)

#Loop Through Content Bytes
for x in response.iter_content():
  print(x)

#Loop Through Line By Line
for x in response.iter_lines():
  print(x)

As tried to print the response of all the properties and methods, the output is very long. Only some part of the output is shown in the following screenshot.

I hope you found this guide useful. If so, do share it with others who are willing to learn Python. If you have any questions related to this article, feel free to ask us in the comments section.

The Editor

Here at CodeLucky.com, we aim to provide you with the highest-quality content on various Technological Topics like Programming and Data Science. If you have any feedback please feel free to share it at [email protected].