Python Urllib Response

The urllib package provides a simple and powerful interface for making HTTP requests with Python. It is a great utility, and although it's not included by default, it is used by millions of other packages.

The urllib package is diverse, with various functions, classes, and objects for performing multiple tasks. The essence of this tutorial is to learn about the response module.

What is the urllib.response?

The urllib.response is a module that defines the functions and classes used for request responses in a file-like interface.

Functions defined in the urllib.response module.

The module defines a set of functions used internally by urllib.request module. The functions described in this module include:

1. url – the URL of the resource fetched. This is commonly used to check for follow redirect operations.
2. headers – returns the headers of the response in the EmailMessage instance.
3. status – returns the status code of the server.

Example 1

We can fetch the headers from a given request using the headers of the HTTPResponse object. An example is shown below:

from urllib.request import urlopen
from pprint import pprint

with urlopen("http://cloudenv.io") as response:
    headers = response.info().items()
pprint(headers)

The above example returns detailed header information about the response. An example output is shown below:

[('Cache-Control', 'public, max-age=0'),
 ('Content-Length', '58331'),
 ('Content-Type', 'text/html; charset=utf-8'),
 ('Date', 'Sat, 31 Aug 2024 21:54:41 GMT'),
 ('Etag', 'W/"e3db-X1GA8tEbgFmV23dBwGgiej23qTU"'),
 ('Vary', 'Accept-Encoding'),
 ('Vary', 'Accept-Encoding'),
 ('X-Powered-By', 'Express'),
 ('Connection', 'close')]

Although you have all the header information, you probably do not need to use all of it. For example, you can filter specific headers as shown:

from urllib.request import urlopen
from pprint import pprint

try:
    with urlopen("https://cloudenv.io") as response:
        status_code = response.getcode()
        pprint(status_code)
except Exception as e:
    pprint(f"An error occurred: {e}")

In these cases, the query returns the status code of the request as shown:

200

We can also fetch the character set directly from the response, as shown in the code snippets below:

from urllib.request import urlopen
from pprint import pprint
with urlopen("https://cloudenv.io") as response:
    body = response.read()
charset = response.headers.get_content_charset()
pprint(charset)

And without a doubt, the code returns the response to the character encoding:

'utf-8'
(base)

Conclusion

In this post, you learned about the urllib.response module, which defines classes and functions used by other urllib modules.