[sunjester tuts] HTTP GET/Web Requests with Python

[sunjester 1]

Introduction

I normally don't use python very much. However, I can always jump right into using it since I know how to properly read documentation. I am going to show you how you can read some documentation and start making requests to websites through python and the requests library. As with all my tutorials we will be using Linux (ubuntu xenial), and no I will not cater to the Windows kids.

 

Requirements

This tutorial will require a few things, including python. I am using 2.7 (most of you probably use 3, I don't.).

• Python >= 2.7
  
• pip >= 19.03
  
• python requests library >= 2.21
  

Checking Versions with Pip

You can easily check the requests version, or if it's even installed like this

pip show requests

 

Documentation

The website for the requests library is found here. We will be following the documentation and using the software as intended by the developers, which should give you an insight into reading documentation and finding/using other libraries.

 

Including the Library

It's pretty easy to add the library to your script. Like almost all other libraries, using the import keyword.

import requests

Getting HTML

We can use a simple line of code to requests an HTTP GET request for HTML of a web site.

req = requests.get("https://www.imdb.com/title/tt0093177/") 

This is where reading the documentation will come in handy. The code above will make the request to the IMDB page, but printing the req variable probably won't give you what you are looking for, so let's see what the will do.

 

 

It printed the response code. Back on the website (documentation) we see that it's a response object, which is probably why it has the angle brackets around it also.

 

 

Response Object Documentation

Since it returned an object, let's go see what the properties of this object are in the documentation. Here is the documentation for the Response object of the Requests library. We can see there are a few things here, let's try to use content and see if we get the source.

#!/usr/bin/env python
import requests

req = requests.get("https://www.imdb.com/title/tt0093177/") 
print req.content

 

Yep, there is the source.

[nsarat 0]

thanks ive always wanted to get into python

[socalgurl 1]

im glad this is here