Dynamic GET requests

Now we know that Python can programmatically download a website as long as we have the URL. If we have to download multiple pages that only differ in the query string, then we can write a script to do this without repeatedly rerunning the script, and instead download everything we need in one run.

How to do it...

Check out this URL-- https://www.packtpub.com/all?search=&offset=12&rows=&sort=. Here, the query string variable that defines the page number (offset) is multiples of 12:

To download all the images in all of these pages, we can rewrite the previous recipe as follows:

1. Import the required modules:

import urllib.request 
import urllib.parse 
import re 
from os.path import basename

2. Define the URL and query string:

url = 'https://www.packtpub.com/' 
queryString = 'all?search=&offset='

3. Iterate the offset through multiples of 12:

for i in range(0, 200, 12): 
    query = queryString + str(i) 
    url += query 
    print(url) 
    response = urllib.request.urlopen...