Python – Download File from URL

Filed Under: Python
Python Download File From Url

Downloading a File from URL is a very common task in Python scripts. A real life example is to download images from a website to the local system and then process it in our Python program.

In this tutorial, we will learn different ways to download file from a URL in Python.

Using requests library to download file from URL in Python Scripts

If your requirement is to get the file from a given URL using GET HTTP request, then the Python requests module is perfect for you.

import requests

file_url = 'https://www.journaldev.com/wp-content/uploads/2019/08/Python-Tutorial.png'

file_object = requests.get(file_url)

with open('Python-Tutorial.png', 'wb') as local_file:
    local_file.write(file_object.content)

The file will be downloaded in the same directory as the Python script. If you want to change the directory location, you can provide a complete path or relative path in the open() function call.

Recommended Reading: Python with Statement

Linux aficionado? Use Python wget library to download file from URL

If you love Linux commands and want to have similar flavor in your Python program, you can use wget library to download the file from a URL.

Python wget library is not part of the default installation, so you can install it using the PIP package manager.

# pip install wget

Here is the Python program to download a file from URL using wget library.

import wget

file_url = 'https://www.journaldev.com/wp-content/uploads/2019/08/Python-Tutorial.png'

dest_file = '/Users/pankaj/pt.png'

wget.download(file_url, dest_file)

The destination file argument is optional. If we don’t provide that then the file will be saved in the same directory as the script and filename will be the same as the remote file name.

Downloading File from a URL that Redirects

Sometimes we get short URLs that redirect to the actual file. The requests library get() method automatically follows the redirect and download the actual file. If you look at the get() implementation, it sets allow_redirects parameter as True.

def get(url, params=None, **kwargs):
    kwargs.setdefault('allow_redirects', True)
    return request('get', url, params=params, **kwargs)

Downloading a Large File from URL in Python

If the file is large, then it’s not a good idea to get all the content in one go. It will require a lot of memory and might cause out of memory error.

We can pass stream=True to requests get() method to open a file stream and download it in chunks. Then we can use a for loop to read the chunks and write it into the local file.

import requests

file_url = 'https://www.journaldev.com/wp-content/uploads/2019/08/Python-Tutorial.png'

file_stream = requests.get(file_url, stream=True)

with open('Python-Tutorial.png', 'wb') as local_file:
    for data in file_stream:
        local_file.write(data)

print('Done')

Conclusion

It’s very easy to download a file from URL in Python. The requests module is perfect for it. We can also specify the HTTP methods to download the file.

Reference: Requests Module Official Docs

Comments

  1. jai says:

    how to read list of files in URL and download selected one?

  2. Task says:

    Very helpful code Thnks.

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages