Python shutil module

Filed Under: Python

When you need to work with high-level file operations like copying contents of a file, create a new copy of a file and archiving it, Python shutil module is the way to go.

Python shutil

Python shutil module enables us to operate with file objects easily and without diving into file objects a lot. It takes care of low-level semantics like creating file objects, closing the files once they are copied and allows us to focus on the business logic of our program. Let’s see shutil module in action here.

Python shutil example

Let’s look into different examples to understand shutil module.

Copy File

Using shutil’s copyfile() function, it is easy to copy a file to a new file in current directory only.

Here is a sample program on how we can make a new clone of existing file in our current directory:


import os
import shutil

print('BEFORE:', os.listdir('.'))
shutil.copyfile('file_copy.py', 'file_copy.py.copy')
print('AFTER:', os.listdir('.'))

Let’s see the output for this program:

Python shutil Copy file

Copy file to current directory


Note that copyfile() function takes name of the new file to be created.

Copying Files to another directory

Using shutil’s copy() function, it is easy to copy a file to another directory.

Let’s look at a code snippet on how this can be done:


import os
import shutil

os.mkdir('journaldev')
print('BEFORE:', os.listdir('journaldev'))
shutil.copy('file_copy.py', 'journaldev')
print('AFTER:', os.listdir('journaldev'))

Let’s see the output for this program:

python shutil Copy file to new directory

Copy file to new directory


This function is different from copyfile() function as the later takes a file name as a parameter whereas the copy() function takes directory name as an input.

Finally, the permissions of the file are also cloned when copying a file with both the functions but metadata is not copied, which means that new file created will have a freshly created time instead of the original file’s time.

Copying file with Metadata

If you need to make an exact clone of the file, along with the permissions and the metadata of a file as well, we can make use of the copy2() function. Note that this might not work completely on a POSIX based system.

Here is a sample program on how we use this function::


import os
import shutil
import time

def file_metadata(file_name):
    stat_info = os.stat(file_name)
    print('  Mode    :', oct(stat_info.st_mode))
    print('  Created :', time.ctime(stat_info.st_ctime))
    print('  Accessed:', time.ctime(stat_info.st_atime))
    print('  Modified:', time.ctime(stat_info.st_mtime))

os.mkdir('journaldev')
print('SOURCE FILE:')
file_metadata('file_copy.py')

shutil.copy2('file_copy.py', 'journaldev')

print('DESTINATION FILE:')
file_metadata('journaldev/file_copy.py')

We run this function on a POSIX system so only the Mode of the file and Modified date is preserved:

Copy File Metadata (as much as possible)

Copy File Metadata (as much as possible)


On other systems, even the created and accessed time would have matched exactly.

Replicating complete directory

With copytree() function, it is possible to completely replicate a directory tree recursively. This means that is there are more directories inside a directory, that directory will be cloned as well.

Let’s look at a code snippet on how we can clone a complete directory:


import pprint
import shutil
import os

shutil.copytree('../shutil', './journaldev')

print('\nAFTER:')
pprint.pprint(os.listdir('./journaldev'))

Let’s see the output for this program:

python shutil Copy directory recursively

Copy directory recursively


Note that we do not print the directory journaldev contents before as the directory name copytree() function takes as input must not exist before running this command.

Removing a Directory

Another simplest example with shutil is removing complete directory. No need to recursively remove files or close file handling connections.

Here is a sample program on how easy it is with rmtree() function:


import pprint
import shutil
import os

print('BEFORE:')
pprint.pprint(os.listdir('.'))

shutil.rmtree('journaldev')

print('\nAFTER:')
pprint.pprint(os.listdir('.'))

Here, we print contents of current directory. Before running the script, the journaldev directory exists in current folder. After running the script, it is deleted along with the contents. Let’s see the output for this program:

python shutil Remove file and directory tree

Remove complete directory tree

Finding files

The which() function presents an excellent tool to find a file on your machine which is present on the PATH.

Here is a sample program with a file example:


import shutil

print(shutil.which('bsondump'))
print(shutil.which('no-such-program'))

Let’s see the output for this program:

Finding a file on PATH

Finding a file on PATH

Monitoring File-system Space

We can even get information about how much is present on our file system with simplete call using disk_usage() function.

Here is a sample program:


import shutil

total_b, used_b, free_b = shutil.disk_usage('.')

gb = 10 ** 9

print('Total: {:6.2f} GB'.format(total_b / gb))
print('Used : {:6.2f} GB'.format(used_b / gb))
print('Free : {:6.2f} GB'.format(free_b / gb))

Let’s see the output for this program:

File System Space

File System Space

Conclusion

In this lesson, we studied how we can work with high-level file operations like copying contents of a file, create a new copy of a file etc. without diving into complex File Handling operations with shutil module in Python.

Read more Python posts here.

Download the Source Code

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages