Python with Statement – with open file

Python with statement allows us to write simpler code when working with context managers. The with statement was introduced in Python 2.5 under PEP 343.

1. What is a Context Manager?

A context manager is an object that creates and manages a runtime context. The typical usage of context managers is to save and restore the global state, lock and unlock resources, opening and closing files, etc.

2. The lifecycle of Context Manager

The context manager object must define the enter() and exit() methods. These methods are called when the runtime context is created and when it’s destroyed.

3. Why do we need Python with statement?

When we have to work with a file, we have to first open it. It creates a runtime context manager for the file. After the work is done, we have to manually close the file so that the context manager terminates properly.

We generally use the try-except block to work with the file and finally block to close the file. This makes sure that the file gets closed even if there is an error raised by the try block.

try:
    txt_file = open("abc.txt")
    # do some operations
    txt_file.close()
except FileNotFoundError as e:
    print(e)
finally:
    txt_file.close()

Python with statement takes care of calling the exit() method of the context manager when the code inside the with block is executed.

Let’s rewrite the above code using the with statement.

with open("abc.txt") as file:
    # do some operations
    print("Done")

The code is much simpler to read and we don’t have to worry about closing the file every time.

4. Python with Statement Custom Context Manager Example

We can define our own custom context manager by implementing enter() and exit() methods.

class MyContext:

    def __init__(self):
        print("init method of MyContext")

    def __enter__(self):
        print("entering context of MyContext")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("exit context of MyContext")


with MyContext() as my_context:
    print("my_context code")

Output:

init method of MyContext
entering context of MyContext
my_context code
exit context of MyContext

• The context manager is initialized.
• Then the __enter__() method is called for the context manager object.
• The code inside the with block is executed.
• Finally, the __exit__() method of the context manager is called.

5. Python with open files

Python 3.1 enhanced the with statement to support multiple context managers. Let’s see how to open multiple files using the with statement.

with open("abc.txt") as file1, open("abc.txt") as file2:
    pass

The above code is equivalent to multiple nested with statements.

with open("abc.txt") as file1:
    with open("abc.txt") as file2:
        pass

6. Python with statement exception scenarios

If there is an exception raised in the with block, its type, value, and traceback are passed as arguments to __exit__().

If the __exit__() method returns False, then the exception is re-raised.

class MyContext:

    def __init__(self):
        print("init method of MyContext")

    def __enter__(self):
        print("entering context of MyContext")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f'exit context of MyContext - {exc_type} :: {exc_val} :: {exc_tb}')
        return False


with MyContext() as my_context:
    print("my_context code")
    raise TypeError("TypeError inside with block")

Output:

init method of MyContext
entering context of MyContext
my_context code
exit context of MyContext - <class 'TypeError'> :: TypeError inside with block :: <traceback object at 0x1044e8f48>
Traceback (most recent call last):
  File "/Users/pankaj/Documents/PycharmProjects/hello-world/journaldev/with_statement.py", line 32, in <module>
    raise TypeError("TypeError inside with block")
TypeError: TypeError inside with block

If the __exit__() method returns True, then the exception is consumed and normal execution continues.

class MyContext:

    def __init__(self):
        print("init method of MyContext")

    def __enter__(self):
        print("entering context of MyContext")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f'exit context of MyContext - {exc_type} :: {exc_val} :: {exc_tb}')
        return True


with MyContext() as my_context:
    print("my_context code")
    raise TypeError("TypeError inside with block")

print("Done")

Output:

init method of MyContext
entering context of MyContext
my_context code
exit context of MyContext - <class 'TypeError'> :: TypeError inside with block :: <traceback object at 0x102149e08>
Done

7. Conclusion

Python with statement takes care of managing the life cycle of the context manager. The code looks small and removes the chance of leaving the context manager open due to poor programming.

8. References

• Context Manager
• Python.org Docs