Using context managers in Python
6 min read
If you ever used Python, most likely you used context managers without even knowing what they are and what they do or how they work - you just used them as per the documentation of some 3rd party package. In this article, I will try to explain what exactly are context managers and what are their benefits.
1. Context managers (by definition)
I hate to be a bookworm (I really do, trust me), but sometimes good old definitions come in handy. This is the case with context managers. So here it goes:
Context managers are responsible for allocating and releasing resources when needed.
Is a question mark popping up in your head? Expected. I will walk you with a practical example :)
1.1 Practical example - no context manager
The idea is to open up a file with Python and read the file contents of that specific file. Let's take a look at the code below:
file = open("textfile.txt", "r") file.read() file.close()
Simple eh? We open up a file called
read mode (
r) and we read all of the contents of that file by making use of the
read() method. Once we are done with injecting the content inside of the memory (or a variable if you want), we close the file.
Now - imagine that you forgot to close the file. Now imagine that you work with tons of files and somehow forget to close all of them. This can lead to substantial resource leaks and you want to avoid that.
Enter context managers :)
1.2 Practical example - context manager
The idea is once again to open up a file and read its contents. This time we will use context manager. The whole idea of a context manager is to do the following:
- Open up the resource (allocate it)
- Do something with the resource
- Close the resource once it is not needed anymore (release it)
In Python, we use the context managers by making use of the
with keyword. Let's see what the code looks like now:
with open("textfile.txt", "r") as file: file.read() # some other code while the resource (file) is allocated
Similar, yet different. Where did the
file.close() go? Funny that you ask - we don't need it anymore.
But... but... you said that we need to close the file to avoid resource leaks bla bla...
Yes, I did, and it is true. But what if I told you that context managers do that for you. In the above code, the resource (file) is opened as a variable called
file in a read mode and is made available for usage (additional related code) inside of a block (indentation). Inside of that block, we read the file content. We can also do additional stuff with the file INSIDE of that block, BUT once we leave that block, the context manager will release the resource (in this case close the file) and we can no longer access it.
Handy eh? Let's see how it works under the hood.
2. Context manager under the hood
Each object (in Python everything is an object) that wants to use context manager, by using the
with keyword, has to implement two built-in methods -
For the above use case, if we were to create a custom context manager class it would look something like this:
from types import TracebackType from typing import IO, Optional, Type class CustomClassFile: def __init__(self, file_path: str, file_mode: str) -> None: self.file = open(file_path, file_mode) def __enter__(self) -> IO: return self.file def __exit__( self, exc_type: Optional[Type[BaseException]], exc_value: Optional[BaseException], traceback: Optional[TracebackType], ) -> None self.file.close() with CustomClassFile("textfile.txt", "r") as file: file.read()
So, once we create a new instance of
CustomClassFile, we pass the two arguments -
file_mode. That is fairly simple.
Inside of the
__enter__ method is where we need to define the resource that we want to allocate once the context manager kicks in. In this case, it will be the
__exit__ method signature is a bit weird, but I won't bother you with the details of it. All of those arguments are required by default - so whenever you will create the
__exit__ method, you would use this exact same signature. This is due to the ability to create your own exception handlers (if they occur) upon releasing the resource. What is important is that we call the
self.file.close() method inside of the
__exit__ method. This means that once we get past the
with block, the file will be closed automatically by the context manager :)
Let's see some more examples.
2.1 Context manager as a class by example
The idea here would be to create the custom class that will allocate not the file but rather the contents of a
JSON file and releases the file afterward.
import json from types import TracebackType from typing import Any, IO, Optional, Type class JSONFileLoader: def __init__(self, file_path: str) -> None: self.file: IO = open(file_path, "r") self.data: Any = json.load(self.file) def __enter__(self) -> Any: return self.data def __exit__( self, exc_type: Optional[Type[BaseException]], exc_value: Optional[BaseException], traceback: Optional[TracebackType], ) -> None: if self.file: self.file.close() with JSONFileLoader("file.json") as json_data: print(json_data)
This time when we create a new instance of the
JSONFileLoader we only pass one argument -
__init__ method then also loads the json contents of the file inside of the
__enter__ method will this time allocate the data (as per the idea above) instead of the file itself. That means that once we enter the context manager instead of the whole file like before we will only have the file data available.
__exit__ method remains the same as before - we just close the file.
with JSONFileLoader("file.json") as json_data: print(json_data)
See that now instead of the file, our
__enter__ method actually is returning the
self.data property, which means that
json_data is actually whatever is set inside of the
self.data property. This makes it simple to just - for this example - print out the data. Once we leave the
__exit__ will kick in and close the file for us.
2.2 Context manager as a generator by example
This is an alternative approach to creating context managers in Python. If you don't know what generators are in Python, please take a moment to read about them.
Let's see how to make use of Python's
contextlib module to solve the same idea as the one above.
import json from contextlib import contextmanager from typing import Any, Generator, IO def json_file_loader(file_path: str) -> Generator[Any, None, None]: file: IO = open(file_path, "r") data: Any = json.load(file) yield data file.close() with json_file_loader("file.json") as json_data: print(json_data)
So the idea is that you will create a function and decorate it by using
@contextmanager to mark it as a context manager for Python. We create a function that takes the
file_path as an argument and it opens the file and loads it by using the
Once that is done, we
yield the data. That means we will
pause the execution of the code below from the function
json_file_loader until we exit the context manager. This means that
file.close() will execute ONLY after we exit the context manager as described below.
with json_file_loader("file.json") as json_data: # inside of the context manager (yield kicked in) print(json_data) # context manager exited which means that file.close() will execute
3. Benefits of using context managers
So what would be the perks of using context managers?
- Reduce resource leaks
- Reduce the number of lines of code
- More power over what to expose inside of the
- Multiple ways to implement
- No need to worry about releasing the resources
I hope you find those benefits handy and start to make use of context managers.
Like always, thanks for reading :)