Dependency Injection With Python, Make It Easy!

Updated Oct 11, 2023 • 11 min read

Dependency injection is a technique built on the top of the Inversion of Control.

The main idea is to separate the construction and usage of objects. As a software engineer, your objective is to build software so that it is modular and easy to extend. There are a few general factors that you should take into consideration:

separation of concerns
low coupling
high cohesion

Dependency Injection is a technique that favours these factors and in this blog post, I will try to explain why this is the case and how to use dependency injection with Python to improve your daily life and produced software.

It is a known fact DI (dependency injection) is not widely used inside Python, mostly because of Python’s scripting nature, but you – as an experienced developer – should already know that Python is not only a scripting language but is widely used in professional software development as well. So give me a chance to convince you 😉

Dependency injection - what is that really?

This is a technique build on the top of the Inversion of Control. The main idea is to separate the construction and usage of objects.

Let's consider the following example:

# bad case
class S3FileUploader(FileUploader):
    def __init__(self, bucket_name: str):
        self._bucket_name = bucket_name
        self._sdk_client = boto3.client("s3") # our dependency

S3FileUploader is an implementation of the FileUploader interface and is using boto s3 sdk client; this relationship is called a "dependency".

In the given example, the boto3.client("s3") construction is hardcoded inside S3FileUploader initialiser. This leads us to low cohesion and high coupling of our components, which might be an indication of a bad design.

The solution here is to delegate the responsibility related to initialising an object and inject the object initialised this way as our dependency.

Global state

Let’s be honest, the example above is not the worst-case scenario. Many times I've seen examples where a class depended on something from a global state (been there, done that, learnt my lesson 😅).


# worst case scenario
S3_SDK_CLIENT = boto3.client("s3")

class S3FileUploader(FileUploader):
    def __init__(self, bucket_name: str):
        self._bucket_name = bucket_name

    def upload_files(self, files: List[str]) -> None:
        for file in files:
			...
			S3_SDK_CLIENT.upload_file(...)
			...

Please do not rely on a global state!

There are plenty of conversations on the internet about why this is bad. Allow me to explain why this is not a good idea:

it breaks encapsulation – any other objects can change the state of it
testing is much harder – it requires a lot of mocks flying around

Dependency injection library

Let's revisit our previous example and apply the dependency injection technique.

class S3FileUploader(FileUploader):
    def __init__(self, bucket_name: str, s3_sdk_client: S3SdkClient):
        self._bucket_name = bucket_name
        self._client = s3_sdk_client # this is injected now!

Isn't it better? Now it is clearly visible what is required for this class to work.

To achieve this, we will need some dependency injection library. In all of my projects, I use kink (kodemore/kink) – it's a library created by my friend. 😉

It provides us with a dependency injection container with an elegant pythonic way of using it, together with just one Python decorator to mark classes and functions that need the injection.

In my opinion, it's a very flexible, friendly, and easy-to-use Python library. I encourage you to check out the GitHub page.

Examples

Setup

It's good to follow the convention, where all DI definitions are inside a file called: bootstrap.py

This is something that I came up with during a conversation with one of my friends, so if you have any better approaches, please let me know – I'm open to discussion.


# bootstrap.py

from kink import di
...


def bootstrap_di() -> None:
    logger = create_logger(os.getenv("LOG_LEVEL", "INFO"))
    app_config = AppConfig()
    aws_factory = AWSClientFactory(app_config)

    secret_manager_sdk_client = aws_factory.secret_manager_sdk_client()

    di[Logger] = logger
    di[AppConfig] = app_config

    di[SecretManagerSdkClient] = secret_manager_sdk_client
    di[S3SdkClient] = aws_factory.s3_sdk_client()
    di[S3SdkResource] = aws_factory.s3_sdk_resource()

    di[MongoClient] = lambda _di: get_database_connection(app_config, secret_manager_sdk_client)
    di[Database] = lambda _di: _di[MongoClient].get_database("test_db")
    di[UserDbal] = lambda _di: MongoUserDbal(_di[Database])

Then, in the main project directory inside __init__.py I invoke the above function.


# __init__.py

from di_example.bootstrap import bootstrap_di

bootstrap_di()

So right now I'm sure that all of my defined dependencies will be registered inside the dependency injection container.

In the case of kink, the dependency injection container is like a Python dict object. You can add new dependencies as you are adding new values to the Python dict, which is a cool feature in my opinion.

Also, you can register your dependencies in two ways:

by type
by name

So you can do both:


from kink import di
...

di["s3_sdk_client"] = aws_factory.s3_sdk_client()
# and
di[S3SdkClient] = aws_factory.s3_sdk_client()

I mainly use the second one because I'm a fan of typing in Python 😉

As you have seen, some of my definitions inside the bootstrap_di function use the lambda function. It's because kink supports on-demand service creation. It means that the creation of our dependency will not be executed until this is requested.

Ok, that's all in the case of setting up our DIC (dependency injection container), it's pretty simple, isn't it?

Usage

Ok, all of our services/dependencies are defined and waiting for use. Let's see how we can apply this to our code!


from kink import inject
...

@inject
class S3FileReader(FileReader):
    def __init__(self, bucket_name: str, s3_sdk_client: S3SdkClient, logger: Logger):
        self._bucket_name = bucket_name
        self._client = s3_sdk_client # this is injected now!
        self._logger = logger # this is injected now!

We used the @inject decorator which is doing the auto wiring of our dependencies. Generally speaking, auto wiring is a functionality that checks what's inside the DIC, and then if the type or name matches with what is defined inside the object's initialiser then kink will do the job and will inject exactly what is needed.

Simple, right? This is called constructor injection, but with kink, we can do the same also with functions. Let's consider another example.

from kink import inject
...

@inject
def example_lambda_handler(
event: LambdaEvent,
context: LambdaContext,
app_config: AppConfig, # this is injected
s3_sdk_client: S3SdkClient, # this is injected
secret_manager_sdk_client: SecretManagerSdkClient, # this is injected
logger: Logger, # this is injected
) -> Dict[str, str]:
logger.debug(f"Event = {event}")
logger.debug(f"Context = {context}")

Again the rules are the same as for constructor injection, kink will do the job and will resolve our dependencies automatically.

Benefits of using DI

it's much easier to follow SRP (Single Responsibility Principle)
the code is more reusable – you can inject your services in many places
it's much easier to test – you can inject mocks or test doubles of your dependencies
the code is more readable – you are only looking at behaviors of your components
it can improve the modularity of your application

And much more...

Problems with DI

DI will not resolve all of the problems automatically for you. As a developer, you have to be aware of the responsibilities and roles of your components.

There are far too many dependencies.

The main problem is the greed of our components. So, with an easy way to inject dependencies we are injecting "almost everything" into our component. What do you think, is this component doing only one thing? I will tell you: if it needs to be aware of so many dependencies, then it's definitely not doing one thing – this is against SRP. This is another indicator of bad design but we don't see it at first glance because we are happy with the ease of use of our DIC.

Greedy components should be refactored!

Consider the following example:


from kink import inject
...

@inject
def example_lambda_handler(
    event: LambdaEvent,
    context: LambdaContext,
    app_config: AppConfig,
    s3_sdk_client: S3SdkClient,
    secret_manager_sdk_client: SecretManagerSdkClient,
    dynamodb_sdk_client: DynamoDBSdkClient,
    step_functions_sdk_client: StepFunctionsSdkClient,
    mongo_user_dbal: UserDbal,
    logger: Logger,
) -> Dict[str, str]:
    logger.debug(f"Event = {event}")
    logger.debug(f"Context = {context}")

It's obvious that this controller has more than one responsibility, which is typical DI abuse. The above controller needs to be aware of many dependencies and has to handle many aspects of the business logic. Such examples should be considered bad design and DI misuse.

As it is with everything in life, if you misuse the DI, you can get your project in trouble. So in the end, you will end up with less readable code, it will be more difficult to manage, and you will lose all of the benefits I mentioned above. The final result will be counterproductive.

I would like to mention Uncle Bob's tweet, which is pretty old, but I think it allows you to understand how it works even better.

undefined