Best Practice Configuration with Pydantic-Settings and Environment Variables

The Problem

  • Deploy port number
  • Database connection string
  • API keys
  • Logging level
  • yaml? env variable?
  • etc.

There are a lot of things that need to be configured in an application. If you don’t start well from the start you will end up with a mess of configuration management.

The Solution

Using pydantic-settings is considered a best practice for configuration management in Python applications. It gives you a single source of truth for your configuration. You can easily use it with environment variables. Everything is (can be) validated and type-checked.

Incorporating pydantic-settings also helps using pydantic for other parts of your application!

Tip

Those tools are stable, but are also under active development. It’s good to revisit them from time to time to see if they have new features!

How to use them

Tapyr only configures the logging level to keep things minimalistic. Please refer to the Pydantic-Settings documentation for more information.

Environment Variables and YAML/TOML files

You should never hardcode sensitive information like API keys in your code. Instead, use environment variables. All deployment platforms support environment variables. You can set them in your .env file or directly in your deployment platform.

Then for not sensitive configuration you may want to use yaml/toml files. The pydantic-settings handles all of them out-of-the-box, more information in its documentation.

Why not simple config.py?

Pydantic-Settings provides some immensely useful features that are hard to replicate with a simple config.py file out of the box.

  • Type checking - Is the value of the correct type?
  • Validation - Is the value within the correct range?
  • Settings discovery - Where do the settings come from? It automatically checks environment variables, .env files, and more.
  • Since recently it also allows overriding settings from CLI!