Configuration Loaders

Loaders are in charge of loading configuration from various sources, like .ini files or environment variables, and expose configuration as a dict-like object. Loaders are ment to be chained, so that classyconf checks one by one for a given configuration variable.

If a loader doesn’t find the configuration variable it raises a KeyError so that the next loader get’s checked. If no loader returns any value, and no default value was set, an UnknownConfiguration exception is thrown.

Classyconf comes with some loaders already included in classyconf.loaders.

By default the library will check the environment with the Environment loader. You can change that behaviour, by customizing the loaders and the order in wich configuration discovery happens.

Loaders can be set in the Meta class when extending Configuration or passed as a param when instantiating it. The later takes precedence and overrides loaders defined in Meta. The order within the list of loaders matters and defines the lookup order.

from classyconf import Configuration, IniFile, Environment, Value, EnvFile

class AppConfig(Configuration):

    class Meta:
        loaders = [
            Environment(),
            IniFile("/path/to/config.ini")
        ]

    DEBUG = Value(default=False)
    OTHER_CONFIG = Value(default=0)


# Checks for enviroment variables first
# Then lookup settings in the `config.ini` file
config = AppConfig()

# Override loaders and only lookup in the `.env` file
test_config = AppConfig(loaders=[EnvFile("/path/to/.env")])

Naming conventions and namespaces for settings

There happen to be some formatting conventions for configuration parameters based on where they are set. For example, it is common to name environment variables in uppercase:

$ DEBUG=yes OTHER_CONFIG=10 ./app.py

Since the environment is a global and shared dictionary, it is a good practice to also apply some prefix to each setting to avoid collisions with other known settings, like LOCALE, TZ, etc. This prefix works as a namespace for your app.

$ MY_APP_DEBUG=yes MY_APP_OTHER_CONFIG=10 ./app.py

but if you were to set this config in an .ini file, each setting should probably be in lower case, the namespace is implicit in the file path, i.e: /etc/myapp/config.ini.

[settings]
debug=yes
other_config=10

Command line arguments have yet another conventions:

$ ./app.py --debug=yes --another-config=10

Classyconf let’s you follow these aesthetics patterns by setting a keyfmt function when instantiating the loaders.

By default, the Environment is instantiated with keyfmt=EnvPrefix('') so that it looks for UPPER_CASED settings. But it can be easyly tweaked to address the prefix issue by using keyfmt=EnvPrefix("MY_APP_"), and look for MY_APP_UPPER_CASED to play nice with other env variables.

from classyconf import Configuration, IniFile, Environment, Value, EnvPrefix

class AppConfig(Configuration):

    class Meta:
        loaders = [
            Environment(keyfmt=EnvPrefix(prefix="MY_APP_")),
            IniFile("/etc/myapp/config.ini")
        ]

    DEBUG = Value(default=False)
    OTHER_CONFIG = Value(default=0)

config = AppConfig()

# looks for `MY_APP_DEBUG` in environment, then  `debug` in `settings` section of config.ini
config.DEBUG

Keep reading to find out more about different loaders and their configurations.

Environment

class classyconf.loaders.Environment(keyfmt=EnvPrefix(""))

Get’s configuration from the environment, by inspecting os.environ.

Parameters:keyfmt (function) – A function to pre-format variable names.

The Environment loader gets configuration from os.environ. Since it is a common pattern to write env variables in caps, the loader accepts a keyfmt function to pre-format the variable name before the lookup occurs. By default it is EnvPrefix("") which combines str.upper() and an empty prefix.

Note

In the case of CLI apps, it would be recommended to set some sort of namespace so that you don’t accidentally override other programs behaviour, like LOCALE or EDITOR, but instead MY_APP_LOCALE, etc. So consider using the EnvPrefix("MY_APP_") approach.

from classyconf import Configuration, Environment, Value

class AppConf(Configuration):
    debug = Value(default=False)


config = AppConf(loaders=[Environment(keyfmt=str.upper)])
config.debug  # will look for a `DEBUG` variable

EnvFile

class classyconf.loaders.EnvFile(filename='.env', keyfmt=EnvPrefix(""))
Parameters:
  • filename (str) – Path to the .env file.
  • keyfmt (function) – A function to pre-format variable names.

The EnvFile loader gets configuration from .env file. If the file doesn’t exist, this loader will be skipped without raising any errors.

# .env file
DEBUG=1

from classyconf import Configuration, EnvFile, Value

class AppConf(Configuration):
    debug = Value(default=False)


config = AppConf(loaders=[EnvFile(file='.env', keyfmt=str.upper)])
config.debug  # will look for a `DEBUG` variable instead of `debug`

Note

You might want to use dump-env, a utility to create .env files.

IniFile

class classyconf.loaders.IniFile(filename, section='settings', keyfmt=<function IniFile.<lambda>>)
Parameters:
  • filename (str) – Path to the .ini/.cfg file.
  • section (str) – Section name inside the config file.
  • keyfmt (function) – A function to pre-format variable names.

The IniFile loader gets configuration from .ini or .cfg files. If the file doesn’t exist, this loader will be skipped without raising any errors.

CommandLine

class classyconf.loaders.CommandLine(parser, get_args=<function get_args>)

Extract configuration from an argparse parser.

Parameters:
  • parser (argparse.ArgumentParser) – An argparse parser instance to extract variables from.
  • get_args (function) – A function to extract args from the parser.

This loader lets you extract configuration variables from parsed command line arguments. By default it works with argparse parsers.

import argparse
from classyconf import Configuration, Value, NOT_SET, CommandLine


parser = argparse.ArgumentParser(description='Does something useful.')
parser.add_argument('--debug', '-d', dest='debug', default=NOT_SET, help='set debug mode')

class AppConf(Configuration):
    DEBUG = Value(default=False)

config = AppConf(loaders=[CommandLine(parser=parser)])
print(config.DEBUG)

Something to notice here is the NOT_SET value. CLI parsers often force you to put a default value so that they don’t fail. In that case, to play nice with classyconf, you must set one. But that would break the discoverability chain that classyconf encourages. So by setting this special default value, you will allow classyconf to keep the lookup going.

The get_args function converts the argparse parser’s values to a dict that ignores NOT_SET values.

Dict

class classyconf.loaders.Dict(values_mapping)
Parameters:values_mapping (dict) – A dictionary of hardcoded settings.

This loader is great when you want to pin certain settings without having to change/override other loaders, files or defaults. It really comes handy when you are extending a Configuration class.

from classyconf import Configuration, Value, IniFile, Dict

class AppConfig(Configuration):
    class Meta:
        loaders = [IniFile("/opt/myapp/config.ini"), IniFile("/etc/myapp/config.ini")]

    NUMBER = Value(default=1)
    DEBUG = Value(default=False)
    LABEL = Value(default="foo")
    OTHER  = Value(default="bar")


class TestConfig(AppConfig):
    class Meta:
        loders = [Dict({"DEBUG": True, "NUMBER": 0})]

RecursiveSearch

class classyconf.loaders.RecursiveSearch(starting_path=None, filetypes=(('.env', <class 'classyconf.loaders.EnvFile'>), (('*.ini', '*.cfg'), <class 'classyconf.loaders.IniFile'>)), root_path='/')
Parameters:
  • starting_path (str) – The path to begin looking for configuration files.
  • filetypes (tuple) – tuple of tuples with configuration loaders, order matters. Defaults to (('*.env', EnvFile), (('*.ini', *.cfg',), IniFile)
  • root_path (str) – Configuration lookup will stop at the given path. Defaults to the current user directory

This loader tries to find .env or *.ini|*.cfg files and load them with the EnvFile and IniFile loaders respectively.

It will start looking at the starting_path directory for configuration files and walking up the filesystem tree until it finds any or reaches the root_path.

Warning

It is important to note that this loader uses the glob module internally to discover .env and *.ini|*.cfg files. This could be problematic if the project includes many files that are unrelated, like a pytest.ini file along side with a settings.ini. An unexpected file could be found and be considered as the configuration to use.

Consider the following file structure:

project/
  settings.ini
  app/
    settings.py

When instantiating your RecursiveSearch, if you pass /absolute/path/to/project/app/ as starting_path the loader will start looking for configuration files at project/app.

# Code example in project/app/settings.py
import os

from classyconf import config
from classyconf.loaders import RecursiveSearch

app_path = os.path.dirname(__file__)
config.loaders = [RecursiveSearch(starting_path=app_path)]

By default, the loader will try to look for configuration files until it finds valid configuration files or it reaches root_path. The root_path is set to the root directory / initialy.

Suppose the following file structure:

projects/
  any_settings.ini
  project/
    app/
      settings.py

You can change this behaviour by setting any parent directory of the starting_path as the root_path when instantiating RecursiveSearch:

# Code example in project/app/settings.py
import os

from classyconf import Configuration
from classyconf.loaders import RecursiveSearch

app_path = os.path.dirname(__file__)
project_path = os.path.realpath(os.path.join(app_path, '..'))
rs = RecursiveSearch(starting_path=app_path, root_path=project_path)
config = Configuration(loaders=[rs])

The example above will start looking for files at project/app/ and will stop looking for configuration files at project/, actually never looking at any_settings.ini and no configuration being loaded at all.

The root_path must be a parent directory of starting_path, otherwise it raises an InvalidPath exception:

from classyconf.loaders import RecursiveSearch

# /baz is not parent of /foo/bar, so this raises an InvalidPath exception here
rs = RecursiveSearch(starting_path="/foo/bar", root_path="/baz")

Writing your own loader

If you need a custom loader, you should just extend the AbstractConfigurationLoader.

class classyconf.loaders.AbstractConfigurationLoader

For example, say you want to write a Yaml loader. It is important to note that by raising a KeyError exception from the loader, classyconf knows that it has to keep looking down the loaders chain for a specific config.

import yaml
from classyconf.loaders import AbstractConfigurationLoader


class YamlFile(AbstractConfigurationLoader):
    def __init__(self, filename, keyfmt=str.lower):
        self.filename = filename
        self.config = None
        self.keyfmt = keyfmt

    def _parse(self):
        if self.config is not None:
            return
        with open(self.filename, 'r') as f:
            self.config = yaml.load(f)

    def __contains__(self, item):
        try:
            self._parse()
        except:
            return False

        return self.keyfmt(item) in self.config

    def __getitem__(self, item):
        try:
            self._parse()
        except:
            # KeyError tells classyconf to keep looking elsewhere!
            raise KeyError("{!r}".format(item))

        return self.config[self.keyfmt(item)]

    def reset(self):
        self.config = None

Then configure classyconf to use it.

from classyconf import Configuration

class AppConf(Configuration):
    class Meta:
        loaders = [YamlFile('/path/to/config.yml')]