Warning

This documentation refers to latest GIT version of confix which has not been released yet.

confix documentation

About

Confix is a language-agnostic configuration parser for Python. It lets you define the default configuration of an app as a standard Python class, then overwrite its attributes from a static configuration file (be it YAML, JSON, INI or TOML) and / or via environment variables. This is useful to avoid storing sensitive data (e.g. passwords) in the source code and validate configuration on startup (via validators, mandatory attributes and type checking).

API reference

Exceptions

class confix.Error(msg)[source]

Base exception class from which derive all others.

class confix.ValidationError(msg)[source]

Raised when a confix.schema() validation fails. You can define a custom validator and have it raise this exception instead of returning False in order to provide a custom error message.

class confix.NotParsedError(msg)[source]

Called when get_parsed_conf() is called but confix.parse() has not been called yet.

class confix.AlreadyParsedError[source]

Raised when confix.parse() or confix.parse_with_envvars() is called twice.

class confix.AlreadyRegisteredError[source]

Raised by confix.register() when registering the same section twice.

class confix.NotParsedError[source]

Raised when confix.get_parsed_conf() is called but :func:`confix.parse() has not been called yet.

class confix.UnrecognizedKeyError[source]

Raised on parse if the configuration file defines a key which is not defined by the default configuration class. You’re not supposed to catch this but instead fix the configuration file.

class confix.RequiredKeyError[source]

Raised when the configuration file doesn’t specify a key which was required via schema(required=True). You’re not supposed to catch this but instead fix the configuration file.

class confix.TypesMismatchError[source]

Raised when configuration file overrides a key having a type which is different than the original one defined in the configuration class. You’re not supposed to catch this but instead fix the configuration file.

Functions

confix.register(section=None)

A decorator which registers a configuration class which will be parsed later. If section is None it is assumed that the configuration file will not be split in sub-sections otherwise section is the name of a specific section which will be referenced by the configuration file. All class attributes starting with an underscore will be ignored, same for methods, classmethods or any other non-callable type. A class decoratored with this method becomes dict()-able. Keys can be accessed as normal attributes or also as a dict. All attribute names starting with an underscore will be ignored. The class can also define classmethods.

confix.schema(default=_DEFAULT, required=False, validator=None)[source]

A schema can be used to validate configuration key’s values or state they are mandatory. default is the default key value. If required is True it is mandatory for the configuration file (or the environment variable) to specify that key. validator is a function or a list of functions which will be called for validating the overridden value. A validator function will fail if it returns False or raise ValidationError.

confix.parse(conf_file=None, file_parser=None, type_check=True)

Parse configuration class(es) replacing values if a configuration file is provided. conf_file is a path to a configuration file or an existing file-like object. If conf_file is None configuration class will be parsed anyway in order to validate its schemas (confix.schema()). file_parser is a callable parsing the configuration file and converting it to a dict. If None a default parser will be picked up depending on the file extension. You may want to override this either to support new file extensions or types. If type_check is True TypesMismatchError will be raised in case an an option specified in the configuration file has a different type than the one defined in the configuration class.

confix.parse_with_envvars(conf_file=None, file_parser=None, type_check=True, case_sensitive=False)

Same as confix.parse() but also takes environment variables into account. It must be noted that environment variables take precedence over the configuration file (if specified). Only upper cased environment variables are taken into account. By default (case_sensitive=False) environment variable "FOO" will override a key with the same name in a non case sensitive fashion ('foo', 'Foo', 'FOO', etc.). Also multiple “sections” are not supported so if multiple config classes define a key 'foo' all of them will be overwritten. If case_sensitive is True then it is supposed that the config class(es) define all upper cased keys.

confix.get_parsed_conf()[source]

Return the whole parsed configuration as a dict. If confix.parse() has not been called yet raise confix.NotParsedError.

Validators

Validators are simple utility functions which can be used with confix.schema() s.

confix.istrue(value)[source]

Assert value evaluates to True.

confix.isin(value, seq)[source]

Assert value is in a sequence.

confix.isnotin(value, seq)[source]

Assert value is not in a sequence.

confix.isemail(value)[source]

Assert value is a valid email.

Usage by examples

Override a key via configuration file

python file:

# main.py
from confix import register, parse

@register()
class config:
    username = 'ftp'
    password = None

parse('config.yaml')
print(config.username)
print(config.password)

configuration file:

# config.yml
password: secret

shell:

$ python main.py
ftp
secret
Things to note:
  • password got changed by configuration file.
  • parse() did the trick.
  • configuration fields (“keys”) can be accessed as attributes (config.name).

Override a key via environment variables

python file:

# main.py
from confix import register, parse_with_envvars

@register()
class config:
    username = 'ftp'
    password = None

parse_with_envvars()
print(config.username)
print(config.password)

shell:

$ PASSWORD=secret python main.py
ftp
secret
Things to note:
  • "PASSWORD" environment variable changed the value of "password" class attribute. key which is treated in a case insensitive fashion.
  • to change this behavior use parse_with_envvars(case_sensitive=True)) but in that case also the class attributed must be upper case ("PASSWORD").

Using configuration file and environment variables

You can overwrite default configuration by using both a configuration file and environment variables. Environment variables take precedence over the configuration file though.

python file:

# main.py
from confix import register, parse_with_envvars

@register()
class config:
    username = 'ftp'
    password = None
    host = localhost

parse_with_envvars(config_file='config.yml')
print(config.username)
print(config.password)
print(config.host)
# config.yml
username: john
password: secret
host: localhost

shell:

$ PASSWORD=somecrazypass python main.py
john
somecrazypass
localhost
Things to note:
  • "password" was specified in the configuration file but also by the environment variable and this takes precedence over the configuration file.

Errors: configuration definition

One of the key features of confix is that the config class is a definition of all your app configuration. If the configuration file declares a key which is not defined in the config class confix will error out. This is useful in case you made a typo in your configuration file: failing sooner (application startup) rather than later is better.

# main.py
from confix import register, parse

@register()
class config:
    username = 'ftp'
    password = None

parse()

configuration file:

# config.yml
host: localhost

shell:

$ python main.py
Traceback (most recent call last):
  File "main.py", line 9, in <module>
    parse('config.yaml')
  File "/home/giampaolo/svn/confix/confix.py", line 473, in parse
    type_check=type_check)
  File "/home/giampaolo/svn/confix/confix.py", line 289, in __init__
    self.process_conf(conf)
  File "/home/giampaolo/svn/confix/confix.py", line 378, in process_conf
    section=None)
  File "/home/giampaolo/svn/confix/confix.py", line 393, in process_pair
    raise UnrecognizedKeyError(key, new_value, section=section)
confix.UnrecognizedKeyError: configuration file provides key 'host' with value 'localhost' but key 'host' is not defined in the config class
Things to note:
  • key 'host' was specified in the configuration file but not in the default config class.

Errors: types checking

Each key in the config class (may) have a default value. By default confix will raise an exception if the value overwritten by the configuration file (or environment variable) has a different type. This can be disabled with parse('config.yaml', type_check=False).

python file:

# main.py
from confix import register, parse

@register()
class config:
    host = 'localhost'
    port = 80

parse('config.yaml')

configuration file:

# config.yml
host: 10.0.0.1
port: foo

shell:

$ python main.py
Traceback (most recent call last):
  File "main.py", line 9, in <module>
    parse('config.yaml')
  File "/home/giampaolo/svn/confix/confix.py", line 473, in parse
    type_check=type_check)
  File "/home/giampaolo/svn/confix/confix.py", line 289, in __init__
    self.process_conf(conf)
  File "/home/giampaolo/svn/confix/confix.py", line 378, in process_conf
    section=None)
  File "/home/giampaolo/svn/confix/confix.py", line 415, in process_pair
    section=section)
confix.TypesMismatchError: type mismatch for key 'port' (default_value=80) got 'foo'

Required arguments

You can force certain arguments to be required, meaning they have to be specified via configuration file or environment variable.

python file:

# main.py
from confix import register, parse_with_envvars, schema

@register()
class config:
    username = 'ftp'
    password = schema(None, required=True)

parse_with_envvars('config.yaml')
print(config.password)

configuration file:

# config.yml

shell:

$ python main.py
Traceback (most recent call last):
  File "main.py", line 9, in <module>
    parse_with_envvars('config.yaml')
  File "/home/giampaolo/svn/confix/confix.py", line 501, in parse_with_envvars
    envvar_case_sensitive=case_sensitive)
  File "/home/giampaolo/svn/confix/confix.py", line 291, in __init__
    self.process_conf(conf)
  File "/home/giampaolo/svn/confix/confix.py", line 382, in process_conf
    self.run_last_schemas()
  File "/home/giampaolo/svn/confix/confix.py", line 449, in run_last_schemas
    raise RequiredKeyError(key, section=section)
confix.RequiredKeyError: configuration class requires 'password' key to be specified via configuration file or environment variable
$
$ PASSWORD=secret python main.py
secret

Validators

A validator is function which is called to validate the value overridden by the configuration file (or environment variable). If the function returns False or raise confix.ValidationError the validation will fail. In this example we provide a validator which checks the password length. Also, it’s required.

python file:

# main.py
from confix import register, parse_with_envvars, schema

@register()
class config:
    username = 'ftp'
    password = schema(None, required=True, validator=lambda x: len(x) => 6)

parse_with_envvars()
print(config.password)

shell:

$ PASSWORD=foo python main.py
Traceback (most recent call last):
  File "main.py", line 9, in <module>
    parse_with_envvars()
  File "/home/giampaolo/svn/confix/confix.py", line 501, in parse_with_envvars
    envvar_case_sensitive=case_sensitive)
  File "/home/giampaolo/svn/confix/confix.py", line 291, in __init__
    self.process_conf(conf)
  File "/home/giampaolo/svn/confix/confix.py", line 380, in process_conf
    section=None)
  File "/home/giampaolo/svn/confix/confix.py", line 434, in process_pair
    raise exc
confix.ValidationError: 'password' key with value 'foo' didn't pass validation
$
$ PASSWORD=longpassword python main.py
longpassword

Marking keys as mandatory

Certain keys can be marked as mandatory, meaning if they are not specified in the configuration file (or via environment variable) confix will error out. This is useful to avoid storing sensitive data (e.g. passwords) in the source code.

# main.py
from confix import register, schema, parse

@register()
class config:
    password = schema(None, required=True)

parse()
$ python main.py
Traceback (most recent call last):
  File "main.py", line 7, in <module>
    parse()
  File "/home/giampaolo/svn/confix/confix.py", line 693, in parse
    type_check=type_check)
  File "/home/giampaolo/svn/confix/confix.py", line 443, in __init__
    self.process_conf(self.new_conf)
  File "/home/giampaolo/svn/confix/confix.py", line 574, in process_conf
    self.run_last_schemas()
  File "/home/giampaolo/svn/confix/confix.py", line 664, in run_last_schemas
    raise RequiredKeyError(key, section=section)
confix.RequiredKeyError: configuration class requires 'password' key to be specified via configuration file or environment variable

Default validators

confix provides a bunch of validators by default. This example shows all of them:

# main.py
from confix import register, schema, istrue, isin, isnotin, isemail

@register()
class config:
    username = schema('john', validator=istrue)
    status = schema('active', validator=isin(['active', inactive]))
    password = schema(None, mandatory=True,
                      validator=isnotin(['12345', 'password']))
    email = schema('user@domain.com', validator=isemail)

Chained validators

You can define more than one validator per-schema:

# main.py
from confix import register, schema, isemail, isnotin,

@register()
class config:
    email = schema('user@domain.com',
                   validator=[isemail, isnoin(['info@domain.com']))

Custom validators

A validator is a function which receives the overidden value as first argument and fails if it does not return True. confix.ValidationError exception can be raised instead of returning False to provide a detailed error message. Example of a custom validator:

# main.py
from confix import register, parse_with_envvars, schema, ValidationError

def validate_password(value):
    if len(value) < 6:
        raise ValidationError("password is too short (< 6 chars)")
    elif value in ("password", "123456"):
        raise ValidationError("password is too fragile")
    return True

@register()
class config:
    username = 'ftp'
    password = schema(None, required=True, validator=validate_password)

parse_with_envvars()
print(config.password)

Multiple configuration classes

You may want to do this in case you have an app with different components and you want to control everything from a single configuration file having different sections. Example:

python file:

# main.py
from confix import register, parse

@register()
class config:
    debug = False

@register(section='ftp')
class ftp_config:
    port = 21
    username = 'ftp'

@register(section='http')
class http_config:
    port = 80
    username = 'www'

parse('config.yaml')
print(ftp_config.port)
print(ftp_config.username)
print(http_config.port)
print(http_config.username)

configuration file:

# config.yml
ftp:
    username: ftp-custom
http:
    username: http-custom

shell:

$ python main.py
21
ftp-custom
80
http-custom
Things to note:
  • if we would have used parse_with_envvars() and specified a USERNAME environment variable via cmdline username key of both config classes would have been overwritten.
  • we may also have defined a third “root” config class, with no section.

Notes about @register

Classes registered via confix.register() decorator have a bunch of peculiarities:

  • attributes starting with an underscore will be ignored.
  • attributes can be accessed both as normal attributes (config.foo) and as a dict (config['foo']).
  • dict() can be used against the registered class in order to get the whole configuration.
  • the config class can have class methods.
>>> import confix
>>>
>>> @confix.register()
>>> class config:
...     foo = 1
...     bar = 2
...     _apple = 3
...
>>> config.foo
1
>>> config['foo']
1
>>> dict(config)
{'foo': 1, 'bar': 2}
>>>

INI files

INI files are supported but since they are based on “sections” also your configuration class(es) must have sections.

# main.py
from confix import register, parse

@register()
class config:
    foo = 2

parse()
$ python main.py
Traceback (most recent call last):
  File "main.py", line 8, in <module>
    parse('config.ini')
  File "/home/giampaolo/svn/confix/confix.py", line 693, in parse
    type_check=type_check)
  File "/home/giampaolo/svn/confix/confix.py", line 440, in __init__
    self.new_conf = self.get_conf_from_file()
  File "/home/giampaolo/svn/confix/confix.py", line 483, in get_conf_from_file
    raise Error("can't parse ini files if a sectionless "
confix.Error: can't parse ini files if a sectionless configuration class has been registered

This means that if you have an INI file you must define multiple configuration classes, each one with a different section name.

Supporting other file formats

By default confix supports YAML, JSON, INI and TOML configuration formats. If you want to add a new format you can write a parser for that specific format as a function, have it return a dict and pass it to :func`confix.parse()`. Example:

# main.py
from confix import register, parse

@register()
class config:
    foo = 1

def parse_new_format():
    return {}

parse('config.ext', file_parser=parse_new_format)