Warning
This documentation refers to latest GIT version of confix which has not been released yet.
About¶
Confix is a language-agnostic configuration parser for Python. It lets you define the default configuration of an app as a standard Python class, then overwrite its attributes from a static configuration file (be it YAML, JSON, INI or TOML) and / or via environment variables. This is useful to avoid storing sensitive data (e.g. passwords) in the source code and validate configuration on startup (via validators, mandatory attributes and type checking).
API reference¶
Exceptions
-
class
confix.
ValidationError
(msg)[source]¶ Raised when a
confix.schema()
validation fails. You can define a custom validator and have it raise this exception instead of returning False in order to provide a custom error message.
-
class
confix.
NotParsedError
(msg)[source]¶ Called when
get_parsed_conf()
is called butconfix.parse()
has not been called yet.
-
class
confix.
AlreadyParsedError
[source]¶ Raised when
confix.parse()
orconfix.parse_with_envvars()
is called twice.
-
class
confix.
AlreadyRegisteredError
[source]¶ Raised by
confix.register()
when registering the same section twice.
-
class
confix.
NotParsedError
[source] Raised when
confix.get_parsed_conf()
is called but :func:`confix.parse() has not been called yet.
-
class
confix.
UnrecognizedKeyError
[source]¶ Raised on parse if the configuration file defines a key which is not defined by the default configuration class. You’re not supposed to catch this but instead fix the configuration file.
-
class
confix.
RequiredKeyError
[source]¶ Raised when the configuration file doesn’t specify a key which was required via
schema(required=True)
. You’re not supposed to catch this but instead fix the configuration file.
-
class
confix.
TypesMismatchError
[source]¶ Raised when configuration file overrides a key having a type which is different than the original one defined in the configuration class. You’re not supposed to catch this but instead fix the configuration file.
Functions
-
confix.
register
(section=None)¶ A decorator which registers a configuration class which will be parsed later. If section is
None
it is assumed that the configuration file will not be split in sub-sections otherwise section is the name of a specific section which will be referenced by the configuration file. All class attributes starting with an underscore will be ignored, same for methods, classmethods or any other non-callable type. A class decoratored with this method becomes dict()-able. Keys can be accessed as normal attributes or also as a dict. All attribute names starting with an underscore will be ignored. The class can also define classmethods.
-
confix.
schema
(default=_DEFAULT, required=False, validator=None)[source]¶ A schema can be used to validate configuration key’s values or state they are mandatory. default is the default key value. If required is
True
it is mandatory for the configuration file (or the environment variable) to specify that key. validator is a function or a list of functions which will be called for validating the overridden value. A validator function will fail if it returnsFalse
or raiseValidationError
.
-
confix.
parse
(conf_file=None, file_parser=None, type_check=True)¶ Parse configuration class(es) replacing values if a configuration file is provided. conf_file is a path to a configuration file or an existing file-like object. If conf_file is
None
configuration class will be parsed anyway in order to validate its schemas (confix.schema()
). file_parser is a callable parsing the configuration file and converting it to a dict. IfNone
a default parser will be picked up depending on the file extension. You may want to override this either to support new file extensions or types. If type_check is True TypesMismatchError will be raised in case an an option specified in the configuration file has a different type than the one defined in the configuration class.
-
confix.
parse_with_envvars
(conf_file=None, file_parser=None, type_check=True, case_sensitive=False)¶ Same as
confix.parse()
but also takes environment variables into account. It must be noted that environment variables take precedence over the configuration file (if specified). Only upper cased environment variables are taken into account. By default (case_sensitive=False
) environment variable"FOO"
will override a key with the same name in a non case sensitive fashion ('foo'
,'Foo'
,'FOO'
, etc.). Also multiple “sections” are not supported so if multiple config classes define a key'foo'
all of them will be overwritten. If case_sensitive isTrue
then it is supposed that the config class(es) define all upper cased keys.
-
confix.
get_parsed_conf
()[source]¶ Return the whole parsed configuration as a dict. If
confix.parse()
has not been called yet raiseconfix.NotParsedError
.
Validators
Validators are simple utility functions which can be used with
confix.schema()
s.
Usage by examples¶
Override a key via configuration file¶
python file:
# main.py
from confix import register, parse
@register()
class config:
username = 'ftp'
password = None
parse('config.yaml')
print(config.username)
print(config.password)
configuration file:
# config.yml
password: secret
shell:
$ python main.py
ftp
secret
- Things to note:
password
got changed by configuration file.parse()
did the trick.- configuration fields (“keys”) can be accessed as attributes
(
config.name
).
Override a key via environment variables¶
python file:
# main.py
from confix import register, parse_with_envvars
@register()
class config:
username = 'ftp'
password = None
parse_with_envvars()
print(config.username)
print(config.password)
shell:
$ PASSWORD=secret python main.py
ftp
secret
- Things to note:
"PASSWORD"
environment variable changed the value of"password"
class attribute. key which is treated in a case insensitive fashion.- to change this behavior use
parse_with_envvars(case_sensitive=True))
but in that case also the class attributed must be upper case ("PASSWORD"
).
Using configuration file and environment variables¶
You can overwrite default configuration by using both a configuration file and environment variables. Environment variables take precedence over the configuration file though.
python file:
# main.py
from confix import register, parse_with_envvars
@register()
class config:
username = 'ftp'
password = None
host = localhost
parse_with_envvars(config_file='config.yml')
print(config.username)
print(config.password)
print(config.host)
# config.yml
username: john
password: secret
host: localhost
shell:
$ PASSWORD=somecrazypass python main.py
john
somecrazypass
localhost
- Things to note:
"password"
was specified in the configuration file but also by the environment variable and this takes precedence over the configuration file.
Errors: configuration definition¶
One of the key features of confix is that the config class is a definition of all your app configuration. If the configuration file declares a key which is not defined in the config class confix will error out. This is useful in case you made a typo in your configuration file: failing sooner (application startup) rather than later is better.
# main.py
from confix import register, parse
@register()
class config:
username = 'ftp'
password = None
parse()
configuration file:
# config.yml
host: localhost
shell:
$ python main.py
Traceback (most recent call last):
File "main.py", line 9, in <module>
parse('config.yaml')
File "/home/giampaolo/svn/confix/confix.py", line 473, in parse
type_check=type_check)
File "/home/giampaolo/svn/confix/confix.py", line 289, in __init__
self.process_conf(conf)
File "/home/giampaolo/svn/confix/confix.py", line 378, in process_conf
section=None)
File "/home/giampaolo/svn/confix/confix.py", line 393, in process_pair
raise UnrecognizedKeyError(key, new_value, section=section)
confix.UnrecognizedKeyError: configuration file provides key 'host' with value 'localhost' but key 'host' is not defined in the config class
- Things to note:
- key
'host'
was specified in the configuration file but not in the default config class.
- key
Errors: types checking¶
Each key in the config class (may) have a default value. By default confix will
raise an exception if the value overwritten by the configuration file (or
environment variable) has a different type. This can be disabled with
parse('config.yaml', type_check=False)
.
python file:
# main.py
from confix import register, parse
@register()
class config:
host = 'localhost'
port = 80
parse('config.yaml')
configuration file:
# config.yml
host: 10.0.0.1
port: foo
shell:
$ python main.py
Traceback (most recent call last):
File "main.py", line 9, in <module>
parse('config.yaml')
File "/home/giampaolo/svn/confix/confix.py", line 473, in parse
type_check=type_check)
File "/home/giampaolo/svn/confix/confix.py", line 289, in __init__
self.process_conf(conf)
File "/home/giampaolo/svn/confix/confix.py", line 378, in process_conf
section=None)
File "/home/giampaolo/svn/confix/confix.py", line 415, in process_pair
section=section)
confix.TypesMismatchError: type mismatch for key 'port' (default_value=80) got 'foo'
Required arguments¶
You can force certain arguments to be required, meaning they have to be specified via configuration file or environment variable.
python file:
# main.py
from confix import register, parse_with_envvars, schema
@register()
class config:
username = 'ftp'
password = schema(None, required=True)
parse_with_envvars('config.yaml')
print(config.password)
configuration file:
# config.yml
shell:
$ python main.py
Traceback (most recent call last):
File "main.py", line 9, in <module>
parse_with_envvars('config.yaml')
File "/home/giampaolo/svn/confix/confix.py", line 501, in parse_with_envvars
envvar_case_sensitive=case_sensitive)
File "/home/giampaolo/svn/confix/confix.py", line 291, in __init__
self.process_conf(conf)
File "/home/giampaolo/svn/confix/confix.py", line 382, in process_conf
self.run_last_schemas()
File "/home/giampaolo/svn/confix/confix.py", line 449, in run_last_schemas
raise RequiredKeyError(key, section=section)
confix.RequiredKeyError: configuration class requires 'password' key to be specified via configuration file or environment variable
$
$ PASSWORD=secret python main.py
secret
Validators¶
A validator is function which is called to validate the value overridden by the
configuration file (or environment variable). If the function returns False
or raise confix.ValidationError
the validation will fail.
In this example we provide a validator which checks the password length.
Also, it’s required
.
python file:
# main.py
from confix import register, parse_with_envvars, schema
@register()
class config:
username = 'ftp'
password = schema(None, required=True, validator=lambda x: len(x) => 6)
parse_with_envvars()
print(config.password)
shell:
$ PASSWORD=foo python main.py
Traceback (most recent call last):
File "main.py", line 9, in <module>
parse_with_envvars()
File "/home/giampaolo/svn/confix/confix.py", line 501, in parse_with_envvars
envvar_case_sensitive=case_sensitive)
File "/home/giampaolo/svn/confix/confix.py", line 291, in __init__
self.process_conf(conf)
File "/home/giampaolo/svn/confix/confix.py", line 380, in process_conf
section=None)
File "/home/giampaolo/svn/confix/confix.py", line 434, in process_pair
raise exc
confix.ValidationError: 'password' key with value 'foo' didn't pass validation
$
$ PASSWORD=longpassword python main.py
longpassword
Marking keys as mandatory¶
Certain keys can be marked as mandatory, meaning if they are not specified in the configuration file (or via environment variable) confix will error out. This is useful to avoid storing sensitive data (e.g. passwords) in the source code.
# main.py
from confix import register, schema, parse
@register()
class config:
password = schema(None, required=True)
parse()
$ python main.py
Traceback (most recent call last):
File "main.py", line 7, in <module>
parse()
File "/home/giampaolo/svn/confix/confix.py", line 693, in parse
type_check=type_check)
File "/home/giampaolo/svn/confix/confix.py", line 443, in __init__
self.process_conf(self.new_conf)
File "/home/giampaolo/svn/confix/confix.py", line 574, in process_conf
self.run_last_schemas()
File "/home/giampaolo/svn/confix/confix.py", line 664, in run_last_schemas
raise RequiredKeyError(key, section=section)
confix.RequiredKeyError: configuration class requires 'password' key to be specified via configuration file or environment variable
Default validators¶
confix provides a bunch of validators by default. This example shows all of them:
# main.py
from confix import register, schema, istrue, isin, isnotin, isemail
@register()
class config:
username = schema('john', validator=istrue)
status = schema('active', validator=isin(['active', inactive]))
password = schema(None, mandatory=True,
validator=isnotin(['12345', 'password']))
email = schema('user@domain.com', validator=isemail)
Chained validators¶
You can define more than one validator per-schema:
# main.py
from confix import register, schema, isemail, isnotin,
@register()
class config:
email = schema('user@domain.com',
validator=[isemail, isnoin(['info@domain.com']))
Custom validators¶
A validator is a function which receives the overidden value as first argument
and fails if it does not return True
. confix.ValidationError
exception
can be raised instead of returning False
to provide a detailed error
message. Example of a custom validator:
# main.py
from confix import register, parse_with_envvars, schema, ValidationError
def validate_password(value):
if len(value) < 6:
raise ValidationError("password is too short (< 6 chars)")
elif value in ("password", "123456"):
raise ValidationError("password is too fragile")
return True
@register()
class config:
username = 'ftp'
password = schema(None, required=True, validator=validate_password)
parse_with_envvars()
print(config.password)
Multiple configuration classes¶
You may want to do this in case you have an app with different components and you want to control everything from a single configuration file having different sections. Example:
python file:
# main.py
from confix import register, parse
@register()
class config:
debug = False
@register(section='ftp')
class ftp_config:
port = 21
username = 'ftp'
@register(section='http')
class http_config:
port = 80
username = 'www'
parse('config.yaml')
print(ftp_config.port)
print(ftp_config.username)
print(http_config.port)
print(http_config.username)
configuration file:
# config.yml
ftp:
username: ftp-custom
http:
username: http-custom
shell:
$ python main.py
21
ftp-custom
80
http-custom
- Things to note:
- if we would have used
parse_with_envvars()
and specified aUSERNAME
environment variable via cmdlineusername
key of both config classes would have been overwritten. - we may also have defined a third “root” config class, with no section.
- if we would have used
Notes about @register¶
Classes registered via confix.register()
decorator have a bunch of
peculiarities:
- attributes starting with an underscore will be ignored.
- attributes can be accessed both as normal attributes (
config.foo
) and as adict
(config['foo']
).dict()
can be used against the registered class in order to get the whole configuration.- the config class can have class methods.
>>> import confix
>>>
>>> @confix.register()
>>> class config:
... foo = 1
... bar = 2
... _apple = 3
...
>>> config.foo
1
>>> config['foo']
1
>>> dict(config)
{'foo': 1, 'bar': 2}
>>>
INI files¶
INI files are supported but since they are based on “sections” also your configuration class(es) must have sections.
# main.py
from confix import register, parse
@register()
class config:
foo = 2
parse()
$ python main.py
Traceback (most recent call last):
File "main.py", line 8, in <module>
parse('config.ini')
File "/home/giampaolo/svn/confix/confix.py", line 693, in parse
type_check=type_check)
File "/home/giampaolo/svn/confix/confix.py", line 440, in __init__
self.new_conf = self.get_conf_from_file()
File "/home/giampaolo/svn/confix/confix.py", line 483, in get_conf_from_file
raise Error("can't parse ini files if a sectionless "
confix.Error: can't parse ini files if a sectionless configuration class has been registered
This means that if you have an INI file you must define multiple configuration classes, each one with a different section name.
Supporting other file formats¶
By default confix supports YAML, JSON, INI and TOML configuration formats. If you want to add a new format you can write a parser for that specific format as a function, have it return a dict and pass it to :func`confix.parse()`. Example:
# main.py
from confix import register, parse
@register()
class config:
foo = 1
def parse_new_format():
return {}
parse('config.ext', file_parser=parse_new_format)