Configuration File Syntax¶
The rejected configuration uses YAML as the markup language. YAML’s format, like Python code is whitespace dependent for control structure in blocks. If you’re having problems with your rejected configuration, the first thing you should do is ensure that the YAML syntax is correct. yamllint.com is a good resource for validating that your configuration file can be parsed.
The configuration file is split into three main sections: Application, Daemon, and Logging.
The example configuration file provides a good starting point for creating your own configuration file.
Application¶
The application section of the configuration is broken down into multiple top-level options:
| poll_interval | How often rejected should poll consumer processes for status in seconds (int/float) |
| sentry_dsn | If Sentry support is installed, optionally set a global DSN for all consumers (str) |
| stats | Enable and configure statsd metric submission (obj) |
| Connections | A subsection with RabbitMQ connection information for consumers (obj) |
| Consumers | Where each consumer type is configured (obj) |
stats¶
| stats | ||
|---|---|---|
| log | Toggle top-level logging of consumer process stats (bool) | |
| influxdb | Configure the submission of per-message measurements to InfluxDB (obj) | |
| statsd | Configure the submission of per-message measurements to statsd (obj) | |
influxdb¶
| stats > influxdb | ||
|---|---|---|
| scheme | The scheme to use when submitting metrics to the InfluxDB server. Default: http (str) |
|
| host | The hostname or ip address of the InfluxDB server. Default: localhost (str) |
|
| port | The port of the influxdb server. Default: 8086 (int) |
|
| user | An optional username to use when submitting measurements. (str) | |
| password | An optional password to use when submitting measurements. (str) | |
| database | The InfluxDB database to submit measurements to. Default: rejected (str) |
|
statsd¶
| stats > statsd | ||
|---|---|---|
| enabled | Toggle statsd reporting off and on (bool) | |
| prefix | An optional prefix to use when creating the statsd metric path (str) | |
| host | The hostname or ip address of the statsd server (str) | |
| port | The port of the statsd server. Default: 8125 (int) |
|
Connections¶
Each RabbitMQ connection entry should be a nested object with a unique name with connection attributes.
| Connection Name | ||
|---|---|---|
| host | The hostname or ip address of the RabbitMQ server (str) | |
| port | The port of the RabbitMQ server (int) | |
| vhost | The virtual host to connect to (str) | |
| user | The username to connect as (str) | |
| pass | The password to use (str) | |
| heartbeat_interval | Optional: the AMQP heartbeat interval (int) default: 300 sec | |
Consumers¶
Each consumer entry should be a nested object with a unique name with consumer attributes.
| Consumer Name | ||
|---|---|---|
| consumer | The package.module.Class path to the consumer code (str) | |
| connections | The connections to connect to (list) - See Consumer Connections | |
| qty | The number of consumers per connection to run (int) | |
| queue | The RabbitMQ queue name to consume from (int) | |
| ack | Explicitly acknowledge messages (no_ack = not ack) (bool) | |
| max_errors | Number of errors encountered before restarting a consumer (int) | |
| sentry_dsn | If Sentry support is installed, set a consumer specific sentry DSN (str) | |
| drop_exchange | The exchange to publish a message to when it is dropped. If not specified, dropped messages are not republished anywhere. | |
| drop_invalid_messages | Drop a message if the type property doesn’t match the specified message type (str) | |
| message_type | Used to validate the message type of a message before processing. This attribute can be set to a string that is matched against the AMQP message type or a list of acceptable message types. (str, array) | |
| error_exchange | The exchange to publish messages that raise
ProcessingException to (str) |
|
| error_max_retry | The number of ProcessingException raised on a message
before a message is dropped. If not specified messages will never be dropped (int) |
|
| influxdb_measurement | When using InfluxDB, the measurement name for per-message measurements. Defaults to the consumer name. (str) | |
| config | Free-form key-value configuration section for the consumer (obj) | |
Consumer Connections¶
The consumer connections configuration allows for one or more connections to be made by a single consumer. This configuration section supports two formats. If a list of connection names are specified, the consumer will connect to and consume from the all of the specified connections.
Consumer Name:
connections:
- connection1
- connection2
If the connections list include structured values, additional settings can be
set. For example, you may want to consume from one RabbitMQ broker and publish to
another, as is illustrated below:
Consumer Name:
connections:
- name: connection1
consume: True
publisher_confirmation: False
- name: connection2
consume: False
publisher_confirmation: True
In the above example, the consumer will have two connections, connection1 and
connection2. It will only consume from connection1 but can publish
messages connection2 by specifying the connection name in the
publish_message() method.
Structured Connections¶
When specifying a structured consumer connection, the following attributes are available.
| Consumer Name > connections | ||
|---|---|---|
| name | The connection name, as specified in the Connections section of the application configuration. | |
| consume | Specify if the connection should consume on the connection. (bool) | |
| publisher_confirmation | Enable publisher confirmations. (bool) | |
Daemon¶
This section contains the settings required to run the application as a daemon. They are as follows:
| user | The username to run as when the process is daemonized (bool) |
| group | Optional The group name to switch to when the process is daemonized (str) |
| pidfile | The pidfile to write when the process is daemonized (str) |
Logging¶
rejected uses logging.config.dictConfig to create a flexible method for configuring the python standard logging module. If rejected is being run in Python 2.6, logutils.dictconfig.dictConfig is used instead.
The following basic example illustrates all of the required sections in the dictConfig format, implemented in YAML:
version: 1
formatters: []
verbose:
format: '%(levelname) -10s %(asctime)s %(process)-6d %(processName) -15s %(name) -10s %(funcName) -20s: %(message)s'
datefmt: '%Y-%m-%d %H:%M:%S'
handlers:
console:
class: logging.StreamHandler
formatter: verbose
debug_only: True
loggers:
rejected:
handlers: [console]
level: INFO
propagate: true
myconsumer:
handlers: [console]
level: DEBUG
propagate: true
disable_existing_loggers: true
incremental: false
Note
The debug_only node of the Logging > handlers > console section is not part of the standard dictConfig format. Please see the Logging Caveats section below for more information.
Logging Caveats¶
In order to allow for customizable console output when running in the foreground and no console output when daemonized, a debug_only node has been added to the standard dictConfig format in the handler section. This method is evaluated when logging is configured and if present, it is removed prior to passing the dictionary to dictConfig if present.
If the value is set to true and the application is not running in the foreground, the configuration for the handler and references to it will be removed from the configuration dictionary.
Troubleshooting¶
If you find that your application is not logging anything or sending output to the terminal, ensure that you have created a logger section in your configuration for your consumer package. For example if your Consumer instance is named myconsumer.MyConsumer make sure there is a myconsumer logger in the logging configuration.