bridgedb.safelog

Filters for log sanitisation.

The Safelog*Filter classes within this module can be instantiated and adding to any logging.Handler, in order to transparently filter substrings within log messages which match the given pattern. Matching substrings may be optionally additionally validated by implementing the doubleCheck() method before they are finally replaced with the replacement string. For example:

>>> import io
>>> import logging
>>> from bridgedb import safelog
>>> handler = logging.StreamHandler(io.BytesIO())
>>> logger = logging.getLogger()
>>> logger.addHandler(handler)
>>> logger.addFilter(safelog.SafelogEmailFilter())
>>> logger.info("Sent response email to: blackhole@torproject.org")

Module Overview:

bridgedb.safelog
 |
 |_ setSafeLogging - Enable or disable safelogging globally.
 |_ logSafely - Utility for manually sanitising a portion of a log message
 |
 \_ BaseSafelogFilter - Base class for log message sanitisation filters
    |   |_ doubleCheck - Optional stricter validation on matching substrings
    |   \_ filter - Determine if some part of a log message should be filtered
    |
    |_ SafelogEmailFilter - Filter for removing email addresses from logs
    |_ SafelogIPv4Filter - Filter for removing IPv4 addresses from logs
    |_ SafelogIPv6Filter - Filter for removing IPv6 addresses from logs

setSafeLogging(safe)

Enable or disable automatic filtering of log messages.

Parameters:safe (bool) – If True, filter email and IP addresses from log messages automagically.

logSafely(string)

Utility for manually sanitising a portion of a log message.

Parameters:string (str) – If SAFELOGGING is enabled, sanitise this string by replacing it with "[scrubbed]". Otherwise, return the string unchanged. Return type:str Returns:"[scrubbed]" or the original string.

class BaseSafelogFilter(name='')

Bases: logging.Filter

Base class for creating log message sanitisation filters.

A BaseSafelogFilter uses a compiled regex pattern to match particular items of data in log messages which should be sanitised (if SAFELOGGING is enabled in bridgedb.conf).

Note

The pattern is used only for string matching purposes, and not for validation. In other words, a pattern which matches email addresses should simply match something which appears to be an email address, even though that matching string might not technically be a valid email address vis-á-vis RFC 5321.

In addition, a BaseSafelogFilter uses a easyFind, which is simply a string or character to search for before running checking against the regular expression, to attempt to avoid regexing everything which passes through the logger.

Variables:

• pattern – A compiled regular expression, whose matches will be scrubbed from log messages and replaced with replacement.
• easyFind (str) – A simpler string to search for before to match by regex.
• replacement (str) – The string to replace pattern matches with. (default: "[scrubbed]")

Initialize a filter.

Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.

pattern = <_sre.SRE_Pattern object>

easyFind = 'FILTERME'

replacement = '[scrubbed]'

doubleCheck(match)

Subclasses should override this function to implement any additional substring filtering to decrease the false positive rate, i.e. any additional filtering or validation which is more costly than checking against the regular expression, pattern.

To use only the pattern matching in filter(), and not use this method, simply do:

return True

Parameters:match (str) – Some portion of the :ivar:`logging.LogRecord.msg` string which has already passed the checks in filter(), for which additional validation/checking is required. Return type:bool Returns:True if the additional validation passes (in other words, the match should be filtered), and None or False otherwise.

filter(record)

Filter a log record.

The log record is filtered, and thus sanitised by replacing matching substrings with the replacement string, if the following checks pass:

1. SAFELOGGING is currently enabled.
2. The record.msg string contains easyFind.
3. The record.msg matches the regular expression, pattern.

Parameters:record (logging.LogRecord) – Basically, anything passed to logging.log().

class SafelogEmailFilter(name='')

Bases: bridgedb.safelog.BaseSafelogFilter

A log filter which removes email addresses from log messages.

Initialize a filter.

Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.

pattern = <_sre.SRE_Pattern object>

easyFind = '@'

filter(record)

Filter a log record.

The log record is filtered, and thus sanitised by replacing matching substrings with the replacement string, if the following checks pass:

1. SAFELOGGING is currently enabled.
2. The record.msg string contains easyFind.
3. The record.msg matches the regular expression, pattern.

Parameters:record (logging.LogRecord) – Basically, anything passed to logging.log().

class SafelogIPv4Filter(name='')

Bases: bridgedb.safelog.BaseSafelogFilter

A log filter which removes IPv4 addresses from log messages.

Initialize a filter.

Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.

pattern = <_sre.SRE_Pattern object>

easyFind = '.'

doubleCheck(match)

Additional check to ensure that match is an IPv4 address.

filter(record)

Filter a log record.

The log record is filtered, and thus sanitised by replacing matching substrings with the replacement string, if the following checks pass:

1. SAFELOGGING is currently enabled.
2. The record.msg string contains easyFind.
3. The record.msg matches the regular expression, pattern.

Parameters:record (logging.LogRecord) – Basically, anything passed to logging.log().

class SafelogIPv6Filter(name='')

Bases: bridgedb.safelog.BaseSafelogFilter

A log filter which removes IPv6 addresses from log messages.

Initialize a filter.

Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.

pattern = <_sre.SRE_Pattern object>

easyFind = ':'

doubleCheck(match)

Additional check to ensure that match is an IPv6 address.

filter(record)

Filter a log record.

The log record is filtered, and thus sanitised by replacing matching substrings with the replacement string, if the following checks pass:

1. SAFELOGGING is currently enabled.
2. The record.msg string contains easyFind.
3. The record.msg matches the regular expression, pattern.

Parameters:record (logging.LogRecord) – Basically, anything passed to logging.log().