.. _notifyman: *********************** Notification Management *********************** Relevant SQL commands: `NOTIFY `_, `LISTEN `_, `UNLISTEN `_. Asynchronous notifications offer a means for PostgreSQL to signal application code. Often these notifications are used to signal cache invalidation. In 9.0 and greater, notifications may include a "payload" in which arbitrary data may be delivered on a channel being listened to. By default, received notifications will merely be appended to an internal list on the connection object. This list will remain empty for the duration of a connection *unless* the connection begins listening to a channel that receives notifications. The `postgresql.notifyman.NotificationManager` class is used to wait for messages to come in on a set of connections, pick up the messages, and deliver the messages to the object's user via the `collections.Iterator` protocol. Listening on a Single Connection ================================ The ``db.iternotifies()`` method is a simplification of the notification manager. It returns an iterator to the notifications received on the subject connection. The iterator yields triples consisting of the ``channel`` being notified, the ``payload`` sent with the notification, and the ``pid`` of the backend that caused the notification:: >>> db.listen('for_rabbits') >>> db.notify('for_rabbits') >>> for x in db.iternotifies(): ... channel, payload, pid = x ... break >>> assert channel == 'for_rabbits' True >>> assert payload == '' True >>> assert pid == db.backend_id True The iterator, by default, will continue listening forever unless the connection is terminated--thus the immediate ``break`` statement in the above loop. In cases where some additional activity is necessary, a timeout parameter may be given to the ``iternotifies`` method in order to allow "idle" events to occur at the designated frequency:: >>> for x in db.iternotifies(0.5): ... if x is None: ... break The above example illustrates that idle events are represented using `None` objects. Idle events are guaranteed to occur *approximately* at the specified interval--the ``timeout`` keyword parameter. In addition to providing a means to do other processing or polling, they also offer a safe break point for the loop. Internally, the iterator produced by the ``iternotifies`` method *is* a `NotificationManager`, which will localize the notifications prior to emitting them via the iterator. *It's not safe to break out of the loop, unless an idle event is being handled.* If the loop is broken while a regular event is being processed, some events may remain in the iterator. In order to consume those events, the iterator *must* be accessible. The iterator will be exhausted when the connection is closed, but if the connection is closed during the loop, any remaining notifications *will* be emitted prior to the loop ending, so it is important to be prepared to handle exceptions or check for a closed connection. In situations where multiple connections need to be watched, direct use of the `NotificationManager` is necessary. Listening on Multiple Connections ================================= The `postgresql.notifyman.NotificationManager` class is used to manage *connections* that are expecting to receive notifications. Instances are iterators that yield the connection object and notifications received on the connection or `None` in the case of an idle event. The manager emits events as a pair; the connection object that received notifications, and *all* the notifications picked up on that connection:: >>> from postgresql.notifyman import NotificationManager >>> # Using ``nm`` to reference the manager from here on. >>> nm = NotificationManager(db1, db2, ..., dbN) >>> nm.settimeout(2) >>> for x in nm: ... if x is None: ... # idle ... break ... ... db, notifies = x ... for channel, payload, pid in notifies: ... ... The manager will continue to wait for and emit events so long as there are good connections available in the set; it is possible for connections to be added and removed at any time. Although, in rare circumstances, discarded connections may still have pending events if it not removed during an idle event. The ``connections`` attribute on `NotificationManager` objects is a set object that may be used directly in order to add and remove connections from the manager:: >>> y = [] >>> for x in nm: ... if x is None: ... if y: ... nm.connections.add(y[0]) ... del y[0] ... The notification manager is resilient; if a connection dies, it will discard the connection from the set, and add it to the set of bad connections, the ``garbage`` attribute. In these cases, the idle event *should* be leveraged to check for these failures if that's a concern. It is the user's responsibility to explicitly handle the failure cases, and remove the bad connections from the ``garbage`` set:: >>> for x in nm: ... if x is None: ... if nm.garbage: ... recovered = take_out_trash(nm.garbage) ... nm.connections.update(recovered) ... nm.garbage.clear() ... db, notifies = x ... for channel, payload, pid in notifies: ... ... Explicitly removing connections from the set can also be a means to gracefully terminate the event loop:: >>> for x in nm: ... if x in None: ... if done_listening is True: ... nm.connections.clear() However, doing so inside the loop is not a requirement; it is safe to remove a connection from the set at any point. Notification Managers ===================== The `postgresql.notifyman.NotificationManager` is an event loop that services multiple connections. In cases where only one connection needs to be serviced, the `postgresql.api.Database.iternotifies` method can be used to simplify the process. Notification Manager Constructors --------------------------------- ``NotificationManager(*connections, timeout = None)`` Create a NotificationManager instance that manages the notifications coming from the given set of connections. The ``timeout`` keyword is optional and can be configured using the ``settimeout`` method as well. Notification Manager Interface Points ------------------------------------- ``NotificationManager.__iter__()`` Returns the instance; it is an iterator. ``NotificationManager.__next__()`` Normally, yield the pair, connection and notifications list, when the next event is received. If a timeout is configured, `None` may be yielded to signal an idle event. The notifications list is a list of triples: ``(channel, payload, pid)``. ``NotificationManager.settimeout(timeout : int)`` Set the amount of time to wait before the manager yields an idle event. If zero, the manager will never wait and only yield notifications that are immediately available. If `None`, the manager will never emit idle events. ``NotificationManager.gettimeout() -> [int, None]`` Get the configured timeout; returns either `None`, or an `int`. ``NotificationManager.connections`` The set of connections that the manager is actively watching for notifications. Connections may be added or removed from the set at any time. ``NotificationManager.garbage`` The set of connections that failed. Normally empty, but when a connection gets an exceptional condition or explicitly raises an exception, it is removed from the ``connections`` set, and placed in ``garbage``. Zero Timeout ------------ When a timeout of zero, ``0``, is configured, the notification manager will terminate early. Specifically, each connection will be polled for any pending notifications, and once all of the collected notifications have been emitted by the iterator, `StopIteration` will be raised. Notably, *no* idle events will occur when the timeout is configured to zero. Zero timeouts offer a means for the notification "queue" to be polled. Often, this is the appropriate way to collect pending notifications on active connections where using the connection exclusively for waiting is not practical:: >>> notifies = list(db.iternotifies(0)) Or with a NotificationManager instance:: >>> nm.settimeout(0) >>> db_notifies = list(nm) In both cases of zero timeout, the iterator may be promptly discarded without losing any events. Summary of Characteristics -------------------------- * The iterator will continue until the connections die. * Objects yielded by the iterator are either `None`, an "idle event", or an individual notification triple if using ``db.iternotifies()``, or a ``(db, notifies)`` pair if using the base `NotificationManager`. * When a connection dies or raises an exception, it will be removed from the ``nm.connections`` set and added to the ``nm.garbage`` set. * The NotificationManager instance will *not* hold any notifications during an idle event. Idle events offer a break point in which the manager may be discarded. * A timeout of zero will cause the iterator to only yield the events that are pending right now, and promptly end. However, the same manager object may be used again. * A notification triple is a tuple consisting of ``(channel, payload, pid)``. * Connections may be added and removed from the ``nm.connections`` set at any time.