Project

Event driven I/O where the light is always green.

Introduction

Traffic provides access to high-level event driven I/O in Python.

Managing process I/O in an event driven manner has a number of benefits. There are many Python packages and modules which help perform this task. Most of these interfaces, saving larger frameworks, focus on notifying the process that data can be read or that data can be written. Subsequently, relying on other interfaces to perform the actual transfer.

Traffic transforms can-transfer events into did-transfer events.

In addition to managing the subscription to I/O events, Traffic takes the next step and performs the potential operation–usually a read(2) or write(2) call, so that no further operation is necessary aside from the processing of the transfer.

Structure

Traffic is structured as a single Python package. Everything pertinent to Traffic’s operation is contained within the traffic package.

traffic.libkqueue

kqueue(2) implementation of the Traffic Interface.

traffic.libloop

Functions and classes for managing a simple event loop.

traffic.project

Project description; name, version, release date.

traffic.release

Deals with distribution and contains project release management code.

traffic.documentation

Houses the project’s documentation.

Requirements

Traffic needs a working kqueue(2) interface in order to operate. Most BSD-based operating systems support this interface. Using Traffic under Linux is possible using libkqueue.

Traffic requires a Python implementation that supports the Python C-API–either Two or Three–that supports writable buffers.

Conventions

Transits are normally named after the type of object that can be transferred by the transit. Notably, “Octets” transfers bytes and “Sockets” transfers the file descriptors of accepted socket connections.

Terminology:

Transit

An object that transfers. While this is notably ambiguous, it is purposefully so. A Transit’s functionality is rather open ended. The transferred subject is an arbitrary object and the destination is arbitrary as well. Usually, however, it’s either octets or sockets–integer file descriptors from accept.

Cycle

Usually referring to a traffic.libkqueue.Traffic cycle. Cycles are are used to provide a precise window of access to the set of events that occurred. After a cycle is closed, the event state on the Transits is reset in order to make room for the next cycle.

Kernel Point

Generic term for file descriptor. This term is used to reference file descriptors in the context of the abstraction.

Input Transit

A Transit that transfers into the process. Usually regarding an Octets or Sockets transit.

Output Transit

A Transit that transfers the resource out of the process.

Polarity

An abuse of the word, but it is used internally to identify the bit field that designates whether the Transit sends or receives. “On” indicates gain, receiving, and “Off” indicates loss, sending. Polarity is noted for both internal and external, but only the internal field is exposed.

In Traffic

Preposition regarding whether or not a Transit has been acquired by a Traffic instance. Usually used in traffic/lib.py.c to note that a given segment of code can or can not be ran if the Transit is In Traffic.

Exhaustion

A Transit is exhausted after an Exhaust event occurs until another resource has been acquired by the Transit. These events are emitted when the entire resource has been Transferred and there is nothing more to send or no more space available in order to receive. Exhaustion can be used control the flow of a stream. Starving a Transit of resources keeps it from performing transfers.

Termination

The process of ending a Transit’s ability to transfer. A Transit is Terminated when transfers are no longer possible.

Disregard

Has the effect of termination without the represented channel being terminated. A Transit “disregards” the Transit resources. For file descriptor based transits, this means a call to close(2) followed by Termination. Notably, for sockets, this allows the connection to survive Termination–stream sockets will be shutdown(2) upon Termination.

Transfer-Loss

The loss of events and, potentially, data produced by a transit. In cases where exceptions occur during the processing of events produced by a Traffic Cycle, the failure, potentially, causes Transfer-Loss. More appropriately, event loss, but transfers tend to be critical, so Transfer-Loss is used to highlight the worst case.

Transfer Resource

An object used by a Transit in order to perform a transfer. Usually, a Python buffer object. In the case Input Transits, a mutable buffer.

Transfer Event

The event emitted by the Transit when some or all of the Transfer Resource has been transferred.

Defense

Traffic is not a framework. It strives to provide a minimal feature set covering the common needs of event driven programming. It distinguishes itself by conflating I/O events with the product of their corresponding I/O operations in a way that is tailored specifically for Python.

kqueue(2) is used as the underlying event drive. In systems where kqueue(2) is not implemented by the kernel, libkqueue can often be used to compensate. libkqueue has support for Microsoft Windows platforms and Linux based operating systems. See http://mark.heily.com/project/libkqueue

Future

traffic.libkqueue.Descriptors.channel

Some operating systems support communication of file descriptors across processes. Implement a Descriptors class in order to allow communication of file descriptors across processes.

traffic.libnetmap

Maybe. Netmap Implementation, http://info.iet.unipi.it/~luigi/netmap/

Mentions

Descriptors

libancillary, by Nicolas George, was used as a map to implementing the Descriptors type for traffic.libkqueue. Without his library, it would have been a much bigger pain to figure it all out.

http://gitorious.org/libancillary http://stackoverflow.com/questions/4489433/sending-file-descriptor-over-unix-domain-socket-and-select