This chapter discusses the mechanics of Traffic. In part, this document’s intended audience are people who are interested in working on Traffic. However, individuals troublshooting a dependent project may also find it useful in order to illuminate whether an observed behaviour should be expected or why Traffic behaves in the observed fashion.
Notably, many of these sections presume some familiarity with the traffic/libkqueue.py.c source.
The primary implementation targets kqueue, kernel-based I/O. This subchapter covers those parts.
In order to achieve high performance, Traffic performs system calls in batches while the GIL is released. This allows I/O to be performed concurrently with GIL processing. An ideal Traffic application will have at least two threads: one that runs the Traffic loop, and one that consumes a task queue filled by the Traffic loop. Given that objects cannot be created while not holding the GIL, receiving data is possible by receiving into mutable buffers.
A Transit is an object that performs a transfer of some sort. Usually, this is referring to data. However, in other cases, Traffic does cut some corners in order to allow shapes through the round hole. The base class is the traffic.libkqueue.Transit. It holds much of the boilerplate for supporting the Transit model.
The Transit data structure:
struct Transit {
PyObject_HEAD
Traffic traffic;
Transit prev, next;
kerror_t kerror;
uint8_t delta;
uint8_t junction;
Transit lltransfer;
int8_t event;
}
The traffic field is a reference to the traffic.libkqueue.Traffic instance that has acquired the given Transit. Traffic objects themselves are Transits and refer to themselves. When a Transit is created, this field remains empty until it has been given to an traffic.libkqueue.Traffic.acquire() method.
Every Transit pre-allocates the necessary memory for participating in Traffic.
The Transit.prev and Transit.next fields are used when a Transit is acquire by a Traffic instance with traffic.libkqueue.Traffic.acquire(). These fields are essentially used by the traffic.libkqueue.Traffic instance to keep track of all the acquired Transits. It is necessary in order to terminate a Traffic instance.
In cases of uncontrolled termination, usually due to errors emitted by the kernel, this field contains the system error that caused the termination of the Transit. This field should never contain temporary errors. On unix systems, the errors are usually listed in intro(2)
Note
Not implemented.
The source of the kerror field. There are a limited number of libc calls performed in order to manage a process’ Traffic. A list of identifiers is used to map to an associated call so that errnos can be associated with the call that noted the failure.
The delta field is a bitmap of Event Qualifications that are to be applied to the junction in the next traffic cycle. The delta field exists in order to allow state changes that may cause events to be noted while the GIL is held.
The junction field is where kernel and process Event Qualifications are stored until the event can be manifested. This is the final destination of where Transit.delta.
The lltransfer is the linked list field of Transits that have produced events. When this is not NULL, the Transit.event field is also non-zero.
The event field is a bit map of events that produced by this particular Transit in this particular cycle.
Event qualifications are bits in fields that denote that an event should occur. Event qualifications are ultimately destined for a place on the Transit.junction field. However, Transit.delta is the first stop for some Transit interactions.
Primarily, there are two qualifications:
- teq_terminate
- Transit Event Qualifier Terminate: this field indicates termination has been signalled. Termination can come from the process or the kernel.
- teq_transfer
- Transit Event Qualifier Transfer: this field indicates that a transfer
- teq_polarity
- Transit Event Qualifier Polarity: only used in Transit.junction to note whether the Transit sends or receives.
The qualifications are tracked for both the process Transit and the kernel noted capability of the represented object–kevent(2) produce an event. This means that for each Transit, a bitmap containing the state of the Transit with regards to the kernel and the process are recorded. When both the kernel and the process qualifications are met, a transfer event is generated or a terminate event is performed and generated.
List the events that can occur and describe them.
The end of a Transit’s life can be a tricky time. How termination occurs depends on a few factors. Many of those factors influence the process at different stages creating some confusing points within an implementation.
Transfers can occur when a given transit
With regards to receiving data, performing transfers as an effect of receiving their corresponding event poses a problem: What if the process can not handle the data at this point in time? How does the process control the flow? Traffic solves this problem with reception the same way that it does for sending: by forcing the user to allocate the memory necessary for transferrence–usually with bytearray and array.array. Given a mutable buffer, Traffic will receive into that buffer until it has “exhausted” the buffer. Once full, Traffic will emit an “exhaust” event associated with connection signalling the process that another buffer will need to be allocated in order to continue the reception of data.
Given that a Transit represents input or output, sockets require special handling by Traffic. That is, the input Transit can’t be fully released if the corresponding output Transit is still being used. Additionally, close(2) should not be called after termination as the relating Transit may still be using that file descriptor. It is only until both Transits referring to the file descriptor are terminated that a close(2) can actually be performed.
However, socket traffic.libkqueue.Octets instances will perform shutdown(2) upon termination. This draws a clear distinction between regular file descriptors which are terminated with close(2). The implications of this are most notable in cases where child processes are inheriting traffic.libkqueue.Octets.
Traffic takes steps to avoid memory management. By isolating that concern, the user is allowed to control memory in a way that suits the particular application.
However, the allocation of a Transit object provides a Traffic instance with all the necessary memory for the Transit to participate In-Traffic. By taking care here, Traffic reduces the number of possible errors and the locations that they can occur.
Future versions are envisioned to include a netmap implementation: http://info.iet.unipi.it/~luigi/netmap/