Multivariate Time Series

backends: Redis.

An application which implements a specialised remote stdnet.odm.Structure for managing numeric multivariate timeseries and perform remote analysis on them. The main classes for this application are ColumnTS, the stand alone data structure, and the correspondent ColumnTSField which can be used as a stdnet.odm.StructureField on a stdnet.odm.StdModel.

The API is straightforward:

from datetime import date
from stdnet.apps.columnts import ColumnTS

ts = ColumnTS(id='test')
ts.add(date(2012,2,21), {'open': 603.87, 'close': 614.00})

It can also be used as a datastructure fields. For example:

from stdnet import odm
from stdnet.apps.columnts import ColumnTSField

class Ticker(odm.StdModel):
    code = odm.SymbolField()
    data = ColumnTSField()

Statistical Analysis

istats & stats

These two methods execute statistical analysis on the data stored in one ColumnTS. The ColumnTS.istats method performs analysis by selecting time range by rank, while ColumnTS.stats method performs analysis by selecting ranges by a start and an end date (or datetime).

multi

evaluate

To perform analysis you write lua scripts:

self:range()

ts.evaluate(script)

API

ColumnTS

class stdnet.apps.columnts.ColumnTS(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)

A specialised stdnet.odm.TS structure for numeric multivariate timeseries.

front(*fields)

Return the front pair of the structure

back(*fields)

Return the back pair of the structure

info(start=None, end=None, fields=None)

Provide data information for this ColumnTS. If no parameters are specified it returns the number of data points for each fields, as well as the start and end date.

fields()

Tuple of ordered fields for this ColumnTS.

numfields()

Number of fields.

istats(start=0, end=-1, fields=None)

Perform a multivariate statistic calculation of this ColumnTS from start to end.

Parameters:

• start – Optional index (rank) where to start the analysis.
• end – Optional index (rank) where to end the analysis.
• fields – Optional subset of fields() to perform analysis on. If not provided all fields are included in the analysis.

stats(start, end, fields=None)

Perform a multivariate statistic calculation of this ColumnTS from a start date/datetime to an end date/datetime.

Parameters:

• start – Start date for analysis.
• end – End date for analysis.
• fields – Optional subset of fields() to perform analysis on. If not provided all fields are included in the analysis.

imulti_stats(start=0, end=-1, series=None, fields=None, stats=None)

Perform cross multivariate statistics calculation of this ColumnTS and other optional series from start to end.

Parameters:

• start – the start rank.
• start – the end rank
• field – name of field to perform multivariate statistics.
• series – a list of two elements tuple containing the id of the a columnTS and a field name.
• stats – list of statistics to evaluate. Default: [‘covariance’]

multi_stats(start, end, series=None, fields=None, stats=None)

Perform cross multivariate statistics calculation of this ColumnTS and other series.

Parameters:

• start – the start date.
• start – the end date
• field – name of field to perform multivariate statistics.
• series – a list of two elements tuple containing the id of the a columnTS and a field name.
• stats – list of statistics to evaluate. Default: [‘covariance’]

merge(*series, **kwargs)

Merge this ColumnTS with several other series.

Parameters series:  

a list of tuples where the nth element is a tuple of the form:

(wight_n, ts_n1, ts_n2, ..., ts_nMn)

The result will be calculated using the formula:

ts = weight_1*ts_11*ts_12*...*ts_1M1 + weight_2*ts_21*ts_22*...*ts_2M2 +
     ...

classmethod merged_series(*series, **kwargs)

Merge series and return the results without storing data in the backend server.

ColumnTSField

class stdnet.apps.columnts.ColumnTSField(model=None, pickler=None, value_pickler=None, class_field=False, **kwargs)

A multivariate timeseries field.

Redis Implementation

The implementation uses several redis structures for a given ColumnTS instance.

• A zset for holding times in an ordered fashion.
• A set for holding fields names, obtained via the ColumnTS.fields() method.
• A string for each field to hold numeric values.

This composite data-structure looks and feels like a redis zset. However, the ordered set doesn’t actually store the data, it is there to maintain order and facilitate retrieval by times (scores) and rank.

For a given field, the data is stored in a sequence of 9-bytes string with the initial byte (byte0) indicating the type of data:

<byte0><byte1,...,byte8>