Data Loader

A data loader object coordinates which data provider to use and provides hooks for pre-processing, post-processing and storing data on your database if required.

The default data loader is dynts.data.TimeSerieLoader. To select a different one, simply pass it to the dynts.evaluate() functions:

>>> import dynts
>>> import mydata.loader import customdataloader
>>> ts = dynts.evaluate('YHOO,AMZN', loader = customdataloader)

Alternatively, one can specify a new loader in the settings:

>>> from dynts.conf import settings
>>> settings.default_loader = customdataloader

The data loader class

class dynts.data.TimeSerieLoader

Cordinates the loading of timeseries data into dynts.dsl.Symbol. This class can be overritten by a custom one if required. There are four different hooks which can be used to customised its behaviour:

• dynts.data.TimeSerieLoader.parse_symbol()
• dynts.data.TimeSerieLoader.preprocess()
• dynts.data.TimeSerieLoader.onresult()
• dynts.data.TimeSerieLoader.onfinishload()

preprocessdata

Class holding data returned by the dynts.data.TimeSerieLoader.preprocess() method. It contains two attributes:

• intervals None or a tuple of two-elements tuples.
• result None or any intermediate result.

If intervals is None or an empty container, the dynts.data.DataProvider.load() method won’t be called, and the result attribute will be passed to the dynts.data.TimeSerieLoader.onresult() method.

symboldata

alias of SymbolData

load(providers, symbols, start, end, logger, backend, **kwargs)

Load symbols data.

Parameters:

• providers – Dictionary of registered data providers.
• symbols – list of symbols to load.
• start – start date.
• end – end date.
• logger – instance of logging.Logger or None.
• backend – dynts.TimeSeries backend name.

There is no need to override this function, just use one the three hooks available.

dates(start, end)

Internal function which perform pre-conditioning on dates:

Parameters:

• start – start date.
• end – end date.

This function makes sure the start and end date are consistent. It never fails and always return a two-element tuple containing start, end with start less or equal end and end never after today. There should be no reason to override this function.

parse_symbol(symbol, providers)

Parse a symbol to obtain information regarding ticker, field and provider. Must return an instance of symboldata.

Parameters:

• symbol – string associated with market data to load.
• providers – dictionary of dynts.data.DataProvider instances available.

For example:

intc
intc:open
intc:volume:google
intc:google

are all valid inputs returning a SymbolData instance with the following triplet of information:

intc,None,yahoo
intc,open,yahoo
intc,volume,google
intc,None,google

assuming yahoo is the provider in dynts.conf.Settings.default_provider.

This function is called before retrieving data.

default_provider_for_ticker(ticker, field)

Calculate the provider when not available in the symbol. By default it returns dynts.conf.Settings.default_provider.

getsymbol(ticker, field, provider)

Convert ticker, field and provider to symbol code. The inverse of dynts.data.TimeSerieLoader.parse_symbol().

preprocess(ticker, start, end, logger, backend, **kwargs)

Preprocess hook. This is first loading hook and it is called before requesting data from a dataprovider. It must return an instance of TimeSerieLoader.preprocessdata. By default it returns:

self.preprocessdata(intervals = ((start,end),))

It could be overritten to modify the intervals. If the intervals is None or an empty container, the dynts.data.DataProvider.load() method won’t be called, otherwise it will be called as many times as the number of intervals in the return tuple (by default once).

onresult(ticker, result, logger, backend, **kwargs)

Post-processing hook for results returned by calls to dynts.data.DataProvider.load(), or obtained from the dynts.data.TimeSerieLoader.preprocess() method. By default return result:

return result

It could be used to store data into a cache or database.

onfinishload(data, logger, backend, **kwargs)

Another post-processing hook invoked when the loading is finished. By default retun data.