Pyrant

A pure-Python implementation of Tokyo Tyrant protocol. Python 2.4+ is required.

More information about Tokyo Cabinet:
http://1978th.net/tokyocabinet/
More information about Tokyo Tyrant:
http://1978th.net/tokyotyrant/

Usage example (note the automatically managed support for table database):

>>> import pyrant
>>> TEST_HOST, TEST_PORT = '127.0.0.1', 1983
>>> t = pyrant.Tyrant(host=TEST_HOST, port=TEST_PORT)    # default port is 1978
>>> if t.table_enabled:
...     t['key'] = {'name': 'foo'}
...     print t['key']['name']
... else:
...     t['key'] = 'foo'
...     print t['key']
foo
>>> del t['key']
>>> print t['key']
Traceback (most recent call last):
    ...
KeyError: 'key'
class pyrant.Tyrant(host='127.0.0.1', port=1978, separator=None, literal=False)

A Python dictionary API for Tokyo Tyrant.

Parameters:
  • host – Tyrant host address
  • port – Tyrant port number
  • separator – if set, will be used to get/put lists as values. For table databases the separator applies to column values.
  • literal – if set, returned data is not encoded to Unicode (default is False)

Usage:

>>> import pyrant
>>> t = pyrant.Tyrant(host=TEST_HOST, port=TEST_PORT)

# remove anything that could be left from previous time
>>> t.clear()

# make sure there are zero records in the database
>>> len(t)
0

Tyrant provides pythonic syntax and data pythonification, while the the lower-lever TyrantProtocol closely follows the orginal Tokyo Tyrant API and only converts incoming data to strings (Unicode). You decide which to use. Generally you would want more pythonic API (Tyrant) for most cases and the lower-level interface to reduce overhead or to fix broken data which cannot be properly converted by means of the higher-level API.

It is also important that Tokyo Cabinet has a great query extension for table databases. This extension is supported by pyrant.query.Query which only requires TyrantProtocol to work and can be easily accessed via Tyrant.query:

>>> t.query
[]
call_func(func, key, value, record_locking=False, global_locking=False)
Calls specific function.
clear()
Removes all records from the remote database.
concat(key, value, width=None)
Concatenates columns of the existing record.
generate_key()
Returns a unique primary key for given database. Tries to obtain the key using database’s built-in function genuid. If genuid fails to provide the key, a UUID is generated instead.
get(key, default=None)
Returns value for key. If no record is found, returns default.
get_size(key)
Returns the size of the value for key.
get_stats()
Returns the status message of the database as dictionary.
iteritems()
Returns a generator with key/value pairs. The data is read from the database in chunks to alleviate the issues of a) too many database hits, and b) too heavy memory usage when only a part of the list is actually used.
iterkeys()
Iterates keys using remote operations.
keys()
Returns the list of keys in the database.
multi_add(values, chunk_size=1000, no_update_log=False)

Adds given values as new records and returns a list of automatically generated primary keys for these records. Wrapper for multi_set().

Parameters:
  • values – any iterable; in fact, you can pass a generator and it will be processed in chunks in order to save on resources and cut down the database hit at the same time.
  • chunk_size – size of chunks in which the data will be fed to the database. If set to zero, data is fed to database in one chunk.
multi_del(keys, no_update_log=False)
Removes given records from the database.
multi_get(keys)

Returns records that match given keys. Missing keys are silently ignored, i.e. the number of results may be lower than the number of requested keys. The records are returned as key/value pairs. Usage:

>>> g = t.multi_get(['foo', 'bar', 'galakteko opasnoste'])
>>> g
[('foo', {'one': 'one'}), ('bar', {'two': 'two'})]
Parameters:
  • keys – the list of keys.
multi_set(items, no_update_log=False)

Stores the given records in the database. The records may be given as an iterable sequence. Usage:

>>> t.multi_set([('foo', {'one': 'one'}), ('bar', {'two': 'two'})])

which is equevalent with the call:

>>> t.multi_set({'foo': {'one': 'one'}, 'bar':{'two': 'two'}})
Parameters:
  • items – the sequence of records to be stored.
prefix_keys(prefix, maxkeys=None)
Get forward matching keys in a database. The return value is a list object of the corresponding keys.
query

Returns a Query object for the database.

Note

Only available for Table Databases.

setdefault(key, value)
>>> t.setdefault('foo', {'one': 'one'})
{u'one': u'one'}
>>> t.setdefault('foo', {'two': 'two'})
{u'one': u'one'}
sync()

Synchronizes updated content with the database.

Tokyo Cabinet is not durable, as data committed to the database is not immediately flushed to disk. Unwritten data may lost in the event of e.g. a system crash.

Use sync to force a flush of unwritten data to disk, but be aware that this also locks the writer process and blocks queries.

The better approach is to use database replication (copy the data to another database instance) and backup often.

table_enabled
Returns True is current database type is TDB so TDB-specific extensions are enabled.
update(mapping=None, **kwargs)

Updates given objects from a dict, list of key and value pairs or a list of named params. Usage:

mapping = (
    ('john', dict(name='John')),
)
t.update(mapping, mary={'name': 'Mary'})

See built-in dict.update method for more information.

Previous topic

Welcome to pyrant’s documentation!

Next topic

Protocol

This Page