Tokyo Cabinet extension

A storage/query backend for Tokyo Cabinet.

Allows direct access to the database and is thus extremely fast. However, it locks the database and is therefore not suitable for environments where concurrent access is required. Please use Tokyo Tyrant for such environments.

status:beta database:Tokyo Cabinet dependencies:tokyo-python, pyrant suitable for:general purpose, embedded

Warning

this module is not intended for production despite it may be stable. Bug reports and patches are welcome.

Note

this module should not depend on Pyrant; just needs some refactoring.

Note

support for metasearch is planned.

Usage:

>>> import os
>>> import doqu
>>> DB_SETTINGS = {
...     'backend': 'doqu.ext.tokyo_cabinet',
...     'path': '_tc_test.tct',
... }
>>> assert not os.path.exists(DB_SETTINGS['path']), 'test database must not exist'
>>> db = doqu.get_db(DB_SETTINGS)
>>> class Person(doqu.Document):
...     structure = {'name': unicode}
...     def __unicode__(self):
...         u'%(name)s' % self
...
>>> Person.objects(db)    # the database is expected to be empty
[]
>>> db.connection['john'] = {'name': 'John'}
>>> mary = Person(name='Mary')
>>> mary_pk = mary.save(db)
>>> q = Person.objects(db)
>>> q
[<Person John>, <Person Mary>]
>>> q.where(name__matches='^J')
[<Person John>]
>>> q    # the original query was not modified by the descendant
[<Person John>, <Person Mary>]
>>> db.connection.close()
>>> os.unlink(DB_SETTINGS['path'])

class doqu.ext.tokyo_cabinet.StorageAdapter(**kw)

Parameters:

• path – relative or absolute path to the database file (e.g. test.tct)

Note

Currently only table flavour of Tokyo Cabinet databases is supported. It is uncertain whether it is worth supporting other flavours as they do not provide query mechanisms other than access by primary key.

clear()

Clears the whole storage from data, resets autoincrement counters.

connect()

Connects to the database. Raises RuntimeError if the connection is not closed yet. Use StorageAdapter.reconnect() to explicitly close the connection and open it again.

delete(primary_key)

Permanently deletes the record with given primary key from the database.

disconnect()

Closes internal store and removes the reference to it.

get(doc_class, primary_key)

Returns document object for given document class and primary key.

get_many(doc_class, primary_keys)

Returns a list of documents with primary keys from given list. Basically this is just a simple wrapper around get() but some backends can reimplement the method in a much more efficient way.

get_or_create(doc_class, **kwargs)

Queries the database for records associated with given document class and conforming to given extra condtions. If such records exist, picks the first one (the order may be random depending on the database). If there are no such records, creates one.

Returns the document instance and a boolean value “created”.

reconnect()

Gracefully closes current connection (if it’s not broken) and connects again to the database (e.g. reopens the file).

save(data, primary_key=None)

Saves given model instance into the storage. Returns primary key.

Parameters:

• data – dict containing all properties to be saved
• primary_key – the key for given object; if undefined, will be generated

Note that you must provide current primary key for a document object which is already in the database in order to update it instead of copying it.

class doqu.ext.tokyo_cabinet.QueryAdapter(*args, **kw)

The Query class.

count()

Same as __len__ but without fetching the records (i.e. faster).

delete()

Deletes all records that match current query.

order_by(names, reverse=False)

Defines order in which results should be retrieved.

Parameters:

• names – the field name by which the results should be sorted. Must be a string. The name is a bit misleading but the base backend API prescribes it.

Examples:

q.order_by('name')                  # ascending
q.order_by('name', reverse=True)    # descending

Warning

The basic Doqu backend API allows multiple sorting fields. However, Tokyo Cabinet does not more than one. Specifying multiple names will raise TypeError.

values(name)

Returns an iterator that yields distinct values for given column name.

Note

this is currently highly inefficient because the underlying library does not support columns mode (tctdbiternext3). Moreover, even current implementation can be optimized by removing the overhead of creating full-blown document objects (though preserving data type is necessary).

where(**conditions)

Returns Query instance filtered by given conditions. The conditions are defined exactly as in Pyrant’s high-level query API. See pyrant.query.Query.filter documentation for details.

where_not(**conditions)

Returns Query instance. Inverted version of where().