Backend API

Abstract classes for unified storage/query API with various backends.

Derivative classes are expected to be either complete implementations or wrappers for external libraries. The latter is assumed to be a better solution as Doqu is only one of the possible layers. It is always a good idea to provide different levels of abstraction and let others combine them as needed.

The backends do not have to subclass BaseStorageAdapter and BaseQueryAdapter. However, they must closely follow their API.

class doqu.backend_base.BaseStorageAdapter(**kw)

Abstract adapter class for storage backends.

clear()

Clears the whole storage from data, resets autoincrement counters.

connect()

Connects to the database. Raises RuntimeError if the connection is not closed yet. Use reconnect() to explicitly close the connection and open it again.

delete(key)

Deletes record with given primary key.

disconnect()

Closes internal store and removes the reference to it.

get(doc_class, primary_key)

Returns document instance for given document class and primary key. Raises KeyError if there is no item with given key in the database.

get_many(doc_class, primary_keys)

Returns a list of documents with primary keys from given list. Basically this is just a simple wrapper around get() but some backends can reimplement the method in a much more efficient way.

get_or_create(doc_class, **kwargs)

Queries the database for records associated with given document class and conforming to given extra condtions. If such records exist, picks the first one (the order may be random depending on the database). If there are no such records, creates one.

Returns the document instance and a boolean value “created”.

get_query()

Returns a Query object bound to this storage.

reconnect()

Gracefully closes current connection (if it’s not broken) and connects again to the database (e.g. reopens the file).

save(model, data, primary_key=None)

Saves given model instance into the storage. Returns primary key.

Parameters:

• model – model class
• data – dict containing all properties to be saved
• primary_key – the key for given object; if undefined, will be generated

Note that you must provide current primary key for a model instance which is already in the database in order to update it instead of copying it.

class doqu.backend_base.BaseQueryAdapter(storage, model)

Query adapter for given backend.

count()

Returns the number of records that match given query. The result of q.count() is exactly equivalent to the result of len(q). The implementation details do not differ by default, but it is recommended that the backends stick to the following convention:

• __len__ executes the query, retrieves all matching records and tests the length of the resulting list;
• count executes a special query that only returns a single value: the number of matching records.

Thus, __len__ is more suitable when you are going to iterate the records anyway (and do no extra queries), while count is better when you just want to check if the records exist, or to only use a part of matching records (i.e. a slice).

delete()

Deletes all records that match current query.

order_by(name)

Returns a query object with same conditions but with results sorted by given column.

Parameters:

• name – string: name of the column by which results should be sorted. If the name begins with a -, results will come in reversed order.

values(name)

Returns a list of unique values for given column name.

where(**conditions)

Returns Query instance filtered by given conditions. The conditions are specified by backend’s underlying API.

where_not(**conditions)

Returns Query instance. Inverted version of where().

exception doqu.backend_base.ProcessorDoesNotExist

This exception is raised when given backend does not have a processor suitable for given value. Usually you will need to catch a subclass of this exception.

class doqu.backend_base.LookupManager

Usage:

lookup_manager = LookupManager()

@lookup_manager.register('equals', default=True)  # only one lookup can be default
def exact_match(name, value):
    '''
    Returns native Tokyo Cabinet lookup triplets for given
    backend-agnostic lookup triplet.
    '''
    if isinstance(value, basestring):
        return (
            (name, proto.RDBQCSTREQ, value),
        )
    if isinstance(value, (int, float)):
        return (
            (name, proto.RDBQCNUMEQ, value),
        )
    raise ValueError

Now if you call lookup_manager.resolve('age', 'equals', 99), the returned value will be (('age', proto.RDBCNUMEQ, 99),).

A single generic lookup may yield multiple native lookups because some backends do not support certain lookups directly and therefore must translate them to a combination of elementary conditions. In most cases resolve() will yield a single condition. Its format is determined by the query adapter.

exception_class

alias of LookupProcessorDoesNotExist

resolve(name, operation, value)

Returns a set of backend-specific conditions for given backend-agnostic triplet, e.g.:

('age', 'gt', 90)

will be translated by the Tokyo Cabinet backend to:

('age', 9, '90')

or by the MongoDB backend to:

{'age': {'$gt': 90}}

exception doqu.backend_base.LookupProcessorDoesNotExist

This exception is raised when given backend does not support the requested lookup.

class doqu.backend_base.ConverterManager

An instance of this class can manage property processors for given backend. Processor classes must be registered against Python types or classes. The processor manager allows encoding and decoding data between a model instance and a database record. Each backend supports only a certain subset of Python datatypes and has its own rules in regard to how None values are interpreted, how complex data structures are serialized and so on. Moreover, there’s no way to guess how a custom class should be processed. Therefore, each combination of data type + backend has to be explicitly defined as a set of processing methods (to and from).

exception_class

alias of DataProcessorDoesNotExist

from_db(datatype, value)

Converts given value to given Python datatype. The value must be correctly pre-encoded by the symmetrical PropertyManager.to_db() method before saving it to the database.

Raises DataProcessorDoesNotExist if no suitable processor is defined by the backend.

to_db(value, storage)

Prepares given value and returns it in a form ready for storing in the database.

Raises DataProcessorDoesNotExist if no suitable processor is defined by the backend.

exception doqu.backend_base.DataProcessorDoesNotExist

This exception is raised when given backend does not have a datatype processor suitable for given value.