The reference implementation

The u1db reference implementation is written in Python, with a SQLite back end. It can be used as a real working implementation by Python code. It is also used to document and test how u1db should work; it has a comprehensive test suite. Implementation authors should port the u1db reference test suite in order to test that their implementation is correct; in particular, sync conformance is defined as being able to sync with the reference implementation.

Fetch with bzr branch lp:u1db or from Launchpad.

To open a new database, use u1db.open:

u1db.open(path, create, document_factory=None)

Open a database at the given location.

Will raise u1db.errors.DatabaseDoesNotExist if create=False and the database does not already exist.

Parameters:

• path – The filesystem path for the database to open.
• create – True/False, should the database be created if it doesn’t already exist?
• document_factory – A function that will be called with the same parameters as Document.__init__.

Returns:

An instance of Database.

Opening returns a Database object:

class u1db.Database

A JSON Document data store.

This data store can be synchronized with other u1db.Database instances.

close()

Release any resources associated with this database.

create_doc(content, doc_id=None)

Create a new document.

You can optionally specify the document identifier, but the document must not already exist. See ‘put_doc’ if you want to override an existing document. If the database specifies a maximum document size and the document exceeds it, create will fail and raise a DocumentTooBig exception.

Parameters:

• content – A Python dictionary.
• doc_id – An optional identifier specifying the document id.

Returns:

Document

create_doc_from_json(json, doc_id=None)

Create a new document.

You can optionally specify the document identifier, but the document must not already exist. See ‘put_doc’ if you want to override an existing document. If the database specifies a maximum document size and the document exceeds it, create will fail and raise a DocumentTooBig exception.

Parameters:

• json – The JSON document string
• doc_id – An optional identifier specifying the document id.

Returns:

Document

create_index(index_name, *index_expressions)

Create an named index, which can then be queried for future lookups. Creating an index which already exists is not an error, and is cheap. Creating an index which does not match the index_expressions of the existing index is an error. Creating an index will block until the expressions have been evaluated and the index generated.

Parameters:

• index_name – A unique name which can be used as a key prefix
• index_expressions –

  index expressions defining the index information.

  Examples:

  “fieldname”, or “fieldname.subfieldname” to index alphabetically sorted on the contents of a field.

  “number(fieldname, width)”, “lower(fieldname)”

delete_doc(doc)

Mark a document as deleted. Will abort if the current revision doesn’t match doc.rev. This will also set doc.content to None.

delete_index(index_name)

Remove a named index.

Parameters:index_name – The name of the index we are removing

get_all_docs(include_deleted=False)

Get the JSON content for all documents in the database.

Parameters:include_deleted – If set to True, deleted documents will be returned with empty content. Otherwise deleted documents will not be included in the results. Returns:(generation, [Document]) The current generation of the database, followed by a list of all the documents in the database.

get_doc(doc_id, include_deleted=False)

Get the JSON string for the given document.

Parameters:

• doc_id – The unique document identifier
• include_deleted – If set to True, deleted documents will be returned with empty content. Otherwise asking for a deleted document will return None.

Returns:

a Document object.

get_doc_conflicts(doc_id)

Get the list of conflicts for the given document.

The order of the conflicts is such that the first entry is the value that would be returned by “get_doc”.

Returns:[doc] A list of the Document entries that are conflicted.

get_docs(doc_ids, check_for_conflicts=True, include_deleted=False)

Get the JSON content for many documents.

Parameters:

• doc_ids – A list of document identifiers.
• check_for_conflicts – If set to False, then the conflict check will be skipped, and ‘None’ will be returned instead of True/False.
• include_deleted – If set to True, deleted documents will be returned with empty content. Otherwise deleted documents will not be included in the results.

Returns:

iterable giving the Document object for each document id in matching doc_ids order.

get_from_index(index_name, *key_values)

Return documents that match the keys supplied.

You must supply exactly the same number of values as have been defined in the index. It is possible to do a prefix match by using ‘*’ to indicate a wildcard match. You can only supply ‘*’ to trailing entries, (eg ‘val’, ‘*’, ‘*’ is allowed, but ‘*’, ‘val’, ‘val’ is not.) It is also possible to append a ‘*’ to the last supplied value (eg ‘val*’, ‘*’, ‘*’ or ‘val’, ‘val*’, ‘*’, but not ‘val*’, ‘val’, ‘*’)

Parameters:

• index_name – The index to query
• key_values – values to match. eg, if you have an index with 3 fields then you would have: get_from_index(index_name, val1, val2, val3)

Returns:

List of [Document]

get_index_keys(index_name)

Return all keys under which documents are indexed in this index.

Parameters:index_name – The index to query Returns:[] A list of tuples of indexed keys.

get_range_from_index(index_name, start_value, end_value)

Return documents that fall within the specified range.

Both ends of the range are inclusive. For both start_value and end_value, one must supply exactly the same number of values as have been defined in the index, or pass None. In case of a single column index, a string is accepted as an alternative for a tuple with a single value. It is possible to do a prefix match by using ‘*’ to indicate a wildcard match. You can only supply ‘*’ to trailing entries, (eg ‘val’, ‘*’, ‘*’ is allowed, but ‘*’, ‘val’, ‘val’ is not.) It is also possible to append a ‘*’ to the last supplied value (eg ‘val*’, ‘*’, ‘*’ or ‘val’, ‘val*’, ‘*’, but not ‘val*’, ‘val’, ‘*’)

Parameters:

• index_name – The index to query
• start_values – tuples of values that define the lower bound of the range. eg, if you have an index with 3 fields then you would have: (val1, val2, val3)
• end_values – tuples of values that define the upper bound of the range. eg, if you have an index with 3 fields then you would have: (val1, val2, val3)

Returns:

List of [Document]

get_sync_target()

Return a SyncTarget object, for another u1db to synchronize with.

Returns:An instance of SyncTarget.

list_indexes()

List the definitions of all known indexes.

Returns:A list of [(‘index-name’, [‘field’, ‘field2’])] definitions.

put_doc(doc)

Update a document. If the document currently has conflicts, put will fail. If the database specifies a maximum document size and the document exceeds it, put will fail and raise a DocumentTooBig exception.

Parameters:doc – A Document with new content. Returns:new_doc_rev - The new revision identifier for the document. The Document object will also be updated.

resolve_doc(doc, conflicted_doc_revs)

Mark a document as no longer conflicted.

We take the list of revisions that the client knows about that it is superseding. This may be a different list from the actual current conflicts, in which case only those are removed as conflicted. This may fail if the conflict list is significantly different from the supplied information. (sync could have happened in the background from the time you GET_DOC_CONFLICTS until the point where you RESOLVE)

Parameters:

• doc – A Document with the new content to be inserted.
• conflicted_doc_revs – A list of revisions that the new content supersedes.

set_document_factory(factory)

Set the document factory that will be used to create objects to be returned as documents by the database.

Parameters:factory – A function that returns an object which at minimum must satisfy the same interface as does the class DocumentBase. Subclassing that class is the easiest way to create such a function.

set_document_size_limit(limit)

Set the maximum allowed document size for this database.

Parameters:limit – Maximum allowed document size in bytes.

sync(url, creds=None, autocreate=True)

Synchronize documents with remote replica exposed at url.

Parameters:

• url – the url of the target replica to sync with.
• creds –

  optional dictionary giving credentials to authorize the operation with the server. For using OAuth the form of creds is:

  {‘oauth’: {

  ‘consumer_key’: ..., ‘consumer_secret’: ..., ‘token_key’: ..., ‘token_secret’: ...

  }}

• autocreate – ask the target to create the db if non-existent.

Returns:

local_gen_before_sync The local generation before the synchronisation was performed. This is useful to pass into whatschanged, if an application wants to know which documents were affected by a synchronisation.

whats_changed(old_generation=0)

Return a list of documents that have changed since old_generation. This allows APPS to only store a db generation before going ‘offline’, and then when coming back online they can use this data to update whatever extra data they are storing.

Parameters:old_generation – The generation of the database in the old state. Returns:(generation, trans_id, [(doc_id, generation, trans_id),...]) The current generation of the database, its associated transaction id, and a list of of changed documents since old_generation, represented by tuples with for each document its doc_id and the generation and transaction id corresponding to the last intervening change and sorted by generation (old changes first)

class u1db.Document(doc_id=None, rev=None, json='{}', has_conflicts=False)

Container for handling a single document.

Variables:

• doc_id – Unique identifier for this document.
• rev – The revision identifier of the document.
• json – The JSON string for this document.
• has_conflicts – Boolean indicating if this document has conflicts

content

Content of the Document.

get_json()

Get the json serialization of this document.

is_tombstone()

Return True if the document is a tombstone, False otherwise.

make_tombstone()

Make this document into a tombstone.

same_content_as(other)

Compare the content of two documents.

set_json(json)

Set the json serialization of this document.