Storing provenance information

The recordstore module provides an abstraction layer around storage of simulation/analysis records, providing a common interface to different storage methods (simple serialisation, relational database, etc.)

Base class

All record store classes have the following methods. Some stores have additional methods (see below).

class sumatra.recordstore.base.RecordStore

Base class for record store implementations.

delete(project_name, label)

Delete the record with the given label from the given project.

delete_all()

Delete all records from the store.

delete_by_tag(project_name, tag)

Delete all records from the given project that have been tagged with the given tag.

export(project_name, indent=2)

Returns a string with a JSON representation of the project record store.

export_records(records, indent=2)

Returns a string with a JSON representation of the given records.

get(project_name, label)

Retrieve the record with the given label from the given project.

has_project(project_name)

Does the store contain any records for the given project?

import_(project_name, content)

Import records in JSON format.

labels(project_name)

Return the labels of all records in the given project.

list(project_name, tags=None)

Return a list of records for the given project.

If tags is not provided, list all records, otherwise list only records that have been tagged with one or more of the tags.

list_projects()

Return the names of all projects that have records in this store.

most_recent(project_name)

Return the most recent record from the given project.

required_attributes = (u'list_projects', u'save', u'get', u'list', u'labels', u'delete', u'delete_all', u'delete_by_tag', u'most_recent', u'has_project')

save(project_name, record)

Store the given record under the given project.

sync(other, project_name)

Synchronize two record stores so that they contain the same records for a given project.

Where the two stores have the same label (within a project) for different records, those records will not be synced. The method returns a list of non-synchronizable records (empty if the sync worked perfectly).

sync_all(other)

Synchronize all records from all projects between two record stores.

update(project_name, field, value, tags=None)

Modify the records for a given project.

Arguments:

field: the name of a record attribute, e.g. “datastore.root” value:

Minimal record store

class sumatra.recordstore.ShelveRecordStore(shelf_name=u'.smt/records')

Bases: sumatra.recordstore.base.RecordStore

Handles storage of simulation/analysis records based on the Python standard shelve module.

The advantage of this record store is that it has no dependencies. The disadvantages are that it allows only local access and does not support the smtweb interface.

Django-based record store

class sumatra.recordstore.DjangoRecordStore(db_file=u'.smt/records')

Bases: sumatra.recordstore.base.RecordStore

Handles storage of simulation/analysis records in a relational database, via the Django object-relational mapper (ORM), which means that any database supported by Django could in principle be used, although for now we assume SQLite or PostgreSQL.

This record store is needed for the smtweb interface.

Client for remote record store

class sumatra.recordstore.HttpRecordStore(server_url, username=None, password=None, disable_ssl_certificate_validation=True)

Bases: sumatra.recordstore.base.RecordStore

Handles storage of simulation/analysis records on a remote server using HTTP.

The server should support the following URL structure and HTTP methods:

/	GET
/<project_name>/[?tags=<tag1>,<tag2>,...]	GET
/<project_name>/tag/<tag>/	GET, DELETE
/<project_name>/<record_label>/	GET, PUT, DELETE

and should both accept and return JSON-encoded data when the Accept header is “application/json”.

The required JSON structure can be seen in recordstore.serialization.

create_project(project_name, long_name=u'', description=u'')

Create an empty project in the record store.

project_info(project_name)

Return a project’s long name and description.

update_project_info(project_name, long_name=u'', description=u'')

Update a project’s long name and description.

Module functions

sumatra.recordstore.get_record_store(uri)

Return the RecordStore object found at the given URI (which may be a URL or filesystem path).

Transferring provenance information

Handles serialization/deserialization of record store contents to/from JSON.

copyright:Copyright 2006-2015 by the Sumatra team, see doc/authors.txt license:BSD 2-clause, see LICENSE for details.

sumatra.recordstore.serialization.encode_record(record, indent=None)

sumatra.recordstore.serialization.encode_project_info(long_name, description)

Encode a Sumatra project as JSON

sumatra.recordstore.serialization.build_record(data)

Create a Sumatra record from a nested dictionary.

sumatra.recordstore.serialization.decode_record(content)

Create a Sumatra record from a JSON string.

sumatra.recordstore.serialization.decode_records(content)

Create multiple Sumatra records from a JSON string.