Model and Query API¶

The object-data mapper (ODM) is the core of the library. It defines an API for mapping data in the backend key-value store to objects in Python. It’is name is closely related to object relational Mapping (ORM), a programming technique for converting data between incompatible type systems in traditional relational databases and object-oriented programming languages.

Model¶

The object data mapper presents a method of associating user-defined Python classes, referred as models, with data in a stdnet.BackendDataServer. These python classes are subclasses of stdnet.odm.StdModel.

StdModel Class¶

class stdnet.odm.StdModel(*args, **kwargs)¶

A Model which contains data in Field. This represents the main class of stdnet.odm module.

has_all_data¶: True if this StdModel instance has all back-end data loaded. This applies to persistent instances only. This property is used when committing changes. If all data is available, the commit will replace the previous object data entirely, otherwise it will only update it.

loadedfields()¶: Generator of fields loaded from database

fieldvalue_pairs(exclude_cache=False)¶

Generator of fields,values pairs. Fields correspond to the ones which have been loaded (usually all of them) or not loaded but modified. Check the load_only query function for more details.

If exclude_cache evaluates to True, fields with Field.as_cache attribute set to True won’t be included.

Return type:	a generator of two-elements tuples

clear_cache_fields()¶: Set cache fields to None. Check Field.as_cache for information regarding fields which are considered cache.

get_attr_value(name)¶: Retrieve the value for the attribute name. The name can be nested following the double underscore notation, for example group__name. If the attribute is not available it raises AttributeError.

clone(**data)¶

Utility method for cloning the instance as a new object.

Parameters:	data – additional which override field data.
Return type:	a new instance of this class.

is_valid()¶

Kick off the validation algorithm by checking all StdModel.loadedfields against their respective validation algorithm.

Return type:	Boolean indicating if the model validates.

todict(exclude_cache=False)¶: Return a dictionary of serialised scalar field for pickling. If the exclude_cache flag is True, fields with Field.as_cache attribute set to True will be excluded.

tojson(exclude_cache=True)¶: Return a JSON serialisable dictionary representation.

load_fields(*fields)¶: Load extra fields to this StdModel.

load_related_model(name, load_only=None, dont_load=None)¶

Load a the ForeignKey field name if this is part of the fields of this model and if the related object is not already loaded. It is used by the lazy loading mechanism of one-to-many relationships.

Parameters:	name – the `Field.name` of the `ForeignKey` to load. load_only – Optional parameters which specify the fields to load. dont_load – Optional parameters which specify the fields not to load.
Returns:	the related `StdModel` instance.

classmethod get_field(name)¶: Returns the Field instance at name if available, otherwise it returns None.

classmethod from_base64_data(**kwargs)¶

Load a StdModel from possibly base64encoded data.

This method is used to load models from data obtained from the tojson() method.

classmethod pk()¶: Returns the primary key Field for this model. This is a proxy for the Metaclass.pk attribute.

Model¶

class stdnet.odm.Model¶

This is the base class for both StdModel and Structure classes. It implements the uuid attribute which provides the universal unique identifier for an instance of a model.

_meta¶: A class attribute which is an instance of ModelMeta, it containes all the information needed by a stdnet.BackendDataServer.

session¶: The Session which loaded the instance. Only available, when the instance has been loaded from a stdnet.BackendDataServer via a query operation.

DoesNotExist¶

Exception raised when an instance of a model does not exist.

alias of ObjectNotFound

DoesNotValidate¶

Exception raised when an instance of a model does not validate. Usually raised when trying to save an invalid instance.

alias of ObjectNotValidated

get_state(**kwargs)¶: Return the current ModelState for this Model. If kwargs parameters are passed a new ModelState is created, otherwise it returns the cached value.

pkvalue()¶: Value of primary key

uuid¶: Universally unique identifier for an instance of a Model.

get_attr_value(name)¶

Provided for compatibility with StdModel.get_attr_value(). For this class it simply get the attribute at name:

return getattr(self, name)

save()¶: Save the model by adding it to the session. If the session is not available, it raises a SessionNotAvailable exception.

delete()¶: Delete the model. If the session is not available, it raises a SessionNotAvailable exception.

Create Model¶

stdnet.odm.create_model(name, *attributes, **params)¶

Create a Model class for objects requiring and interface similar to StdModel. We refers to this type of models as local models since instances of such models are not persistent on a stdnet.BackendDataServer.

Parameters:	name – Name of the model class. attributes – positiona attribute names. These are the only attribute available to the model during the default constructor. params – key-valued parameter to pass to the `ModelMeta` constructor.
Returns:	a local `Model` class.

Model Meta¶

class stdnet.odm.ModelMeta(model, fields, app_label=None, modelkey=None, name=None, register=True, pkname=None, ordering=None, attributes=None, abstract=False, **kwargs)¶

A class for storing meta data for a Model class. To override default behaviour you can specify the Meta class as an inner class of Model in the following way:

from datetime import datetime
from stdnet import odm

class MyModel(odm.StdModel):
    timestamp = odm.DateTimeField(default = datetime.now)
    ...

    class Meta:
        ordering = '-timestamp'
        name = 'custom'

Parameters:	register – if `True` (default), this `ModelMeta` is registered in the global models hashtable. abstract – Check the `abstract` attribute. ordering – Check the `ordering` attribute. app_label – Check the `app_label` attribute. name – Check the `name` attribute. modelkey – Check the `modelkey` attribute. attributes – Check the `attributes` attribute.

This is the list of attributes and methods available. All attributes, but the ones mantioned above, are initialized by the object relational mapper.

abstract¶: If True, This is an abstract Meta class.

model¶: Model for which this class is the database metadata container.

name¶: Usually it is the Model class name in lower-case, but it can be customised.

app_label¶: Unless specified it is the name of the directory or file (if at top level) containing the Model definition. It can be customised.

modelkey¶: The modelkey which is by default given by app_label.name.

ordering¶

Optional name of a Field in the model. If provided, model indices will be sorted with respect to the value of the specified field. It can also be a autoincrement instance. Check the sorting documentation for more details.

Default: None.

dfields¶: dictionary of Field instances.

fields¶: list of all Field instances.

scalarfields¶: Ordered list of all Field which are not StructureField. The order is the same as in the Model definition. The pk field is not included.

indices¶: List of Field which are indices (Field.index attribute set to True).

pk¶: The Field representing the primary key.

related¶: Dictionary of related.RelatedManager for the model. It is created at runtime by the object data mapper.

manytomany¶: List of ManyToManyField names for the model. This information is useful during registration.

attributes¶: Additional attributes for model.

type¶: Model type, either structure or object.

make_object(state=None, backend=None)¶: Create a new instance of model from a state tuple.

pkname()¶: Primary key name. A shortcut for self.pk.name.

pk_to_python(value, backend)¶: Convert the primary key value to a valid python representation.

is_valid(instance)¶: Perform validation for instance and stores serialized data, indexes and errors into local cache. Return True if the instance is ready to be saved to database.

backend_fields(fields)¶: Return a two elements tuple containing a list of fields names and a list of field attribute names.

as_dict()¶: Model metadata in a dictionary

Model State¶

class stdnet.odm.ModelState(instance, iid=None, action=None)¶

The database state of a Model.

action¶: Action to be performed by the backend server when committing changes to the instance of Model for which this is a state.

persistent¶: True if the instance is persistent in the backend server.

iid¶: Instance primary key or a temporary key if not yet available.

autoincrement¶

class stdnet.odm.autoincrement(incrby=1, desc=False)¶

An autoincrement is used in a StdModel Meta class to specify a model with incremental sorting.

incrby¶

The amount to increment the score by when a duplicate element is saved.

Default: 1.

For example, the stdnet.apps.searchengine.Word model is defined as:

class Word(odm.StdModel):
    id = odm.SymbolField(primary_key = True)

    class Meta:
        ordering = -autoincrement()

This means every time we save a new instance of Word, and that instance has an id already available, the score of that word is incremented by the incrby attribute.

Queries¶

Query base class¶

class stdnet.odm.Q(meta, session, select_related=None, ordering=None, fields=None, get_field=None, name=None, keyword=None)¶

Base class for Query and QueryElement.

meta¶: The StdModel._meta attribute.

model¶: the StdModel class for this query.

backend¶: the stdnet.BackendDataServer class for this query.

get_field(field)¶

A Q performs a series of operations and ultimately generate of set of matched elements ids. If on the other hand, a different field is required, it can be specified with the get_field() method. For example, lets say a model has a field called object_id which contains ids of another model, we could use:

qs = session.query(MyModel).get_field('object_id')

to obtain a set containing the values of matched elements object_id fields.

Parameters:	field – the name of the field which will be used to obtained the matched elements value. Must be an index.
Return type:	a new `Q` instance.

backend_query(**kwargs)¶: Build the stdnet.utils.async.BackendQuery for this instance. This is a virtual method with different implementation in Query and QueryElement.

Query¶

class stdnet.odm.Query(*args, **kwargs)¶

A Query is produced in terms of a given Session, using the Session.query() method:

qs = session.query(MyModel)

A query is equivalent to a collection of SELECT statements for a standard relational database. It has a a generative interface whereby successive calls return a new Query object, a copy of the former with additional criteria and options associated with it.

ATTRIBUTES

_meta¶: The StdModel._meta attribute.

model¶: the StdModel class for this query.

session¶: The Session which created the Query via the Session.query() method.

backend¶: the stdnet.BackendDataServer holding the data to query.

_get_field¶

When iterating over a Query, you get back instances of the model class. However, if _get_field is specified you get back values of the field specified. This can be changed via the get_field() method:

qs = query.get_field('name').all()

the results is a list of name values (provided the model has a name field of course).

Default: None.

fargs¶

Dictionary containing the filter lookup parameters each one of them corresponding to a where clause of a select. This value is manipulated via the filter() method.

Default: {}.

eargs¶

Dictionary containing the exclude lookup parameters each one of them corresponding to a where clause of a select. This value is manipulated via the exclude() method.

Default: {}.

ordering¶: optional ordering field.

text¶

optional text to filter result on.

Default: "".

METHODS

__init__(*args, **kwargs)¶: A Query is not initialized directly but via the Session.query() or Manager.query() methods.

filter(**kwargs)¶

Create a new Query with additional clauses corresponding to where or limit in a SQL SELECT statement.

Parameters:	kwargs – dictionary of limiting clauses.
Return type:	a new `Query` instance.

For example:

qs = session.query(MyModel)
result = qs.filter(group = 'planet')

exclude(**kwargs)¶

Returns a new Query with additional clauses corresponding to EXCEPT in a SQL SELECT statement.

Parameters:	kwargs – dictionary of limiting clauses.
Return type:	a new `Query` instance.

Using an equivalent example to the filter() method:

qs = session.query(MyModel)
result1 = qs.exclude(group = 'planet')
result2 = qs.exclude(group__in = ('planet','stars'))

union(*queries)¶

Return a new Query obtained form the union of this Query with one or more queries. For example, lets say we want to have the union of two queries obtained from the filter() method:

query = session.query(MyModel)
qs = query.filter(field1 = 'bla').union(query.filter(field2 = 'foo'))

intersect(*queries)¶: Return a new Query obtained form the intersection of this Query with one or more queries. Workds the same way as the union() method.

sort_by(ordering)¶

Sort the query by the given field

Parameters:	ordering – a string indicating the class:Field name to sort by. If prefixed with `-`, the sorting will be in descending order, otherwise in ascending order.
Return type:	a new `Query` instance.

search(text, lookup=None)¶

Search text in model. A search engine needs to be installed for this function to be available.

Parameters:	text – a string to search.
Return type:	a new `Query` instance.

where(code, load_only=None)¶

For backend supporting scripting, it is possible to construct complex queries which execute the scripting code against each element in the query. The coe should reference an instance of model by this keyword.

Parameters:	code – a valid expression in the scripting language of the database. load_only – Load only the selected fields when performing the query (this is different from the `load_only()` method which is used when fetching data from the database). This field is an optimization which is used by the redis backend only and can be safely ignored in most use-cases.
Returns:	a new `Query`

search_queries(q)¶: Return a new QueryElem for q applying a text search.

load_related(related, *related_fields)¶

It returns a new Query that automatically follows the foreign-key relationship related.

Parameters:	related – A field name corresponding to a `ForeignKey` in `Query.model`. related_fields – optional `Field` names for the `related` model to load. If not provided, all fields will be loaded.

This function is performance boost when accessing the related fields of all (most) objects in your query.

If Your model contains more than one foreign key, you can use this function in a generative way:

qs = myquery.load_related('rel1').load_related('rel2','field1','field2')

Return type:	a new `Query`.

load_only(*fields)¶: This is provides a performance boost in cases when you need to load a subset of fields of your model. The boost achieved is less than the one obtained when using Query.load_related(), since it does not reduce the number of requests to the database. However, it can save you lots of bandwidth when excluding data intensive fields you don’t need.

dont_load(*fields)¶: Works like load_only() to provides a performance boost in cases when you need to load all fields except a subset specified by fields.

items(callback=None)¶: Retrieve all items for this Query.

get(**kwargs)¶: Return an instance of a model matching the query. A special case is the query on id which provides a direct access to the session instances. If the given primary key is present in the session, the object is returned directly without performing any query.

count()¶: Return the number of objects in self. This method is efficient since the Query does not receive any data from the server apart from the number of matched elements. It construct the queries and count the objects on the server side.

delete()¶: Delete all matched elements of the Query. It returns the list of ids deleted.

construct()¶: Build the QueryElement representing this query.

backend_query(**kwargs)¶: Build and return the stdnet.utils.async.BackendQuery. This is a lazy method in the sense that it is evaluated once only and its result stored for future retrieval.

test_unique(fieldname, value, instance=None, exception=None)¶

Test if a given field fieldname has a unique value in model. The field must be an index of the model. If the field value is not unique and the instance is not the same an exception is raised.

Parameters:	fieldname – `Field` name to test vale – `Field` value instance – optional instance of `model` exception – optional exception class to raise if the test fails. Default: `ModelMixin.DoesNotValidate`.
Returns:	value

map_reduce(map_script, reduce_script, **kwargs)¶: Perform a map/reduce operation on this query.

aggregate(kwargs)¶: Aggregate lookup parameters.

QueryElement¶

class stdnet.odm.QueryElement(*args, **kwargs)¶

An element of a Query.

qs¶: the Query which contains this QueryElement.

underlying¶: the element contained in the QueryElement. This underlying is an iterable or another Query.

valid¶: if False this QueryElement has no underlying elements and won’t produce any query.

SearchEngine Interface¶

class stdnet.odm.SearchEngine(backend=None, logger=None, max_in_session=None)¶

Stdnet search engine driver. This is an abstract class which expose the base functionalities for full text-search on model instances. Stdnet also provides a python implementation of this interface.

The main methods to be implemented are add_item(), remove_index() and search_model().

word_middleware¶

A list of middleware functions for preprocessing text to be indexed. A middleware function has arity 1 by accepting an iterable of words and returning an iterable of words. Word middleware functions are added to the search engine via the add_word_middleware() method.

For example this function remove a group of words from the index:

se = SearchEngine()

class stopwords(object):

    def __init__(self, *swords):
        self.swords = set(swords)

    def __call__(self, words):
        for word in words:
            if word not in self.swords:
                yield word

se.add_word_middleware(stopwords('and','or','this','that',...))

max_in_session¶: Maximum number of instances to be reindexed in one session. Default 1000.

backend¶: Backend for this search engine.

register(model, related=None)¶

Register a StdModel with this search SearchEngine. When registering a model, every time an instance is created, it will be indexed by the search engine.

Parameters:	model – a `StdModel` class. related – a list of related fields to include in the index.

words_from_text(text, for_search=False)¶

Generator of indexable words in text. This functions loop through the word_middleware attribute to process the text.

Parameters:

text – string from which to extract words.
for_search –
flag indicating if the the words will be used for search or to index the database. This flug is used in conjunction with the middleware flag for_search. If this flag is True (i.e. we need to search the database for the words in text), only the middleware functions in word_middleware enabled for searching are used.

Default: False.

return a list of cleaned words.

split_text(text)¶: Split text into words and return an iterable over them. Can and should be reimplemented by subclasses.

add_word_middleware(middleware, for_search=True)¶

Add a middleware function to the list of word_middleware, for preprocessing words to be indexed.

Parameters:	middleware – a callable receving an iterable over words. for_search – flag indicating if the middleware can be used for the text to search. Default: `True`.

index_item(item)¶

This is the main function for indexing items. It extracts content from the given item and add it to the index.

Parameters:	item – an instance of a `stdnet.odm.StdModel`.

index_items_from_model(*args, **kwargs)¶

This is the main function for indexing items. It extracts content from a list of items belonging to model and add it to the index.

Parameters:	items – an iterable over instances of of a `stdnet.odm.StdModel`. model – The model of all items. transaction – A transaction for updating indexes.

query(model)¶: Return a query for model when it needs to be indexed.

reindex(*args, **kwargs)¶: Re-index models by removing indexes and rebuilding them by iterating through all the instances of REGISTERED_MODELS.

session()¶: Create a session for the search engine

remove_item(item_or_model, session, ids=None)¶: Remove an item from the search indices

add_item(item, words, transaction)¶

Create indices for item and each word in words. Must be implemented by subclasses.

Parameters:	item – a model instance to be indexed. It does not need to be a `stdnet.odm.StdModel`. words – iterable over words. It has been obtained from the text in item via the `word_middleware`. transaction – The `Transaction` used.

search(text, include=None, exclude=None, lookup=None)¶

Full text search. Must be implemented by subclasses.

Parameters:	test – text to search include – optional list of models to include in the search. If not provided all `REGISTERED_MODELS` will be used. exclude – optional list of models to exclude for the search. lookup – currently not used.

search_model(query, text, lookup=None)¶

Search text in model instances. This is the functions needing implementation by custom serach engines.

Parameters:	query – a `Query` on a `StdModel`. text – text to search lookup – Optional lookup, one of `contains` or `in`.
Return type:	An updated `Query`.

flush(full=False)¶: Clean the search engine

Data Structures¶

Data structures are subclasses of Structure, which in term is a subclass of Model. They are rarely used in stand alone mode, instead they form the back-end of data structure fields.

There are five of them:

List, implemented as a doubly-linked sequence.

Set, a container of unique values.

HashTable, unique associative container.

Zset, an ordered container of unique values.

TS, a time-series implemented as a ordered unique associative container.

An additional structure is provided in the stdnet.apps.columnts module

stdnet.apps.columnts.ColumnTS a numeric multivariate time-series structure (useful for modelling financial data for example).

Note

Stand alone data structures are available for redis back-end only. Usually, one uses these models via a data-structure fields.

Creating Structures¶

Creating the five structures available in stdnet is accomplished in the following way:

from stdnet import odm

models = odm.Router('redis://localhost:6379')
li = models.register(odm.List())

At this point the li instance is registered with a Router and the session API can be used:

with models.session().begin() as t:
    t.add(li)
    li.push_back('bla')
    li.push_back('foo')

If no id is specified, stdnet will create one for you:

>>> l.id
'2d0cbac9'

Base Class and Mixins¶

class stdnet.odm.Structure(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)¶

A Model which is used a base class for remote data-structures. Remote structures are the backend of structured fields but they can also be used as stand alone objects. For example:

import stdnet
db = stdnet.getdb(...)
mylist = db.list('bla')

cache¶: A python StructureCache for this Structure. The cache is used when adding or removing data via Transaction as well as for storing the results obtained form a call to items().

value_pickler¶

Class used for serialize and unserialize values. If None the stdnet.utils.encoders.NumericDefault will be used.

Default None.

field¶: The StructureField which this owns this Structure. Default None.

model¶: The StdModel which contains the field of this Structure. Only available if field is defined.

size()¶: Number of elements in the Structure.

items()¶: All items of this Structure. Implemented by subclasses.

set_cache(data)¶: Set the cache for the Structure. Do not override this function. Use load_data() method instead.

load_data(data)¶: Load data from the stdnet.BackendDataServer.

backend_structure(client=None)¶: Returns a valid stdnet.BackendStructure for this Structure.

class stdnet.odm.Sequence¶

Mixin for a Structure which implements a kind of sequence container. The elements in a sequence container are ordered following a linear sequence.

push_back(*args, **kwargs)¶: Appends a copy of value at the end of the Sequence.

pop_back(*args, **kwargs)¶: Remove the last element from the Sequence.

class stdnet.odm.PairMixin¶

A mixin for structures with which holds pairs. It is the parent class of KeyValueMixin and it is used as base class for the ordered set structure Zset.

pickler¶: An encoder for the additional value in the pair. The additional value is a field key for Hashtable, a numeric score for Zset and a tim value for TS.

items(*args, **kwargs)¶: Iterator over items (pairs) of PairMixin.

values(*args, **kwargs)¶: Iteratir over values of PairMixin.

pair(pair)¶: Add a pair to the structure.

update(*args, **kwargs)¶

Add mapping dictionary to hashtable. Equivalent to python dictionary update method.

Parameters:	mapping – a dictionary of field values.

class stdnet.odm.KeyValueMixin¶

A mixin for ordered and unordered key-valued pair containers. A key-value pair container has the values() and items() methods, while its iterator is over keys.

get(key, default=None)¶

Retrieve a single element from the structure. If the element is not available return the default value.

Parameters:	key – lookup field default – default value when the field is not available

remove(*args, **kwargs)¶: Remove keys from the key-value container.

class stdnet.odm.OrderedMixin¶

A mixin for a Structure which maintains ordering with respect a numeric value, the score.

front()¶: Return the front pair of the structure

back()¶: Return the back pair of the structure

count(start, stop)¶: Count the number of elements bewteen start and stop.

range(start, stop, callback=None, withscores=True, **options)¶: Return a range with scores between start and end.

irange(start=0, end=-1, callback=None, withscores=True, **options)¶: Return the range by rank between start and end.

pop_range(start, stop, callback=None, withscores=True)¶: pop a range by score from the OrderedMixin

ipop_range(start=0, stop=-1, callback=None, withscores=True)¶: pop a range from the OrderedMixin

class stdnet.odm.StructureCache¶

Interface for all Structure.cache classes.

clear()¶: Clear the cache for data

List¶

class stdnet.odm.List(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)¶

A doubly-linked list Structure. It expands the Sequence mixin with functionalities to add and remove from the front of the list in an efficient manner.

pop_front(*args, **kwargs)¶: Remove the first element from of the list.

block_pop_back(*args, **kwargs)¶: Remove the last element from of the list. If no elements are available, blocks for at least timeout seconds.

block_pop_front(*args, **kwargs)¶: Remove the first element from of the list. If no elements are available, blocks for at least timeout seconds.

push_front(*args, **kwargs)¶: Appends a copy of value to the beginning of the list.

Set¶

class stdnet.odm.Set(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)¶

An unordered set Structure. Equivalent to a python set.

add(*args, **kwargs)¶: Add value to the set

update(*args, **kwargs)¶: Add iterable values to the set

discard(*args, **kwargs)¶: Remove an element value from a set if it is a member.

remove(*args, **kwargs)¶: Remove an element value from a set if it is a member.

difference_update(*args, **kwargs)¶: Remove an iterable of values from the set.

OrderedSet¶

class stdnet.odm.Zset(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)¶

An ordered version of Set. It derives from OrderedMixin and PairMixin.

rank(value)¶: The rank of a given value. This is the position of value in the OrderedMixin container.

HashTable¶

class stdnet.odm.HashTable(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)¶: A Structure which is the networked equivalent to a Python dict. Derives from KeyValueMixin.

TS¶

class stdnet.odm.TS(value_pickler=None, name='', field=False, session=None, pkvalue=None, **kwargs)¶

A timeseries is a Structure which derives from OrderedMixin and KeyValueMixin. It represents an ordered associative container where keys are timestamps and values are objects.

rank(dte)¶: The rank of a given dte in the timeseries

ipop(index)¶: Pop a value at index from the TS. Return None if index is not out of bound.

times(start, stop, callback=None, **kwargs)¶: The times between times start and stop.

itimes(start=0, stop=-1, callback=None, **kwargs)¶: The times between rank start and stop.

Sessions¶

A Session is the middleware between a Manager and a stdnet.BackendDataServer. It is obtained from either the Router.session() or, equivalently, from the Manager.session(). A Session is an holding zone for SessionModel and it communicates with the stdnet.BackendDataServer via Transaction.

Session¶

class stdnet.odm.Session(router)¶

The middleware for persistent operations on the back-end. It is created via the Router.session() method.

transaction¶: A Transaction instance. Not None if this Session is in a transactional state

router¶: Instance of the Router which created this Session.

dirty¶: The set of instances in this Session which have been modified.

begin(**options)¶

Begin a new Transaction. If this Session is already in a transactional state, an error will occur. It returns the transaction attribute.

This method is mostly used within a with statement block:

with session.begin() as t:
    t.add(...)
    ...

which is equivalent to:

t = session.begin()
t.add(...)
...
session.commit()

options parameters are passed to the Transaction constructor.

commit()¶: Commit the current transaction. If no transaction is in progress, this method open one. Rarely used directly, see the begin() method for details on how to start and close a transaction using the with construct.

query(model, **kwargs)¶: Create a new Query for model.

empty(model)¶: Returns an empty Query for model.

update_or_create(*args, **kwargs)¶

Update or create a new instance of model. This method can raise an exception if the kwargs dictionary contains field data that does not validate.

Parameters:	model – a `StdModel` kwargs – dictionary of parameters.
Returns:	A two elements tuple containing the instance and a boolean indicating if the instance was created or not.

add(instance, modified=True, **params)¶

Add an instance to the session. If the session is not in a transactional state, this operation commits changes to the back-end server immediately and return what is return by Transaction.commit(). Otherwise it return the input instance.

Parameters:	instance – a `Model` instance. It must be registered with the `router` which created this `Session`. modified – a boolean flag indicating if the instance was modified.
Returns:	the instance.

If the instance is persistent (it is already stored in the database), an updated will be performed, otherwise a new entry will be created once the commit() method is invoked.

delete(instance_or_query)¶

Include an instance or a query to this Session list of data to be deleted. If the session is not in a transactional state, this operation commits changes to the backend server immediately.

Parameters:	instance_or_query – a `Model` instance or a `Query`.

flush(model)¶: Completely flush a Model from the database. No keys associated with the model will exists after this operation.

clean(model)¶: Remove empty keys for a Model from the database. No empty keys associated with the model will exists after this operation.

keys(model)¶: Retrieve all keys for a model.

model(model, create=True)¶: Returns the SessionModel for model which can be Model, or a MetaClass, or an instance of Model.

expunge(instance=None)¶: Remove instance from this Session. If instance is not given, it removes all instances from this Session.

manager(model)¶: Retrieve the Manager for model which can be any of the values valid for the model() method.

Session Model¶

class stdnet.odm.SessionModel(manager)¶

A SessionModel is the container of all objects for a given Model in a stdnet Session.

backend¶: The backend for this SessionModel.

read_backend¶: The read-only backend for this SessionModel.

model¶: The Model for this SessionModel.

new¶: The set of all new instances within this SessionModel. This instances will be inserted in the database.

modified¶: The set of all modified instances within this Session. This instances will.

deleted¶: The set of all instance pks marked as deleted within this Session.

dirty¶: The set of all instances which have changed, but not deleted, within this SessionModel.

iterdirty()¶: Ordered iterator over dirty elements.

add(instance, modified=True, persistent=None, force_update=False)¶

Add a new instance to this SessionModel.

Parameters:	modified – Optional flag indicating if the `instance` has been modified. By default its value is `True`. force_update – if `instance` is persistent, it forces an update of the data rather than a full replacement. This is used by the `insert_update_replace()` method.
Return type:	The instance added to the session

delete(instance, session)¶: delete an instance

pop(instance)¶

Remove instance from the SessionModel. Instance could be a Model or an id.

Parameters:	instance – a `Model` or an `id`.
Return type:	the `Model` removed from session or `None` if it was not in the session.

expunge(instance)¶

Remove instance from the Session. Instance could be a Model or an id.

Parameters:	instance – a `Model` or an id
Return type:	the `Model` removed from session or `None` if it was not in the session.

post_commit(results)¶

Process results after a commit.

Parameters:	results – iterator over `stdnet.instance_session_result` items.
Return type:	a two elements tuple containing a list of instances saved and a list of ids of instances deleted.

flush()¶: Completely flush model from the database. No keys associated with the model will exists after this operation.

clean()¶: Remove empty keys for a model from the database. No empty keys associated with the model will exists after this operation.

keys()¶: Retrieve all keys for a model. Uses the Manager.read_backend.

Transaction¶

class stdnet.odm.Transaction(session, name=None, signal_commit=True, signal_delete=True)¶

Transaction class for pipelining commands to the backend server. An instance of this class is usually obtained via the Session.begin() or the Manager.transaction() methods:

t = session.begin()

or using the with context manager:

with session.begin() as t:
    ...

ATTRIBUTES

session¶: Session which is being transacted.

name¶: Optional Transaction name

backend¶: the stdnet.BackendDataServer to which the transaction is being performed.

signal_commit¶

If True, a signal for each model in the transaction is triggered just after changes are committed. The signal carries a list of updated instances of the model, the Session and the Transaction itself.

default True.

signal_delete¶

If True a signal for each model in the transaction is triggered just after deletion of instances. The signal carries the list ids of deleted instances of the mdoel, the Session and the Transaction itself.

default True.

deleted¶: Dictionary of list of ids deleted from the backend server after a commit operation. This dictionary is only available once the transaction has finished.

saved¶: Dictionary of list of ids saved in the backend server after a commit operation. This dictionary is only available once the transaction has finished.

executed¶: True when this transaction has been executed. A transaction can be executed once only via the commit() method. An executed transaction if finished once a response from the backend server has been processed.

finished¶: True when this transaction is done.

add(instance, **kwargs)¶: A convenience proxy for Session.add() method.

delete(instance)¶: A convenience proxy for Session.delete() method.

expunge(instance=None)¶: A convenience proxy for Session.expunge() method.

query(model, **kwargs)¶: A convenience proxy for Session.query() method.

model(model)¶: A convenience proxy for Session.model() method.

commit()¶: Close the transaction and commit session to the backend.

Managers¶

Manager¶

class stdnet.odm.Manager(model, backend=None, read_backend=None, router=None)¶

Before a StdModel can be used in conjunction with a backend server, a Manager must be associated with it via a Router. Check the registration tutorial for further info:

class MyModel(odm.StdModel):
    group = odm.SymbolField()
    flag = odm.BooleanField()

models = odm.Router()
models.register(MyModel)

manager = models[MyModel]

Managers are used as Session and Query factories for a given StdModel:

session = router[MyModel].session()
query = router[MyModel].query()

A model can specify a custom manager by creating a Manager subclass with additional methods:

class MyModelManager(odm.Manager):

    def special_query(self, **kwargs):
        ...

At this point we need to tell the model about the custom manager, and we do so by setting the manager_class attribute in the StdModel:

class MyModel(odm.StdModel):
    ...
    
    manager_class = MyModelManager

model¶: The StdModel for this Manager. This attribute is assigned by the Object data mapper at runtime.

router¶: The Router which contain this this Manager.

backend¶: The stdnet.BackendDataServer for this Manager.

read_backend¶: A stdnet.BackendDataServer for read-only operations (Queries).

query_class¶: Class for querying. Default is Query.

session_factory¶: alias of Session

session(session=None)¶: Returns a new Session. This is a shortcut for the Router.session() method.

new(*args, **kwargs)¶

Create a new instance of model and commit it to the backend server. This a shortcut method for the more verbose:

instance = manager.session().add(MyModel(**kwargs))

update_or_create(**kwargs)¶: Invokes the Session.update_or_create method.

all()¶

Return all instances for this manager. Equivalent to:

self.query().all()

create_all()¶: A method which can implement table creation. For sql models. Does nothing for redis or mongo.

query(session=None)¶: Returns a new Query for Manager.model.

empty()¶: Returns an empty Query for Manager.model.

filter(**kwargs)¶: Returns a new Query for Manager.model with a filter.

exclude(**kwargs)¶: Returns a new Query for Manager.model with a exclude filter.

search(text, lookup=None)¶: Returns a new Query for Manager.model with a full text search value.

get(**kwargs)¶: Shortcut for self.query().get**kwargs).

pkvalue(instance)¶: Return the primary key value for instance.

RelatedManager¶

class stdnet.odm.related.RelatedManager(field, model=None, instance=None)¶

Base class for managers handling relationships between models. While standard Manager are class properties of a model, related managers are accessed by instances to easily retrieve instances of a related model.

relmodel¶: The StdModel this related manager relates to.

related_instance¶: An instance of the relmodel.

session(session=None)¶: Override Manager.session() so that this RelatedManager can retrieve the session from the related_instance if available.

One2ManyRelatedManager¶

class stdnet.odm.related.One2ManyRelatedManager(field, model=None, instance=None)¶: A specialised RelatedManager for handling one-to-many relationships under the hood. If a model has a ForeignKey field, instances of that model will have access to the related (foreign) objects via a simple attribute of the model.

Many2ManyRelatedManager¶

class stdnet.odm.related.Many2ManyRelatedManager(field, model=None, instance=None)¶

A specialized Manager for handling many-to-many relationships under the hood. When a model has a ManyToManyField, instances of that model will have access to the related objects via a simple attribute of the model.

add(value, session=None, **kwargs)¶: Add value, an instance of formodel to the through model. This method can only be accessed by an instance of the model for which this related manager is an attribute.

remove(value, session=None)¶: Remove value, an instance of self.model from the set of elements contained by the field.

throughquery(session=None)¶: Return a Query on the throughmodel, the model used to hold the many-to-many relationship.

LazyProxy¶

class stdnet.odm.LazyProxy(field)¶

Base class for descriptors used by ForeignKey and StructureField.

field¶: The Field which create this descriptor. Either a ForeignKey or a StructureField.

load(instance, session)¶: Load the lazy data for this descriptor. Implemented by subclasses.

Registration¶

To interact with a stdnet.BackendDataServer, Models must registered. Registration is obtained via a Router which has two methods for registering models. The first one is the Router.register() method which is used to register a model and, possibly, all its related models. The second method is the Router.register_applications() which registers all Model from a list of python dotted paths or python modules.

Check the registering models tutorial for further explanation and examples.

Router¶

class stdnet.odm.Router(default_backend=None, install_global=False)¶

A router is a mapping of Model to the registered Manager of that model:

from stdnet import odm

models = odm.Router()
models.register(MyModel, ...)

# dictionary Notation
query = models[MyModel].query()

# or dotted notation (lowercase)
query = models.mymodel.query()

The models instance in the above snipped can be set globally if one wishes to do so.

pre_commit¶

A signal which can be used to register callbacks before instances are committed:

models.pre_commit.connect(callback, sender=MyModel)

pre_delete¶

A signal which can be used to register callbacks before instances are deleted:

models.pre_delete.connect(callback, sender=MyModel)

post_commit¶

A signal which can be used to register callbacks after instances are committed:

models.post_commit.connect(callback, sender=MyModel)

post_delete¶

A signal which can be used to register callbacks after instances are deleted:

models.post_delete.connect(callback, sender=MyModel)

default_backend¶: The default backend for this Router. This is used when calling the register() method without explicitly passing a backend.

registered_models¶: List of registered Model.

search_engine¶: The SearchEngine for this Router. This must be created by users. Check full text search tutorial for information.

set_search_engine(engine)¶: Set the search engine for this Router.

register(model, backend=None, read_backend=None, include_related=True, **params)¶

Register a Model with this Router. If the model was already registered it does nothing.

Parameters:

model – a Model class.
backend – a stdnet.BackendDataServer or a connection string.
read_backend – Optional stdnet.BackendDataServer for read operations. This is useful when the server has a master/slave configuration, where the master accept write and read operations and the slave read only operations.
include_related – True if related models to model needs to be registered. Default True.
params – Additional parameters for the getdb() function.

Returns:

the number of models registered.

from_uuid(uuid, session=None)¶: Retrieve a Model from its universally unique identifier uuid. If the uuid does not match any instance an exception will raise.

flush(exclude=None, include=None, dryrun=False)¶

Flush registered_models.

Parameters:	exclude – optional list of model names to exclude. include – optional list of model names to include. dryrun – Doesn’t remove anything, simply collect managers to flush.

unregister(model=None)¶: Unregister a model if provided, otherwise it unregister all registered models. Return a list of unregistered model managers or None if no managers were removed.

register_applications(applications, models=None, backends=None)¶

A higher level registration functions for group of models located on application modules. It uses the model_iterator() function to iterate through all Model models available in applications and register them using the register() low level method.

Parameters:	applications – A String or a list of strings representing python dotted paths where models are implemented. models – Optional list of models to include. If not provided all models found in applications will be included. backends – optional dictionary which map a model or an application to a backend connection string.
Return type:	A list of registered `Model`.

For example:

mapper.register_application_models('mylib.myapp')
mapper.register_application_models(['mylib.myapp', 'another.path'])
mapper.register_application_models(pythonmodule)
mapper.register_application_models(['mylib.myapp',pythonmodule])

session()¶: Obatain a new Session for this Router.

create_all()¶: Loop though registered_models and issue the Manager.create_all() method.

add(instance)¶

Add an instance to its backend database. This is a shurtcut method for:

self.session().add(instance)

Model and Query API¶

Model¶

StdModel Class¶

Model¶

Create Model¶

Model Meta¶

Model State¶

autoincrement¶

Queries¶

Query base class¶

Query¶

QueryElement¶

SearchEngine Interface¶

Data Structures¶

Creating Structures¶

Base Class and Mixins¶

List¶

Set¶

OrderedSet¶

HashTable¶

TS¶

Sessions¶

Session¶

Session Model¶

Transaction¶

Managers¶

Manager¶

RelatedManager¶

One2ManyRelatedManager¶

Many2ManyRelatedManager¶

LazyProxy¶

Registration¶

Router¶

Table Of Contents

Previous topic

Next topic

This Page

Navigation

Model and Query API¶

Model¶

StdModel Class¶

Model¶

Create Model¶

Model Meta¶

Model State¶

autoincrement¶

Queries¶

Query base class¶

Query¶

QueryElement¶

SearchEngine Interface¶

Data Structures¶

Creating Structures¶

Base Class and Mixins¶

List¶

Set¶

OrderedSet¶

HashTable¶

TS¶

Sessions¶

Session¶

Session Model¶

Transaction¶

Managers¶

Manager¶

RelatedManager¶

One2ManyRelatedManager¶

Many2ManyRelatedManager¶

LazyProxy¶

Registration¶

Router¶

Table Of Contents

Previous topic

Next topic

This Page

Quick search