Internals and Utilities

Default Settings

Stdnet comes with a singleton setting instance which can be accessed by:

from stdnet.conf import settings

The setting instance contains few default parameters used in throughout the library. This parameters can be changed by the user by simply overriding them.

settings.DEFAULT_BACKEND

the default stdnet.BackendDataServer connection string. Check the registering models documentation.

Default "redis://127.0.0.1:6379/?db=7".

settings.REDIS_PY_PARSER

Set stdnet to use the internal python parser for redis. By default it is set to False which causes the library to choose the best possible available. More information is contained in the redis parser documentation.

Default False.

settings.MAX_CONNECTIONS

The maximim number of connections to have opened at the same time.

Default unlimited.

settings.RedisConnectionClass

The Redis Connection class. If not set the stdnet.lib.connection.Connection will be used.

Default None

To change settings:

from stdnet.conf import settings

settings.DEFAULT_BACKEND = 'redis://127.0.0.1:6379/?db=5'

Serialization

Stdnet comes with a bunch of extendible utilities for serializing model data into different formats.

Get serializer

stdnet.odm.get_serializer(name, **options)

Retrieve a serializer register as name. If the serializer is not available a ValueError exception will raise. A common usage pattern:

qs = MyModel.objects.query().sort_by('id')
s = odm.get_serializer('json')
s.dump(qs)

Register serializer

stdnet.odm.register_serializer(name, serializer)

Register a new serializer to the library.

Parameters:

• name – serializer name (it can override existing serializers).
• serializer – an instance or a derived class of a stdnet.odm.Serializer class or a callable.

Serializer

class stdnet.odm.Serializer(**options)

The stdnet serializer base class. During initialization, the options dictionary is used to override the default_options. These are specific to each Serializer implementation.

default_options

Dictionary of default options which are overwritten during initialisation. By default it is an empty dictionary.

options

Dictionary of options.

data

CList of data to dump into a stream.

dump(qs)

Add a Query qs into the collection of data to dump into a stream. No writing is done until the write() method.

write(stream=None)

Write the serialized data into a stream. If stream is not provided, a python StringIO is used.

Returns:the stream object.

load(models, stream, model=None)

Load a stream of data into the database.

Parameters:

• models – the Router which must contains all the model this method will load.
• stream – bytes or an object with a read method returning bytes.
• model – Optional StdModel we need to load. If not provided all models in stream are loaded.

This method must be implemented by subclasses.

JsonSerializer

class stdnet.odm.JsonSerializer(**options)

The default Serializer of stdnet. It serialise/unserialise models into json data. It has one option given by the indent of the json string for pretty serialisation.

on_load_model(model, model_data)

Callback when a model is about to be loaded. If it returns the model, the model will get loaded otherwise it will skip the loading.

on_finished_load()

Callback when loading of data is finished

JSON utilities

JSONDateDecimalEncoder

class stdnet.utils.jsontools.JSONDateDecimalEncoder(skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

The default JSON encoder used by stdnet. It provides JSON serialization for four additional classes:

• datetime.date as a {'__date__': timestamp} dictionary
• datetime.datetime as a {'__datetime__': timestamp} dictionary
• decimal.Decimal as a {'__decimal__': number} dictionary

See also

It is the default encoder for stdnet.odm.JSONField

date_decimal_hook

stdnet.utils.jsontools.date_decimal_hook(dct)

The default JSON decoder hook. It is the inverse of stdnet.utils.jsontools.JSONDateDecimalEncoder.

flat_to_nested

stdnet.utils.jsontools.flat_to_nested(data, instance=None, attname=None, separator=None, loads=None)

Convert a flat representation of a dictionary to a nested representation. Fields in the flat representation are separated by the splitter parameters.

Parameters:

• data – a flat dictionary of key value pairs.
• instance – optional instance of a model.
• attribute – optional attribute of a model.
• separator – optional separator. Default "__".
• loads – optional data unserializer.

Return type:

a nested dictionary

addmul_number_dicts

stdnet.utils.jsontools.addmul_number_dicts(series)

Multiply dictionaries by a numeric values and add them together.

Parameters:

• series –

  a tuple of two elements tuples. Each serie is of the form:

  (weight,dictionary)
  

  where weight is a number and dictionary is a dictionary with numeric values.

• skip – optional list of field names to skip.

Only common fields are aggregated. If a field has a non-numeric value it is not included either.

Encoders

Classes used for encoding and decoding stdnet.odm.Field values.

class stdnet.utils.encoders.Encoder

Virtaul class for encoding data in data structures. It exposes two methods for encoding and decoding data to and from the data server.

type

The type of data once loaded into python

dumps(x)

Serialize data for database

loads(x)

Unserialize data from database

require_session()

True if this Encoder requires a stdnet.odm.Session.

load_iterable(iterable, session=None)

Load an iterable. By default it returns a generator of data loaded via the loads() method.

Parameters:

• iterable – an iterable over data to load.
• session – Optional stdnet.odm.Session.

Returns:

an iterable over decoded data.

These are all available Encoder:

class stdnet.utils.encoders.NoEncoder

A dummy encoder class

class stdnet.utils.encoders.Default(charset='utf-8', encoding_errors='strict')

The default unicode encoder. It converts bytes to unicode when loading data from the server. Viceversa when sending data.

class stdnet.utils.encoders.NumericDefault(charset='utf-8', encoding_errors='strict')

It decodes values into unicode unless they are numeric, in which case they are decoded as such.

class stdnet.utils.encoders.Bytes(charset='utf-8', encoding_errors='strict')

The binary encoder

class stdnet.utils.encoders.Json(charset='utf-8', encoding_errors='strict', json_encoder=None, object_hook=None)

A JSON encoder for maintaning python types when dealing with remote data structures.

class stdnet.utils.encoders.PythonPickle(protocol=2)

A safe pickle serializer. By default we use protocol 2 for compatibility between python 2 and python 3.

class stdnet.utils.encoders.DateTimeConverter

Convert to and from python datetime objects and unix timestamps

class stdnet.utils.encoders.DateConverter

Exceptions

class stdnet.StdNetException

A general StdNet exception

class stdnet.ImproperlyConfigured

A stdnet.StdNetException raised when stdnet is somehow improperly configured

class stdnet.QuerySetError

A stdnet.StdNetException raised by a stdnet.odm.Query.

class stdnet.FieldError

Generic Field error

class stdnet.FieldValueError

A stdnet.FieldError raised when passing a wrong value to a field. This exception is cought during the model instance validation algorithm in stdnet.odm.base.Metaclass.is_valid().

Signals

Stdnet includes a signal dispatcher which helps allow decoupled applications get notified when actions occur elsewhere in the framework. In a nutshell, signals allow certain senders to notify a set of receivers that some action has taken place. They are especially useful when many pieces of code may be interested in the same events.

The data mapper provide with the following built-in signals in the stdnet.odm module:

• pre_commit triggered before new instances or changes on existing instances are committed to the backend server.
• post_commit triggered after new instances or changes on existing instances are committed to the backend server.

It is also possible to add callback to single instances in the following way:

instance = MyModel(...)
instance.post_commit(callable)

Miscellaneous

Populate

stdnet.utils.populate(datatype='string', size=10, start=None, end=None, converter=None, choice_from=None, **kwargs)

Utility function for populating lists with random data. Useful for populating database with data for fuzzy testing. Supported data-types

• string

  For example:

  populate('string',100, min_len=3, max_len=10)
  

  create a 100 elements list with random strings with random length between 3 and 10

• date

  For example:

  from datetime import date
  populate('date',200, start = date(1997,1,1), end = date.today())
  

  create a 200 elements list with random datetime.date objects between start and end

• integer

  For example:

  populate('integer',200, start = 0, end = 1000)
  

  create a 200 elements list with random int between start and end

• float

  For example:

  populate('float', 200, start = 0, end = 10)
  

  create a 200 elements list with random floats between start and end

• choice (elements of an iterable)

  For example:

  populate('choice', 200, choice_from = ['pippo','pluto','blob'])
  

  create a 200 elements list with random elements from choice_from.

Testing

Test case classes and plugins for stdnet testing. Requires pulsar.

TestCase

class stdnet.utils.test.TestCase(methodName='runTest')

A unittest.TestCase subclass for testing stdnet with synchronous and asynchronous connections. It contains several class methods for testing in a parallel test suite.

multipledb

class attribute which indicates which backend can run the test. There are several options:

• multipledb = False The test case does not require a backend and only one TestCase class is added to the test-suite regardless of which backend has been tested.
• multipledb = True, the default falue. Create as many TestCase classes as the number of backend tested, each backend will run the tests.
• multipledb = string, list, tuple, Only those backend will run tests.

backend

A stdnet.BackendDataServer for this TestCase class. It is a class attribute which is different for each TestCase class and it is created by the setUpClass() method.

data_cls

A DataGenerator class for creating data. The data is created during the setUpClass() class method.

data

The DataGenerator instance created from data_cls.

model

The default StdModel for this test. A class attribute.

models

A tuple of models which can be registered by this test. The model is always the model at index 0 in models.

mapper

A stdnet.odm.Router with all models registered with backend.

data_cls

alias of DataGenerator

classmethod backend_params()

Optional backend parameters for tests in this TestCase class.

classmethod setUpClass()

Set up this TestCase before test methods are run. here is where a backend server instance is created and it is unique for this TestCase class. It create the mapper, a stdnet.odm.Router with all models registered. There shouldn’t be any reason to override this method, use after_setup() class method instead.

classmethod after_setup()

This class method can be used to setup this TestCase class after the setUpClass() was called. By default it does nothing.

classmethod session(**kwargs)

Create a new stdnet.odm.Session bind to the TestCase.backend attribute.

classmethod query(model=None)

Shortcut function to create a query for a model.

classmethod multi_async(iterable, **kwargs)

Treat iterable as a container of asynchronous results.

assertEqualId(instance, value, exact=False)

Assert the value of a primary key in a backend agnostic way.

Parameters:

• instance – the StdModel to check the primary key value.
• value – the value of the id to check against.
• exact – if True the exact value must be matched. For redis backend this parameter is not used.

DataGenerator

class stdnet.utils.test.DataGenerator(size, sizes=None)

A generator of data. It must be initialised with the size parameter obtained from the command line which is avaiable as a class attribute in TestCase.

sizes

A dictionary of sizes for this generator. It is a class attribute with the following entries: tiny, small, normal, big and huge.

size

The actual size of the data to be generated. Obtained from the sizes and the input size code during initialisation.

generate()

Called during initialisation to generate the data. kwargs are additional key-valued parameter passed during initialisation. Must be implemented by subclasses.

populate(datatype='string', size=None, **kwargs)

A shortcut for the stdnet.utils.populate() function. If size is not given, the size is used.

random_string(min_len=5, max_len=30)

Return a random string