Performance

We dig deeper into our example by exploring additional features of the API, including the manipulation of related models and transactions.

Transactions

Under the hood, stdnet performs server updates and queries via a Session. You can write your application without using a session directly, and in several cases this is good enough. However, when dealing with lots of operations, you may be better off using Transaction. A transaction is started with the Session.begin() method and concluded with the Session.commit() method. A session for registered models can be obtained from the model manager. For example, using the Fund model in tutorial 1:

session = Fund.objects.session()
session.begin() # start a new transaction
session.add(Fund(name='Markowitz', ccy='EUR'))
session.add(Fund(name='SterlingFund', ccy='GBP'))
session.commit() # commit changes to the server

Transactions are pivotal for two reasons:

• They guarantee atomicity and therefore consistency of model instances when updating/deleting.
• They speed up updating, deleting and retrieval of several independent block of data at once.

For certain type of operations, the use of transactions becomes almost compulsory as the speed up achived can be of 2 to 3 order of magnitude.

This snippet demonstrates how to speed up the creation of several instances of model Fund using a with statement:

with Fund.objects.transaction() as t:
    for kwargs in data:
        t.add(Fund(**kwargs))

Or for more than one model:

with Fund.objects.transaction(Instrument) as t:
    for kwargs in data1:
        t.add(Fund(**kwargs))
    for kwargs in data2:
        t.add(Instrument(**kwargs))
    ...

As soon as the with statement finishes, the transaction commit changes to the server via the commit() method.

Use load_only

One of the main advantages of using key-values databases as opposed to traditional relational databases, is the ability to add or remove Field without requiring database migration. In addition, the JSONField can be a factory of fields for a given model (when used with the JSONField.as_string set to False). For complex models, Field can also be used as cache.

In these situations, your model may contain a lot of fields, some of which could contain a lot of data (for example, text fields), or require expensive processing to convert them to Python objects. If you are using the results of a Query in a situation where you know you don’t need those particular fields, you can tell stdnet to load a subset of fields from the database by using the Query.load_only() or Query.dont_load() methods.

For example I need to load all my EUR Funds from the example application but I don’t need to see the description and ccy:

qs = Fund.objects.filter(ccy="EUR").load_only('name')

Importantly, the load_only method can also be applied to related objects fields. For example if I need to load Positions from example application and only the currency field is required from the instrument field one could issue the command:

qs = Position.objects.query().load_only('instrument__ccy')

This is equivalent to the use of Query.load_related():

qs = Position.objects.query().load_related('instrument', 'ccy')

Use dont_load

Opposite of load_only, it can be used to avoid loading a subsets of fields:

qs = Fund.objects.filter(ccy="EUR").dont_load('description', 'ccy')

Use load_related

The Query.load_related() method is another performance boost tool when used in the right circumstances. It is similar to the eager loading in SqlAlchemy and the select_related in Django. The method returns a new Query() that automatically load the ForeignKey field at the same time as the parent object, within a single back-end roundtrip.

This is a performance booster which results in (sometimes much) larger queries but means later use of foreign-key relationships won’t require database queries.

Let’s consider the Position model in the tutorial application and lets assume we need to access the instrument and fund foreign keys:

positions = Position.objects.query()
for p in positions:
    # requires one database roundtrip
    i = p.instrument
    # requires another database roundtrip
    f = p.fund

Since foreign key are not loaded by default, when accessing the field for the first time, stdnet needs to load it from the backend-server. Check one to many relationship tutorial for more information on lazy loading.

The above example can be drammatically speeded up by modifying the query in the following way:

positions = Position.objects.query().load_related('instrument')\
                                    .load_related('fund')
for p in positions:
    # No database roundtrip
    i = p.instrument
    # No database roundtrip
    f = p.fund

Get single fields

It is possible to obtain only the values of a given field. If I need to obtain all the Funds names from the example application I could issue the following command:

names = Fund.objects.query().get_field('name')

The Q.get_field() method returns a new query which evaluates to a list of field values.