django-memoize is an implementation of memoization technique for Django. You can think of it as a cache for function or method results.
Install the extension with one of the following commands:
$ easy_install django-memoize
or alternatively if you have pip installed:
$ pip install django-memoize
Add ‘memoize’ to your INSTALLED_APPS in settings.py:
INSTALLED_APPS = [ '...', 'memoize', ]
Memoization is managed through a
from memoize import Memoizer memoizer = Memoizer()
Hovewer, we recommend to use already defined instance of
use its methods:
from memoize import memoize, delete_memoized, delete_memoized_verhash @memoize(timeout=60) def count_objects(): pass delete_memoized(count_objects)
In memoization, the functions arguments are also included into the cache_key.
Memoize is also designed for methods, since it will take into account the repr of the ‘self’ or ‘cls’ argument as part of the cache key.
The theory behind memoization is that if you have a function you need to call several times in one request, it would only be calculated the first time that function is called with those arguments. For example, a model object that determines if a user has a role. You might need to call this function many times during a single request. To keep from hitting the database every time this information is needed you might do something like the following:
class Person(models.Model): @memoize(timeout=50) def has_membership(self, role_id): return Group.objects.filter(user=self, role_id=role_id).count() >= 1
Using mutable objects (classes, etc) as part of the cache key can become tricky. It is suggested to not pass in an object instance into a memoized function. However, the memoize does perform a repr() on the passed in arguments so that if the object has a __repr__ function that returns a uniquely identifying string for that object, that will be used as part of the cache key.
For example, a model person object that returns the database id as part of the unique identifier.:
class Person(models.Model): def __repr__(self): return "%s(%s)" % (self.__class__.__name__, self.id)
Deleting memoize cache¶
You might need to delete the cache on a per-function bases. Using the above
example, lets say you change the users permissions and assign them to a role,
but now you need to re-calculate if they have certain memberships or not.
You can do this with the
If only the function name is given as parameter, all the memoized versions
of it will be invalidated. However, you can delete specific cache by
providing the same parameter values as when caching. In following
example only the
user-role cache is deleted:
user_has_membership('demo', 'admin') user_has_membership('demo', 'user') delete_memoized('user_has_membership', 'demo', 'user')
Memoizer(cache=<django.core.cache.backends.locmem.LocMemCache object>, cache_prefix='memoize')¶
This class is used to control the memoizer objects.
memoize(timeout=300, make_name=None, unless=None)¶
Use this to cache the result of a function, taking its arguments into account in the cache key.
Information on Memoization.
@memoize(timeout=50) def big_foo(a, b): return a + b + random.randrange(0, 1000)
>>> big_foo(5, 2) 753 >>> big_foo(5, 3) 234 >>> big_foo(5, 2) 753
The returned decorated function now has three function attributes assigned to it.
- The original undecorated function. readable only
The cache timeout value for this function. For a custom value to take affect, this must be set before the function is called.
readable and writable
A function used in generating the cache_key used.
readable and writable
- timeout – Default: 300. If set to an integer, will cache for that amount of time. Unit of time is in seconds.
- make_name – Default None. If set this is a function that accepts a single argument, the function name, and returns a new string to be used as the function name. If not set then the function name is used.
- unless – Default None. Cache will always execute the caching facilities unelss this callable is true. This will bypass the caching entirely.
delete_memoized(f, *args, **kwargs)¶
Deletes the specified functions caches, based by given parameters. If parameters are given, only the functions that were memoized with them will be erased. Otherwise all versions of the caches will be forgotten.
@memoize(50) def random_func(): return random.randrange(1, 50) @memoize() def param_func(a, b): return a+b+random.randrange(1, 50)
>>> random_func() 43 >>> random_func() 43 >>> delete_memoized('random_func') >>> random_func() 16 >>> param_func(1, 2) 32 >>> param_func(1, 2) 32 >>> param_func(2, 2) 47 >>> delete_memoized('param_func', 1, 2) >>> param_func(1, 2) 13 >>> param_func(2, 2) 47
Delete memoized is also smart about instance methods vs class methods.
When passing a instancemethod, it will only clear the cache related to that instance of that object. (object uniqueness can be overridden by defining the __repr__ method, such as user id).
When passing a classmethod, it will clear all caches related across all instances of that class.
class Adder(object): @memoize() def add(self, b): return b + random.random()
>>> adder1 = Adder() >>> adder2 = Adder() >>> adder1.add(3) 3.23214234 >>> adder2.add(3) 3.60898509 >>> delete_memoized(adder.add) >>> adder1.add(3) 3.01348673 >>> adder2.add(3) 3.60898509 >>> delete_memoized(Adder.add) >>> adder1.add(3) 3.53235667 >>> adder2.add(3) 3.72341788
- fname – Name of the memoized function, or a reference to the function.
- *args – A list of positional parameters used with memoized function.
- **kwargs – A dict of named parameters used with memoized function.
django-memoize uses inspect to order kwargs into positional args when the function is memoized. If you pass a function reference into
fnameinstead of the function name, django-memoize will be able to place the args/kwargs in the proper order, and delete the positional cache.
delete_memoizedis just called with the name of the function, be sure to pass in potential arguments in the same order as defined in your function as args only, otherwise django-memoize will not be able to compute the same cache key.
django-memoize maintains an internal random version hash for the function. Using delete_memoized will only swap out the version hash, causing the memoize function to recompute results and put them into another key.
This leaves any computed caches for this memoized function within the caching backend.
It is recommended to use a very high timeout with memoize if using this function, so that when the version has is swapped, the old cached results would eventually be reclaimed by the caching backend.
Delete the version hash associated with the function.
Performing this operation could leave keys behind that have been created with this version hash. It is up to the application to make sure that all keys that may have been created with this version hash at least have timeouts so they will not sit orphaned in the cache backend.
- Fixed a serious issue where memoize function ignored special f(arg, **kwargs) input use-case. See test test_10ab_arg_kwarg_memoize for more details.
- Added new Django==1.8 Travis env test
- Memoizer now accepts cache_prefix and cache parameter
- Support for Django >= 1.4
- Default timeout set to Django’s cache timeout
- Port to Python >= 3.3 (requiring Python 2.6/2.7 for 2.x).
- Fixed bug with using per-memoize timeouts greater than the default timeout
- Added better support for per-instance memoization.
- Various bug fixes
- Initial public release