Shorten is based around a dict-like object called a store. Stores allow insertion, deletion and lookup of data through an interface similar to a Python dict, the primary difference being that store keys are generated automatically by a key generator (or keygen).

The corresponding usage between a Python dict and store is illustrated below:

store dict
key, token = store.insert(val) dict[key] = val
del store[token] del dict[key]
key in store key in dict
for key in iter(store) [1] for key in iter(dict)
store.has_token(token) no equivalent
store.get_token(key) no equivalent
len(store) [1] len(dict)
[1](1, 2) This functionality may be missing from some stores due to inefficiency or thread safety.

When data is inserted into a store a Pair is returned. Pairs are tuples containing key and token attributes. Keys are used to lookup data, while tokens are used to delete data. Tokens are generated from token generators, similar to how keys are generated from key generators.

Making a Store

There are three built-in store types:

store notes
shorten.MemoryStore native Python.
shorten.MemcacheStore requires memcached or Memcached-compatible server.
shorten.RedisStore requires a Redis server.

All stores are available at the module level. The only required 3rd-party library is redis-py, needed for WatchError. This dependency may be removed at a later date.

from shorten import MemoryStore, MemcacheStore, RedisStore

Redis stores require a Redis client (redis is assumed) and the name of a key that the keygen will use as a counter.

import redis
from shorten import RedisStore

client = redis.Redis()
store = RedisStore(redis_client=client,

Memcache stores require a Memcached client (pylibmc is recommended, although it requires the C development libraries to compile). The name of the key that will be used as a counter is also required.

import pylibmc
from shorten import MemcacheStore

client = pylibmc.Client([""], binary=True,
                  behaviors={"tcp_nodelay": True, "ketama": True})

store = MemcacheStore(memcache_client=client,

A MemoryStore does not have any dependencies, so it will be used for the remaining examples.

Inserting Values

Use insert() to insert values. A Pair is returned, which can destructured like a tuple.

from shorten import MemoryStore

store = MemoryStore()
key, token = store.insert('aardvark')

# 'aardvark'

# True
key in store

pair = store.insert('bonobo')

# Pair(key='0', token='0')

Revoking Keys

After insertion, keys can be revoked, which will remove the key and its value from the store.

from shorten import MemoryStore

store = MemoryStore()
key, token = store.insert('aardvark')

del store[token]

# False
key in store

Customizing Key Generation

A store’s keys are generated with a KeyGenerator. The default key generators use a counter to increment keys, then convert that number to a string in an alphabet.

Randomized Alphabets

All the examples return keys that are clearly sequential and so you may decide to shuffle the alphabet to produce keys that appear random.

Although they appear random, they’re not: the alphabet order can easily be reconstructed from frequency counting and Benford’s law, allowing someone to predict all future keys.

Never use short URLs to hide your data - use UUIDs or authentication instead.

Alphabets can be anything that is indexable, as long as each symbol in the alphabet is not contained within any other symbol. For instance, ('00', '0', '1') would be an ambiguous alphabet, since 00 could be interpreted as either the symbol 00 or two 0 symbols.

Keys of a minimum length or starting at a certain unencoded value can be generated by specifying min_length or start.

For example, hex keys can be generated:

from shorten import MemoryStore

hexabet = '0123456789abcdef'
store = MemoryStore(alphabet=hexabet, min_length='2')

# '10'
# '11'
# '12'
for i in range(0, 2):
   pair = store.insert('aardvark')

and more exotic alphabets can be constructed as well:

from shorten import MemoryStore

emoticons = (':)', ':(', ':D', ';)', ';(', 'D:', ':o', ':/')
emote_store = MemoryStore(alphabet=emoticons, start=12)

key, token = emote_store.insert('aardvark')

# ':(:D'

Customizing Token Generation

A token generator can be any object with a create_token(key) method. Shorten has provides token generator classes:

See A Mock Google Token Generator for a more comprehensive example.


A Formatter is used to format the internal representation of a key or token. This is useful for Redis and SQL databases, which often need to prefix keys and columns in order to avoid clashes.

Any class or mixin with methods format_token(token) and format_key(key) can be used.

There are two formatters included in Shorten:

  • Formatter (the default) and
  • NamespacedFormatter, which takes a string namespace and prefixes keys and tokens with '{namespace}:keys' and '{namespace}:tokens' respectively.

A Redis Example

A NamespacedFormatter can keep keys and tokens from overwriting each other and unrelated data in Redis:

import redis
from shorten import RedisStore, NamespacedFormatter

formatter = NamespacedFormatter('testing')

store = RedisStore(redis_client=redis,
   counter_key = 'testing:counter_key',

If a value is inserted, the stored key will be 'testing:keys:0' in Redis, but the returned key will be '0'

key, token = store.insert('aardvark')

# '0'

# True

Table Of Contents

Related Topics

This Page

Fork me on GitHub