Table Database — TDB

New in version 0.4.0.

TDB

class tokyocabinet.TDB

Example:

from tokyocabinet import *

tdb = TDB()

# if need be, you should call tune/setcache/setxmsiz/setdfunit before
# open, ex. with default values:
tdb.tune(0, -1, -1, 0)
tdb.setcache(0, 0, 0)
tdb.setxmsiz(0)
tdb.setdfunit(0)

# open the database
tdb.open("casket.tct", TDBOWRITER | TDBOCREAT)

# store records
for key, value in [
                   ("foo", {"age": "30", "name": "John", "sex": "m"}),
                   ("bar", {"age": "56", "name": "Paul", "sex": "m"}),
                   ("baz", {"age": "22", "name": "Ella", "sex": "f"})
                  ]:
    tdb[key] = value

# retrieve one record
print(tdb["foo"])

# traverse records
for key in tdb:
    print(key, tdb[key])

# close the database
tdb.close()

Note

For all methods taking either a key argument or a pair (key, value), key must be either str (Python2) or bytes (Python3) and value must be a dict.

All items in value must be pairs of str (Python2) or bytes (Python3). Empty keys in value are not allowed.

len(tdb)
Return the number of records in the database tdb.
tdb[key]
Return the value of tdb‘s record corresponding to key. Raises KeyError if key is not in the database.
tdb[key] = value
Set tdb[key] to value.
del tdb[key]
Remove tdb[key] from tdb. Raises KeyError if key is not in the database.
key in tdb
Return True if tdb has a key key, else False.
key not in tdb
Equivalent to not key in tdb.
iter(tdb)
Return an iterator over the keys of the database.
tune(bnum, apow, fpow, opts)

Tune a database.

Parameters:
  • bnum – the number of elements in a bucket array. If specified as 0 or as a negative value, the default value (131071) is used.
  • apow – (power of 2) TODO. If specified as a negative value, the default value (4) is used (means: 2**4).
  • fpow – (power of 2) TODO. If specified as a negative value, the default value (10) is used (means 2**10).
  • opts – options, see TDB.tune()/TDB.optimize() options.

Note

Tuning an open database is an invalid operation.

setcache(rcnum, lcnum, ncnum)

Set the cache size.

Parameters:
  • rcnum – the maximum number of records to be cached. If specified as 0 or as a negative value, caching is disabled (default).
  • lcnum – the maximum number of leaf nodes to be cached. If specified as 0 or as a negative value, the default value (4096) is used (for indexes).
  • ncnum – the maximum number of non-leaf nodes to be cached. If specified as 0 or as a negative value, the default value (512) is used (for indexes).

Note

Setting the cache size on an open database is an invalid operation.

setxmsiz(xmsiz)

Set the extra mapped memory size.

Parameter:xmsiz – the amount of extra mapped memory (in what unit?). If specified as 0 or as a negative value, the extra mapped memory is disabled. Default is 67108864 (unit?).

Note

Setting the extra memory size on an open database is an invalid operation.

setdfunit(dfunit)

Set auto defragmentation’s unit step number.

Parameter:dfunit – the unit step number(?). If specified as 0 or as a negative value, auto defragmentation is disabled (default).

Note

Setting this on an open database is an invalid operation.

open(path, mode)

Open a database.

Parameters:
close()

Close the database.

Note

TDBs are closed when garbage-collected.

clear()
Remove all records from the database.
copy(path)

Copy the database file.

Parameter:path – path to the destination file.
begin()
Begin a transaction.
commit()
Commit a transaction.
abort()
Abort a transaction.
get(key)
Return the value corresponding to key. Equivalent to tdb[key].
remove(key)
Delete a record from the database. Equivalent to del tdb[key].
put(key, value)
Store a record in the database. Equivalent to tdb[key] = value.
putkeep(key, value)
Store a record in the database, unlike the standard forms (tdb[key] = value or put()), this method raises KeyError if key is already in the database.
putcat(key, value)
Merge a value with an existing one, does not override existing items in current value. If there is no corresponding record, a new record is stored.
sync()
Flush modifications to the database file.
searchkeys(prefix[, max])
Return a frozenset of keys starting with prefix. If given, max is the maximum number of keys to fetch, if omitted or specified as a negative value no limit is applied.
setindex(column, type)

Add an index to a column.

Parameters:
  • column (str (Python2)/bytes (Python3)) – name of the column. An empty string means key.
  • type – index type, see TDB.setindex() types.
uid()
Return a new unique id.
query()
Return a query object (TDBQuery). See Querying a Table Database — TDBQuery.
static metasearch(queries, type)

Combine queries and return the result set as a tuple of keys.

Parameters:
optimize([bnum=0[, apow=-1[, fpow=-1[, opts=255]]]])

Optimize a database.

Parameters:
  • bnum – the number of elements in a bucket array. If specified as 0 or as a negative value, the default value (twice the number of records) is used.
  • apow – (power of 2) TODO. If specified as a negative value, the current setting is kept.
  • fpow – (power of 2) TODO. If specified as a negative value, the current setting is kept.
  • opts – options, see TDB.tune()/TDB.optimize() options. If specified as 255 (UINT8_MAX), the current setting is kept.

Note

Optimizing a read only database, or during a transaction, is an invalid operation.

Methods keys(), values() and items() are not yet implemented (mainly because I didn’t settle on how to do it: should they return Iterable, Iterator, MappingView, etc.?). Any help would be greatly appreciated in this matter.

For the time being, for those of you who really need these methods, it’s trivial to implement them in python. Here is an example using generators:

from tokyocabinet import TDB as _TDB

class TDB(_TDB):

    def keys(self):
        return (key for key in self)

    def values(self):
        return (self[key] for key in self)

    def items(self):
        return ((key, self[key]) for key in self)
path
The path to the database file.
size
The size in bytes of the database file.

TDB.open() modes

tokyocabinet.TDBOREADER
Open a database in read-only mode.
tokyocabinet.TDBOWRITER
Open a database in read-write mode.

The following constants can only be combined with TDBOWRITER :

  • tokyocabinet.TDBOCREAT

    Create a new database file if it does not exists.

  • tokyocabinet.TDBOTRUNC

    Create a new database file even if one already exists (truncates existing file).

  • tokyocabinet.TDBOTSYNC

    Sync the database file on every transaction.

The following constants can be combined with either TDBOREADER or TDBOWRITER :

  • tokyocabinet.TDBONOLCK

    Opens the database file without file locking.

  • tokyocabinet.TDBOLCKNB

    Locking is performed without blocking.

TDB.tune()/TDB.optimize() options

tokyocabinet.TDBTLARGE
The size of the database can be larger than 2GB.
tokyocabinet.TDBTDEFLATE
Each record is compressed with Deflate encoding.
tokyocabinet.TDBTBZIP
Each record is compressed with BZIP2 encoding.
tokyocabinet.TDBTTCBS
Each record is compressed with TCBS encoding.

TDB.setindex() types

tokyocabinet.TDBITLEXICAL
String index.
tokyocabinet.TDBITDECIMAL
Numeric index.
tokyocabinet.TDBITTOKEN
Token index.
tokyocabinet.TDBITQGRAM
Q-gram index.
tokyocabinet.TDBITOPT
Optimize the index.
tokyocabinet.TDBITVOID
Remove the index.
tokyocabinet.TDBITKEEP
Fail if the index already exists.

TDB.metasearch() types

tokyocabinet.TDBMSUNION
Union.
tokyocabinet.TDBMSISECT
Intersection.
tokyocabinet.TDBMSDIFF
Difference.

Querying a Table Database — TDBQuery

class tokyocabinet.TDBQuery

When first returned by TDB.query() a query result set potentially includes all keys contained in the database. You can narrow down a search by calling filter() and limit the number of results with limit().

filter(column, condition, expr)

Filter the result set on the condition expressed by the parameters.

Parameters:
  • column (str (Python2)/bytes (Python3)) – name of the column. An empty string means key.
  • condition – see TDBQuery.filter() conditions.
  • expr – expression.

Note

Calling filter() multiple times with different conditions is equivalent to applying a logical AND between the conditions.

sort(column, type)

Sort the result set.

Parameters:
  • column (str (Python2)/bytes (Python3)) – name of the column. An empty string means key.
  • type – sort type (and direction), see TDBQuery.sort() types.
limit([max[, skip]])

Limit the number of keys in the result set.

Parameters:
  • max – the maximum number of keys to return. If specified as a negative value no limit is applied.
  • skip – the number of keys to skip (from the beginning). If specified as a 0 or as a negative value no skipping is done.
search()
Execute the query and return the result set as a tuple of keys.
remove()
Remove all records corresponding to the result set from the database.
hint()
TODO.
process(callback)

Apply callback to each record in the result set.

Parameter:callbackcallback must be a callable that accept a pair key, value as its arguments. callback can return one, or a combination, of these constants to trigger post-processing or to stop iterating.

TDBQuery.filter() conditions

value refers to tdb[key][column] if column is not an empty string otherwise value is key (that will need a better description).

String conditions :

tokyocabinet.TDBQCSTREQ
value == expr
tokyocabinet.TDBQCSTRINC
expr in value #substring test
tokyocabinet.TDBQCSTRBW
value.startswith(expr)
tokyocabinet.TDBQCSTREW
value.endswith(expr)
tokyocabinet.TDBQCSTRAND

if expr is expressed as "expr1,expr2":

expr1 in value and expr2 in value
tokyocabinet.TDBQCSTROR

if expr is expressed as "expr1,expr2":

expr1 in value or expr2 in value
tokyocabinet.TDBQCSTROREQ

if expr is expressed as "expr1,expr2":

value == expr1 or value == expr2
tokyocabinet.TDBQCSTRRX
expr is a regular expression.

Numeric conditions :

tokyocabinet.TDBQCNUMEQ
value == expr
tokyocabinet.TDBQCNUMGT
value > expr
tokyocabinet.TDBQCNUMGE
value >= expr
tokyocabinet.TDBQCNUMLT
value < expr
tokyocabinet.TDBQCNUMLE
value <= expr
tokyocabinet.TDBQCNUMBT

if expr is expressed as "expr1,expr2":

value >= min(expr1, expr2) and value <= max(expr1, expr2)
tokyocabinet.TDBQCNUMOREQ

if expr is expressed as "expr1,expr2":

value == expr1 or value == expr2

Full-text search :

tokyocabinet.TDBQCFTSPH
TODO.
tokyocabinet.TDBQCFTSAND
TODO.
tokyocabinet.TDBQCFTSOR
TODO.
tokyocabinet.TDBQCFTSEX
TODO.

All conditions above can be combined with the following :

tokyocabinet.TDBQCNEGATE
Negate the condition.
tokyocabinet.TDBQCNOIDX
Do not use the column index (if one has been set).

TDBQuery.sort() types

tokyocabinet.TDBQOSTRASC
Alphabetical, ascending.
tokyocabinet.TDBQOSTRDESC
Alphabetical, descending.
tokyocabinet.TDBQONUMASC
Numerical, ascending.
tokyocabinet.TDBQONUMDESC
Numerical, descending.

TDBQuery.process() post-processing constants

tokyocabinet.TDBQPPUT
Store a modified value.
tokyocabinet.TDBQPOUT
Remove the record.
tokyocabinet.TDBQPSTOP
Stop iterating over the records.