Full text search

Full text search in most key-value stores is not provided out of the box as it is in traditional relational databases. One may argue that no-sql databases don’t need to provide such feature since search engines such as Solr, ElasticSearch or Sphinx can be used to provide a full text search solution.

Stdnet provides the stdnet.odm.SearchEngine interface for implementing full text search of stdnet models. The interface can be customized with third party applications.

Redis based solution

Stdnet provides a redis-based implementation for the stdnet.odm.SearchEngine so that you can have your models stored and indexed in redis and if you like in the same redis instance.

Installing the search engine is as easy as

• Create a search engine and use the stdnet.odm.Router.set_search_engine() method to install it in the router. Here, we use the models router of our first tutorial:

  from stdnet.apps.searchengine import SearchEngine
  
  models.set_search_engine(SearchEngine(backend, ...))
  

  where backend is either a instance of a stdnet.BackendDataServer or a valid connection string such as:

  redis://127.0.0.1:6379?db=4

  or None, to use the default stdnet.odm.Router.backend. This is the back-end server where the text indices WordItem are stored, and not the back-end server of your models to index. They can be the same. To use the same backend as the default backend of the router:

  models.search_engine = SearchEngine(models.backend, ...)
  

• Register models you want to index to the search engine signletone:

  models.search_engine.register(Instrument)
  

  Check the stdnet.odm.SearchEngine.register() documentation for more information.

Searching model instances for text can be achieved using the stdnet.odm.Query.search method:

models.instrument.search('bla foo...')

Like all stdnet.odm.Query methods, the search method can be chained in an efficient way:

models.instrument.filter(ccy='EUR').search('bla foo...')

API

SearchEngine

class stdnet.apps.searchengine.SearchEngine(backend=None, min_word_length=3, stop_words=None, metaphone=True, stemming=True, splitters=None, **kwargs)

A python implementation for the stdnet.odm.SearchEngine driver.

Parameters:

• min_word_length –

  minimum number of words required by the engine to work.

  Default 3.

• stop_words –

  list of words not included in the search engine.

  Default stdnet.apps.searchengine.ignore.STOP_WORDS

• metaphone –

  If True the double metaphone algorithm will be used to store and search for words. The metaphone should be the last world middleware to be added.

  Default True.

• splitters –

  string whose characters are used to split text into words. If this parameter is set to “_-“, for example, than the word bla_pippo_ciao-moon will be split into bla, pippo, ciao and moon. Set to empty string for no splitting. Splitting will always occur on white spaces.

  Default stdnet.apps.searchengine.ignore.PUNCTUATION_CHARS.

search_model(q, text, lookup=None)

Implements stdnet.odm.SearchEngine.search_model(). It return a new stdnet.odm.QueryElem instance from the input Query and the text to search.

WordItem

class stdnet.apps.searchengine.WordItem(*args, **kwargs)

A model for associating a word with general stdnet.odm.StdModel instance.

object(session)

Instance of model_type with id object_id.