Whoosh lets you index and search dates/times using the whoosh.fields.DATETIME field type. Instead of passing text for the field in add_document(), you use a Python datetime.datetime object:
from datetime import datetime, timedelta from whoosh import fields, index schema = fields.Schema(title=fields.TEXT, content=fields.TEXT, date=fields.DATETIME) ix = index.create_in("indexdir", schema) w = ix.writer() w.add_document(title="Document 1", content="Rendering images from the command line", date=datetime.utcnow()) w.add_document(title="Document 2", content="Creating shaders using a node network", date=datetime.utcnow() + timedelta(days=1)) w.commit()
Once you’ve have an indexed DATETIME field, you can search it using a rich date parser contained in the whoosh.qparser.dateparse.DateParserPlugin:
from whoosh import index from whoosh.qparser import QueryParser from whoosh.qparser.dateparse import DateParserPlugin ix = index.open_dir("indexdir") # Instatiate a query parser qp = QueryParser("content", ix.schema) # Add the DateParserPlugin to the parser qp.add_plugin(DateParserPlugin())
With the DateParserPlugin, users can use date queries such as:
20050912 2005 sept 12th june 23 1978 23 mar 2005 july 1985 sep 12 today yesterday tomorrow now next friday last tuesday 5am 10:25:54 23:12 8 PM 4:46 am oct 31 2010 last tuesday to today today to next friday jan 2005 to feb 2008 -1 week to now now to +2h -1y6mo to +2 yrs 23d
Normally, as with other types of queries containing spaces, the users needs to quote date queries containing spaces using single quotes:
render date:'last tuesday' command date:['last tuesday' to 'next friday']
If you use the free argument to the DateParserPlugin, the plugin will try to parse dates from unquoted text following a date field prefix
This allows the user to type a date query with spaces and special characters following the name of date filed and a colon. The date query can be mixed with other types of queries without quotes:
date:last tuesday render date:oct 15th 2001 5:20am command
If you don’t use the DateParserPlugin, users can still search DATETIME fields using a simple numeric form YYYY[MM[DD[hh[mm[ss]]]]] that is built into the DATETIME field:
from whoosh import index from whoosh.qparser import QueryParser ix = index.open_dir("indexdir") qp = QueryParser("content", schema=ix.schema) # Find all datetimes in 2005 q = qp.parse(u"date:2005") # Find all datetimes on June 24, 2005 q = qp.parse(u"date:20050624") # Find all datetimes from 1am-2am on June 24, 2005 q = qp.parse(u"date:2005062401") # Find all datetimes from Jan 1, 2005 to June 2, 2010 q = qp.parse(u"date:[20050101 to 20100602]")
The best way to deal with time zones is to always index datetime``s in naive UTC form. Any ``tzinfo attribute on the datetime object is _ignored_ by the indexer. If you are working with local datetimes, you should convert them to naive UTC datetimes before indexing.
Please note that the date parser is still somewhat experimental.
When you create the DateParserPlugin you can pass a datetime object to the basedate argument to set the datetime against which relative queries (such as last tuesday and -2 hours) are measured. By default, the basedate is datetime.utcnow() at the moment the plugin is instantiated:
To avoid user queries causing exceptions in your application, the date parser attempts to fail silently when it can’t parse a date query. However, you can register a callback function to be notified of parsing failures so you can display feedback to the user. The argument to the callback function is the date text that could not be parsed (this is an experimental feature and may change in future versions):
errors =  def add_error(msg): errors.append(msg) qp.add_plugin(DateParserPlug(callback=add_error)) q = qp.parse(u"date:blarg") # errors == [u"blarg"]
While the free option is easier for users, it may result in ambiguities. As one example, if you want to find documents containing reference to a march and the number 2 in documents from the year 2005, you might type:
date:2005 march 2
This query would be interpreted correctly as a date query and two term queries when free=False, but as a single date query when free=True. In this case the user could limit the scope of the date parser with single quotes:
date:'2005' march 2
The date parser supports a wide array of date and time formats, however it is not my intention to try to support all types of human-readable dates (for example ten to five the friday after next). The best idea might be to pick a date format that works and try to train users on it, and if they use one of the other formats that also works consider it a happy accident.