The filters Module

functions useful for extracting moments from a group of log files based on the tags we want to extract

class moments.filters.ExtractConfig

used in /c/mindstream/moments-scripts/extract.py

moments.filters.difference(set1, set2)

take two lists return the differences

NOTE THAT ORDER MATTERS! if 1 is a superset of 2, there will be a difference but if 2 is a superset of 1, there won’t be a difference

i.e. larger set first! larger = set1, smaller = set2

moments.filters.difference_journals(journal, other)

take another journal, return a new journal with the entries that are only in the other journal not in ourself

NOTE: we may have entries in journal that are not in other... those will not be returned. this can be called in the opposite direction if those are wanted

moments.filters.extract(journal, tag_list, etype='intersect')

remove any entries tagged with a/all tag in tag_list

saving journal (with missing entries) left to caller (if desired)

moments.filters.extract_many(path, extractions, ignores=[], save=False, extract_type='intersect')

rather than go through all files for every extraction it is nice to go through all extractions during each file

save extractions for each file rather than accumulating until end

will make it trickier for trial runs can print actions or make temp logs

moments.filters.extract_tags(path, extractions=[], ignores=[], save=False, extract_type='intersect')

accept a list of extractions where each extraction consists of a set of tags to look for (using extract_type) and a destination where matching entries should be extracted to

ignores is a list of tags to leave out of the found entries (good for filtering tags generated from the original file path)

this duplicates the logic for scanning all files from extract_tag it feels more readable to separate the two

*2009.08.29 13:20:46 now part of the moments module itself

not to be confused with the Journal.extract method these functions are higher level operations that utilize Journal.extract

also: # take a list of tags, # and the directory or file that you want to use as the source of the tags # go through all files, and remove those tags # saving them in a new separate file (or specified existing file)

# adapted from pose.controllers.tags.extract

moments.filters.filter_entries(journal, updates)

apply filters to the journal’s entries in place #normalize/filter all of the data first... #as the system changes, so do the paths

adapted from medialist.from_journal

could consider adding this as a method to entry or data

moments.filters.filter_list(items, ignores, search=False)

IF SEARCH IS NOT TRUE, IGNORES MUST BE AN EXACT MATCH

take a list of items take a list of itmes to ignore return a new list of items based on first, original, with ignores removed

moments.filters.filter_log(path, filters, save=False)

moments.filters.filter_logs(path, updates=[], save=False)

walk the path, looking for moment logs for each log scan entries for each entry apply filter

moments.filters.find_and_replace(items, updates)

aka find_and_replace

apply all updates in updates list to all items in items list

updates consist of a list of lists where the sub lists contain: (search_string, replace_string)

moments.filters.flatten(journal, filename=None)

create a flat version of the journal without timestamps (or tags?)

moments.filters.flatten_first_lines(journal, separator=' ')

only keep the first line of each journal entry useful for playlists

moments.filters.intersect(set1, set2)

take two lists intersect the items in them

moments.filters.intersect_journal_entries_with_tags(journal, tag_list)

look for all entries in the journal that have all tags in tag_list

moments.filters.intersect_journals(journal, other)

take another journal, return a new journal with only the entries that are in common to both

moments.filters.main()

*2011.08.30 09:09:19 imported from filter_logs that might not be the main functionality of this module

moments.filters.omit_date_tags(items)

take a list of tags leave out any that are a timestamp

moments.filters.remove_dupes(items)

make sure no item appears twice in list i.e. filter for any duplicates

class moments.filters.unaccented_map

Maps a unicode character code (the key) to a replacement code (either a character code or a unicode string).

moments.filters.union(set1, set2)

take two lists union the items in them

moments.filters.union_journal_entries_with_tags(journal, tag_list)

look for all entries in the journal that have any tags in tag_list

moments.filters.union_journals(journal, other)

take another journal combine all entries in it and journal

similar to merge_logs (to be consistent with intersect and difference behavior: return a new Journal with those entries)

this is also what from_entries does