Base Lenses

These lenses form the core of pylens. The Lens class is the base of all other lenses and encapsulates (and hides) much of the complexity of the framework whilst allowed us to extend it.

class pylens.base_lenses.Lens(type=None, name=None, default=None, **options)

Base lens, which all other lenses extend.

Arguments:

  • type: The native type of this lens (e.g. int, str, some LensObject class). If set, this is a STORE lens, otherwise a NON-STORE lens.
  • name: For assigning a name to this lens for debugging purposes.
  • default: the default output of this lens if a NON-STORE lens.
  • options: arbitrary keyword options can be set on a lens to affect its functionality in an extendible way. For example, we might use this to specify how we align data items to their original position in the source string, perhaps by some label, their original position, or by some order in the native type.
get(concrete_input, current_container=None)

Returns a data item from the string concrete_input according to this lens and its type; if this lens has no type, the input string is simply consumed and nothing returned to be used in our abstract structure.

This effectively wraps the _get function (GET proper) of the specific lens, handling all of the common tasks (e.g. input normalisation, creation of stateful item containers, rolling back state for failed parsing branches, etc.).

Arguments:

  • concrete_input: Concrete input string to parse from.
  • current_container: Outer container (e.g. a list, dict, some LensObject class, etc.) into which items are being extracted. Usually this is used only internally, as the parsing descends into sub-lenses, some of which define their own containers (e.g. using lenses such as Group) and some which don’t but instead store items into some higher-level container.
put(item=None, concrete_input=None, current_container=None, label=None)

Puts an item from our abstract model through the lens to generate its string representation, weaving the model items back into an original string if there was one (i.e. if we are PUTTING rather than CREATING).

Frankly, this is the big beast that drives pylens and you should not worry too much about understanding the internals if you simply wish to build lenses.

Arguments:

  • item: - item to PUT.
  • concrete_input: concrete input for weaving between STORE lens values.
  • current_container - outer container from which items will be consumed and PUT back.
  • label - allows the user to set a label on the passed item, to allow for structures that internally contain a label.

This effectively wraps the _put function (PUT proper) of the specific lens, handling all of the common tasks (e.g. input normalisation, creation of stateful containers, rolling back state for failed parsing branches).

Note that we make no distinction between PUT and CREATE (from the literatur): since a previously extracted item will carry information of its concrete structure within its meta data it will use this for weaving in non-stored artifacts; otherwise, default artifacts will be used (as in CREATE).

In some cases the order that items are put back in will differ from their original order, so we are careful to both consume and discard from the outer concrete structure whilst PUTTING with the item’ own concrete structure. If both structures are aligned (i.e. the item goes back in its original order) then a single PUT on the outer concrete reader is performed.

get_and_discard(concrete_input, current_container)

Sometimes we wish to consume input but discard any items GOTten. Note, it is tempting to somehow not use the current_container, though some lenses might one day use the current container state, so we must first store items in the container before reverting it. For example, the opening and closing tags in XML-like structures.

has_type()

Determines if this lens will GET and PUT a variable - a STORE lens.

container_get(lens, concrete_input_reader, current_container)
Convenience function to handle the case where:
  • if there is a container, get and store an item into it
  • if there is no container, call get and check nothing is returned

This simplifies lens such as And and Repeat, whose logic does not have to worry about whether or not it is acting as a STORE lens.

container_put(lens, concrete_input_reader, current_container)

Reciprocal of container_get.

set_sublens(sublens)

Used if only a single sublens is required (e.g. the Forward lens).

extend_sublenses(new_sublenses)

Adds new sublenses to this lens, being sure to preprocess them (e.g. convert strings to Literal lenses, etc.).

class pylens.base_lenses.And(*lenses, **options)

A lens that is formed from the ANDing of two sub-lenses.

class pylens.base_lenses.Or(*lenses, **options)

This is the OR of two lenses.

class pylens.base_lenses.AnyOf(valid_chars, negate=False, **options)

The first useful low-level lens. Matches a single char within a specified set, and can also be negated.

class pylens.base_lenses.Repeat(lens, min_count=1, max_count=None, **options)

Applies a repetition of the givien lens (i.e. kleene-star).

Arguments:
lens - the lens to repeat min_count - the min repetitions max_count - maximum repetitions (must be > 0 if set)
class pylens.base_lenses.Empty(mode=None, **options)

Matches the empty string, used by Optional(). Can also set modes for special empty matches (e.g. at the start or end of a string).

class pylens.base_lenses.Group(lens, **options)

A convenience lens that thinly wraps any lens, basically to set a type. Usually this is used to close off a lenses container.

class pylens.base_lenses.Literal(literal_string, **options)

A lens that deals with a constant string, usually that will not be stored.

Previous topic

Pylens API

Next topic

Core Lenses

This Page