Extending the parser

Modules such as page3 extend the CSS 2.1 parser to add support for CSS 3 syntax. They do so by sub-classing css21.CSS21Parser and overriding/extending some of its methods. If fact, the parser is made of methods in a class (rather than a set of functions) solely to enable this kind of sub-classing.

tinycss is designed to enable you to have parser subclasses outside of tinycss, without monkey-patching. If however the syntax you added is for a W3C specification, consider including your subclass in a new tinycss module and send a pull request: see Hacking tinycss.

Example: star hack

The star hack uses invalid declarations that are only parsed by some versions of Internet Explorer. By default, tinycss ignores invalid declarations and logs an error.

>>> from tinycss.css21 import CSS21Parser
>>> css = '#elem { width: [W3C Model Width]; *width: [BorderBox Model]; }'
>>> stylesheet = CSS21Parser().parse_stylesheet(css)
>>> stylesheet.errors
[ParseError('Parse error at 1:35, expected a property name, got DELIM',)]
>>> [decl.name for decl in stylesheet.rules[0].declarations]
['width']

If for example a minifier based on tinycss wants to support the star hack, it can by extending the parser:

>>> class CSSStarHackParser(CSS21Parser):
...     def parse_declaration(self, tokens):
...         has_star_hack = (tokens[0].type == 'DELIM' and tokens[0].value == '*')
...         if has_star_hack:
...             tokens = tokens[1:]
...         declaration = super(CSSStarHackParser, self).parse_declaration(tokens)
...         declaration.has_star_hack = has_star_hack
...         return declaration
...
>>> stylesheet = CSSStarHackParser().parse_stylesheet(css)
>>> stylesheet.errors
[]
>>> [(d.name, d.has_star_hack) for d in stylesheet.rules[0].declarations]
[('width', False), ('width', True)]

This class extends the parse_declaration() method. It removes any * delimeter Token at the start of a declaration, and adds a has_star_hack boolean attribute on parsed Declaration objects: True if a * was removed, False for “normal” declarations.

Parser methods

In addition to methods of the user API (see Parsing a stylesheet), here are the methods of the CSS 2.1 parser that can be overriden or extended:

CSS21Parser.parse_rules(tokens, context)

Parse a sequence of rules (rulesets and at-rules).

Parameters:

• tokens – An iterable of tokens.
• context – Either 'stylesheet' or an at-keyword such as '@media'. (Most at-rules are only allowed in some contexts.)

Returns:

A tuple of a list of parsed rules and a list of ParseError.

CSS21Parser.read_at_rule(at_keyword_token, tokens)

Read an at-rule from a token stream.

Parameters:

• at_keyword_token – The ATKEYWORD token that starts this at-rule You may have read it already to distinguish the rule from a ruleset.
• tokens – An iterator of subsequent tokens. Will be consumed just enough for one at-rule.

Returns:

An unparsed AtRule.

Raises :

ParseError if the head is invalid for the core grammar. The body is not validated. See AtRule.

CSS21Parser.parse_at_rule(rule, previous_rules, errors, context)

Parse an at-rule.

Subclasses that override this method must use super() and pass its return value for at-rules they do not know.

In CSS 2.1, this method handles @charset, @import, @media and @page rules.

Parameters:

• rule – An unparsed AtRule.
• previous_rules – The list of at-rules and rulesets that have been parsed so far in this context. This list can be used to decide if the current rule is valid. (For example, @import rules are only allowed before anything but a @charset rule.)
• context – Either 'stylesheet' or an at-keyword such as '@media'. (Most at-rules are only allowed in some contexts.)

Raises :

ParseError if the rule is invalid.

Returns:

A parsed at-rule

CSS21Parser.parse_media(tokens)

For CSS 2.1, parse a list of media types.

Media Queries are expected to override this.

Parameters:tokens – A list of tokens Raises :ParseError on invalid media types/queries Returns:For CSS 2.1, a list of media types as strings

CSS21Parser.parse_page_selector(tokens)

Parse an @page selector.

Parameters:tokens – An iterable of token, typically from the head attribute of an unparsed AtRule. Returns:A page selector. For CSS 2.1, this is 'first', 'left', 'right' or None. Raises :ParseError on invalid selectors

CSS21Parser.parse_declarations_and_at_rules(tokens, context)

Parse a mixed list of declarations and at rules, as found eg. in the body of an @page rule.

Note that to add supported at-rules inside @page, CSSPage3Parser extends parse_at_rule(), not this method.

Parameters:

• tokens – An iterable of token, typically from the body attribute of an unparsed AtRule.
• context – An at-keyword such as '@page'. (Most at-rules are only allowed in some contexts.)

Returns:

A tuple of:

• A list of Declaration
• A list of parsed at-rules (empty for CSS 2.1)
• A list of ParseError

CSS21Parser.parse_ruleset(first_token, tokens)

Parse a ruleset: a selector followed by declaration block.

Parameters:

• first_token – The first token of the ruleset (probably of the selector). You may have read it already to distinguish the rule from an at-rule.
• tokens – an iterator of subsequent tokens. Will be consumed just enough for one ruleset.

Returns:

a tuple of a RuleSet and an error list. The errors are recovered ParseError in declarations. (Parsing continues from the next declaration on such errors.)

Raises :

ParseError if the selector is invalid for the core grammar. Note a that a selector can be valid for the core grammar but not for CSS 2.1 or another level.

CSS21Parser.parse_declaration_list(tokens)

Parse a ; separated declaration list.

You may want to use parse_declarations_and_at_rules() (or some other method that uses parse_declaration() directly) instead if you have not just declarations in the same context.

Parameters:tokens – an iterable of tokens. Should stop at (before) the end of the block, as marked by }. Returns:a tuple of the list of valid Declaration and a list of ParseError

CSS21Parser.parse_declaration(tokens)

Parse a single declaration.

Parameters:tokens – an iterable of at least one token. Should stop at (before) the end of the declaration, as marked by a ; or }. Empty declarations (ie. consecutive ; with only white space in-between) should be skipped earlier and not passed to this method. Returns:a Declaration Raises :ParseError if the tokens do not match the ‘declaration’ production of the core grammar.

CSS21Parser.parse_value_priority(tokens)

Separate any !important marker at the end of a property value.

Parameters:tokens – A list of tokens for the property value. Returns:A tuple of the actual property value (a list of tokens) and the priority.

Unparsed at-rules

class tinycss.css21.AtRule(at_keyword, head, body, line, column)

An unparsed at-rule.

at_keyword

The normalized (lower-case) at-keyword as a string. Eg: '@page'

head

The part of the at-rule between the at-keyword and the { marking the body, or the ; marking the end of an at-rule without a body. A TokenList.

body

The content of the body between { and } as a TokenList, or None if there is no body (ie. if the rule ends with ;).

The head was validated against the core grammar but not the body, as the body might contain declarations. In case of an error in a declaration, parsing should continue from the next declaration. The whole rule should not be ignored as it would be for an error in the head.

These at-rules are expected to be parsed further before reaching the user API.

Parsing helper functions

The tinycss.parsing module contains helper functions for parsing tokens into a more structured form:

tinycss.parsing.strip_whitespace(tokens)

Remove whitespace at the beggining and end of a token list.

Whitespace tokens in-between other tokens in the list are preserved.

Parameters:tokens – A list of Token or ContainerToken. Returns:A new sub-sequence of the list.

tinycss.parsing.split_on_comma(tokens)

Split a list of tokens on commas, ie , DELIM tokens.

Only “top-level” comma tokens are splitting points, not commas inside a function or other ContainerToken.

Parameters:tokens – An iterable of Token or ContainerToken. Returns:A list of lists of tokens

tinycss.parsing.validate_value(tokens)

Validate a property value.

Parameters:tokens – an iterable of tokens Raises :ParseError if there is any invalid token for the ‘value’ production of the core grammar.

tinycss.parsing.validate_block(tokens, context)

Raises :

ParseError if there is any invalid token for the ‘block’ production of the core grammar.

Parameters:

• tokens – an iterable of tokens
• context – a string for the ‘unexpected in ...’ message

tinycss.parsing.validate_any(token, context)

Raises :

ParseError if this is an invalid token for the ‘any’ production of the core grammar.

Parameters:

• token – a single token
• context – a string for the ‘unexpected in ...’ message