‘jsondata.JSONData’ - Module¶

Core features for the processing of structured JSON based in-memory data. This comprises the load of a master model from a JSON file, and the incremental addition and removal of branches by loading additional JSON modules into the master model. The resulting data could be saved for later reuse, where complex configuration is varied by user interaction. The implementation is based on the standard packages ‘json’ and ‘jsonschema’.

This module uses for the syntax of JSON data either a preloaded module, or loads the standard module by default. Current supported packages are:

json: The standard json package of the Python distribution.
ujson: ‘Ultra-JSON’, a wrapped C implementation with

high-performance conversion.

The current default module is ‘json’ for syntax processing, the standard package ‘jsonschema’ for the optional validation.

Constants¶

Compliance modes¶

MODE_JSON_RFC4927 = 0: Compliant to IETF RFC4927.
MODE_JSON_RF7951 = 2: Compliant to IETF RF7951.
MODE_JSON_ECMA264 = 10: Compliant to ECMA-264, refer to Chapter 15.12 The JSON Object.
MODE_POINTER_RFC6901 = 20: Compliant to IETF RFC6901.
MODE_PATCH_RFC6902 = 30: Compliant to IETF RFC6902.
MODE_SCHEMA_DRAFT3 = 43: Compliant to IETF DRAFT3.
MODE_SCHEMA_DRAFT4 = 44: Compliant to IETF DRAFT4.

Types of validator¶

OFF = 0: No validation.
DRAFT4 = 1: Use draft4: jsonschema.validator(Draft4Validator)
DRAFT3 = 2: Use draft3:jsonschema.Draft3Validator

The default value is:

DEFAULT = DRAFT4 = 1: Default

Match Criteria¶

Match criteria for node comparison:

MATCH_INSERT = 0: for dicts
MATCH_NO = 1: negates the whole set
MATCH_KEY = 2: for dicts
MATCH_CHLDATTR = 3: for dicts and lists
MATCH_INDEX = 4: for lists
MATCH_MEM = 5: for dicts(value) and lists
MATCH_NEW = 6: for the creation of new

Return Sets¶

FIRST: The first match only.
ALL: All matches.

JSONData¶

class jsondata.JSONData.JSONData(*args, **kargs)[source]¶

Representation of a JSON based object data tree.

This class provides for the handling of the in-memory data by the main hooks ‘data’, and ‘schema’. This includes generic methods for the advanced management of arbitrary ‘branches’ in extension to RCF6902, and additional methods strictly compliant to RFC6902.

Due to the pure in-memory support and addressing by the enclosed module JSONPointer for RFC6901 compliant addressing by in memory caching, the JSONData may outperform designs based on operation on the native JSON representation.

Attributes:

data: The data tree of JSON based objects provided: by the module ‘json’.
schema: The validator for ‘data’ provided by: the module ‘jsonschema’.

Common call parameters provided by the methods of this class are:

targetnode := addressreference: The target node of called method. The ‘targetnode’ in general represents the target of the called method. In most cases this has to be a reference to a container for the modification and/or insertion of resulting elements. The methods require the change of contained items, which involves the application of a ‘key’ pointing to the hook in point of the reference to the modification.
key := key-value: The hook-in point for references of modified entries within the targetnode container. The following values are supported:
sourcenode := addressreference: The in-memory node address of the source branch for the method, e.g. ‘copy’ or ‘move’ operation.

The address references supported in this class refer the resulting in-memory representation of a pointer path. The target is a node within a Python data representation as provided by the package ‘json‘ and compatible packages, e.g. ‘ujson‘. The supported input syntax is one of the following interchangeable formats:

# The reference to a in-memory-node.
addressreference := (
      nodereference
    | addressreference-source
)

nodereference:= (
      <in-memory>
    | ''
)

<in-memory> := "Memory representation of a JSON node, a 'dict'
    or a 'list'. The in-memory Python node reference has to be
    located within the document, due to performance reasons this
    is not verified by default.
    
    The 'nodereference' could be converted from the
    'addressreference-source' representation."

'' := "Represents the whole document in accordance to RFC6901.
    Same as 'self.data'." 

# The source of the syntax for the description of the reference
# pointer path to a node. This is applicable on paths to be created.
addressreference-source := (
    JSONPointer
)
      
JSONPointer:="A JSONPointer object in accordance to RFC6901.
    for additional information on input formats refer to the 
    class documentation.
    This class provides a fully qualified path pointer, which
    could be converted into any of the required representations."

For hooks by ‘key-value’ within addressed containers:

key-value:=(None|<list-index>|<dict-key>) 
    
None := "When the 'key' parameter is 'None', the action 
    optionally could be based on the keys of the 'sourcenode'.  
    The contents of the branch replace the node contents
    when the type of the branch matches the hook."

<list-index>:=('-'|int)

<dict-key>:="Valid for a 'dict' only, sets key/value pair, 
    where present is replace, new is created."

'-' := "Valid for a 'list' only, appends to present."

int := "Valid for a 'list' only, replaces present when
    0 < #int < len(Node)."

In the parameter lists of methods used term ‘pointer’ is either an object of class ‘JSONPointer’, or a list of pointer path entries.

The JSON types ‘object’ and ‘array’ behave in Python slightly different in accordance to RFC6902. The main difference arise from the restrictions on applicable key values. Whereas the ranges are limited logically by the actual container sizes, the object types provide free and unlimited keys. The limit is set by type restriction to unicode and ‘non-nil’ only for keys.

Attributes¶

JSONData.data: JSON object data tree.
JSONData.schema: JSONschema object data tree.

Methods¶

init¶

JSONData.__init__(*args, **kargs)[source]¶

Loads and validates a JSON definition with the corresponding schema file.

Args:

args*: Optional position parameters, these branch_replace corresponding key

parameters.

data

**kargs:

data: JSON data within memory.

default:= None

indent_str: Defied the indentation of ‘str’.

default:= 4

interactive: Hints on command line call for optional change of display format.

default:= False

schema: A valid in-meory JSONschema.

default:= None

validator: [default, draft3, draft4, on, off, ]

Sets schema validator for the data file. The values are: default=validate, draft3=Draft3Validator, off=None

default:= off

printdata: branch=None

Pretty print resulting final data of branch.

default:= top

printschema: branch=None

Pretty print resulting schema.

default:= top

debug: Displays extended state data for developers.

Requires __debug__==True.

verbose: Extends the amount of the display of

processing data.

Returns:

Results in an initialized object.

Raises:

NameError:

JSONDataValue:

jsonschema.ValidationError:

jsonschema.SchemaError:

repr¶

JSONData.__repr__()[source]¶: Dump data.

str¶

JSONData.__str__()[source]¶: Dumps data by pretty print.

branch_add¶

JSONData.branch_add(targetnode, key, sourcenode)[source]¶

Add a complete branch into a target structure of type object.

Present previous branches are replaced, non-existent branches are added. The added branch is created by a deep copy, thus is completely independent from the source.

Call: branch_add ( t, k, s )

i target source add type

t k s from to

0 node key node s t[k] any

1 node None node s t[*] match

Use-Case-0: Any source node type is added as ‘t[k]’.

Use-Case-1: The content keys of node ‘s’ are added each

to the node ‘t’. Therefore the node types of ‘s’ and ‘t’ have to match.

This behaviour is defined in respect to the parameter passing of Python.

Args:

targetnode := nodereference: Target container node where the branch is to be inserted.
key := key-value: Hook for the insertion within target node.
sourcenode := nodereference: Source branch to be inserted into the target tree.

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONDataNodeType: JSONDataKeyError:

branch_copy¶

JSONData.branch_copy(targetnode, key, sourcenode, force=True)[source]¶

Copies the source branch to the target node.

The copy is internally mapped onto the ‘branch_add’ call, thus shares basically the same parameters and behaviour. Due to the required modification of the target only, the copy is slightly different from the ‘branch_move’ call.

Call: branch_copy ( t, k, s )

i target source copy type

t k s from to

0 node key node s t[k] any

1 node None node s t[sk] match

For the description of the Use-Cases refer to branch_add.

Args:

targetnode := nodereference

Target tree the branch is to be inserted.

key := key-value

Key of insertion point within target node.

sourcenode := nodereference

Source branch to be inserted into target tree.

force: If true present are replaced, else only non-present

are copied.

default:=True

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONData:

branch_create¶

JSONData.branch_create(targetnode, branch, value=None)[source]¶

Creates a branch located at targetnode.

The requested branch as created as child value of provided ‘targetnode’. ‘targetnode’ is required to exist.

REMARK: Current version relies for the created nodes on the

content type of the key(str,unicode)/index(int), later versions may use a provided schema.

Call: branch_create ( t, b, v )

i	target	branch	value
i	t	b	v
0	node	list	[any]
1	node	list	[any]
2	node	pointer	[any]
3	node	pointer	[any]

Args:

targetnode := nodereference: Base node for the insertion of branch.
branch := addressreference-source: New branch to be created in the target tree. A Pointer address path relative to the ‘targetnode’.
value: Optional value for the leaf. The value itselfn: could be either an atomic type, or a branch itself in accordance to RFC6902.

Returns:

When successful returns the leaf node, else returns either ‘None’, or raises an exception.

Raises:

JSONData:

branch_move¶

JSONData.branch_move(targetnode, key, sourcenode, skey, force=True, forcext=False)[source]¶

Moves a source branch to target node.

Moves by default only when target is not yet present. The parameters for ‘list’, ‘force’ enabled to overwrite, whereas the parameter ‘forcext’ enables to move all entries and extend the target items.

Due to the Python specific passing of flat parameters as a copy of the reference without access to the actual source entry, these are slightly different from the ‘branch_copy’ and ‘branch_add’ methods modifying the target only. Therefore additional source keys ‘skey’ are required by ‘move’ in order to enable the modification of the source entry.

Call: branch_move ( t, k, s, sk )

i target source move type

t k s sk from to

0 node key node key s[sk] t[k] any

1 node None node key s[sk] t[sk] match

Use-Case-0: Moves any.

Use-Case-1: Moves matching key types only: list-to-list,

or dict-to-dict.

Args:

targetnode := nodereference

Target tree the branch is to be inserted.

key := key-value

Key of insertion point within target node.

sourcenode := nodereference

Source branch to be inserted into target tree.

skey := key-value

Key of the source to be moved to target node.

force: If true present are replaced, else only

non-present are moved.

default:=True

forcext: If true target size will be extended when

required. This is applicable on ‘list’ only, and extends RFC6902. The same effect is given for a ‘list’ by one of:

key:=’-‘
key:=None and skey:=’-‘

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONData: JSONDataKey: KeyError:

branch_remove¶

JSONData.branch_remove(targetnode, key)[source]¶

Removes a branch from a target structure.

The corresponding elements of the ‘targetnode’ tree are removed. The remaining are kept untouched. For tree nodes as leafs the whole corresponding subtree is deleted.

REMARK: No reference checks are done, so the user is responsible

for additional references.

Call: branch_remove ( t, k )

i	target		remove	type
i	t	k	branch	type
0	node	key	t[k]	any
1	node	None	t[*]	any

Use-Case-0: Removes any type of node.
Use-Case-1: Removes all contained items of any type.

Args:

targetnode := nodereference: Container of ‘targetnode’ with items to be removed.
key := key-value: The item to be removed from the ‘targetnode’. When ‘None’, all contained items are removed.

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONDataException:

branch_replace¶

JSONData.branch_replace(targetnode, key, sourcenode)[source]¶

Replaces the value of the target node by the copy of the source branch.

Requires in order to RFC6902, all items to be replaced has to be present. Thus fails if at least one is missing.

Internally the ‘branch_add()’ call is used with a deep copy. When a swallow copy is required the ‘branch_move()’ has to be used.

Args:

targetnode := nodereference: Target tree the branch is to be inserted.
key := key-value: Key of insertion point within target node. If key==None, the whole set of keys is replaced by the content of the ‘sourcenode’.
sourcenode := nodereference: Source branch to be inserted into target tree.
force: If true present are replaced, else only non-present: are copied.

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONData:

branch_test¶

classmethod JSONData.branch_test(targetnode, value)[source]¶

Tests match in accordance to RFC6902.

Args:

targetnode := a valid node: Node to be compared with the value. Due to ambiguity the automated conversion is not reliable, thus it has to be valid.

value: Expected value for the given node.

Returns:

When successful returns ‘True’, else returns ‘False’.

Raises:

JSONData:

getData¶

JSONData.getData()[source]¶: Returns the reference to data.

getPointerPath¶

classmethod JSONData.getPointerPath(node, base, restype=1)[source]¶

Converts a node address into the corresponding pointer path.

The current implementation is search based, thus may have performance issues when frequently applied.

Args:

node: Address of Node to be searched for.

base: A tree top nodes to search for node.

restype: Type of search.

first: The first match only.

all: All matches.

Returns:

Returns a list of lists, where the contained lists are pointer path-lists for matched elements.

restype:=FIRST: ‘[[<first-match>]]’,
restype:=ALL: ‘[[<first-match>],[<second-match>],...]’

Raises:

JSONData:

getSchema¶

JSONData.getSchema()[source]¶: Returns the reference to schema.

getTreeDiff¶

classmethod JSONData.getTreeDiff(n0, n1, difflst=None, alldifs=False, dl=0, path='')[source]¶

Recursive tree compare for Python trees as used for the package ‘json’.

Finds diff in native Python trees assembled by the standard package ‘json’ and compatible, e.g. ‘ujson’.

getValueNode¶

isApplicable¶

JSONData.isApplicable(targetnode, key, branch, matchcondition=None, **kargs)[source]¶

Checks applicability by validation of provided match criteria.

The contained data in ‘datafile’ could be either the initial data tree, or a new branch defined by a fresh tree structure. The ‘targetnode’ defines the parent container where the new branch has to be hooked-in.

Args:

targetnode:

Target container hook for the inclusion of the loaded branch. The branch is treated as a child-branch, hooked into the provided container ‘targetnode’.

branch:

Branch to be imported into the target container. The branch is treated as a child-branch.

matchcondition:

Defines the criteria for comparison of present child nodes in the target container. The value is a list of critarias combined by logical AND. The criteria may vary due to the requirement and the type of applied container: - common: Common provided criteria are:

insert: Just checks whether the branch could be inserted.

In case of ‘list’ by ‘append’, in case of a ‘dict’ by the insert-[]-operator. This is in particular foreseen for the initial creation of new nodes.

present: Checks whether all are present.

no: Inverts the match criteria for the whole current set.

dict: The provided criteria are:
- key: Both share the same key(s).
- child_attr_list: A list of child attributes to be matched.
  
  This may assure e.g. compatibility by a user defined ID, and or a UUID.
default:=[‘key’,]
list: The provided criteria are:
- index: The positions of source and target have to match.
- child_attr_list: A list of child attributes to be matched,
  
  thus e.g. the ‘key’ of dictionaries could be emulated by an arbitrary attribute like ‘mykey’. This may assure e.g. compatibility by a user defined ID, and or a UUID.
- mem: Checks whether the in-memory element is already present.
  
  Even though this is a quite weak criteria, it is probably the only and one common generic criteria for lists.
default:= mem # ATTENTION: almost any call adds a branch!

**kargs:

childattrlist: A list of user defined child attributes which

all together(AND) define the match criteria.

default:=None, returns ‘True’

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

The rule of thumb is:

type-mismatch: Exception
value-mismatch: return False

Success is: no-defined-condition or no-failing-condition

Raises:

JSONData:

JSONDataValue:

printData¶

JSONData.printData(pretty=True, **kargs)[source]¶

Prints structured data.

Args:

pretty: Activates pretty printer for treeview, else flat.

sourcefile: Loads data from ‘sourcefile’ into ‘source’.

default:=None

source: Prints data within ‘source’.

default:=self.data

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONDataAmbiguity:

forwarded from ‘json’

printSchema¶

JSONData.printSchema(pretty=True, **kargs)[source]¶

Prints structured schema.

Args:

pretty: Activates pretty printer for treeview, else flat.

sourcefile: Loads schema from ‘sourcefile’ into ‘source’.

default:=None

source: Prints schema within ‘source’.

default:=self.schema

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONDataAmbiguity:

forwarded from ‘json’

pop¶

JSONData.pop(key)[source]¶: Transparently passes the ‘pop()’ call to ‘self.data’.

setSchema¶

JSONData.setSchema(schemafile=None, targetnode=None, **kargs)[source]¶

Sets schema or inserts a new branch into the current assigned schema.

The main schema(targetnode==None) is the schema related to the current instance. Additional branches could be added by importing the specific schema definitions into the main schema. These could either kept volatile as a temporary runtime extension, or stored into a new schema file in order as extension of the original for later combined reuse.

Args:

schemafile:

JSON-Schema filename for validation of the subtree/branch. See also **kargs[‘schema’].

targetnode:

Target container hook for the inclusion of the loaded branch.

**kargs:

schema:

In-memory JSON-Schema as an alternative to schemafile. When provided the ‘schemafile’ is ignored.

default:=None

validator: [default, draft3, off, ]

Sets schema validator for the data file. The values are: default=validate, draft3=Draft3Validator, off=None.

default:= validate

persistent:

Stores the ‘schema’ persistently into ‘schemafile’ after completion of update including addition of branches. Requires valid ‘schemafile’.

default:=False

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

JSONData:

JSONDataSourceFile:

JSONDataValue:

validate¶

JSONData.validate(data, schema, validator=None)[source]¶

Validate data with schema by selected validator.

Args:

data:

JSON-Data.

schema:

JSON-Schema for validation.

validator:

Validator to be applied, current supported:

schema:

In-memory JSON-Schema as an alternative to schemafile. When provided the ‘schemafile’ is ignored.

default:=None

validator: [default, draft3, draft4, off, on, ]

default|MODE_SCHEMA_ON: The current default.
draft3|MODE_SCHEMA_DRAFT3: The first supported JSONSchema IETF-Draft.
draft4|MODE_SCHEMA_DRAFT4: The current supported JSONSchema IETF-Draft.
off|MODE_SCHEMA_OFF:: No validation.

Sets schema validator for the data file.

default:= MODE_SCHEMA_DRAFT4

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

ValidationError: SchemaError: JSONDataValue:

Operators¶

‘+’¶

JSONData.__add__(x)[source]¶: Adds the structure ‘x’ to ‘self’, performs deep-operation.

‘&&’¶

JSONData.__and__()[source]¶: Gets the intersection of ‘x’ and ‘self’, performs deep-operation.

‘()’¶

JSONData.__call__(x)[source]¶

Evaluates the pointed value from the document.

Args:: x: A valid JSONPointer.
Returns:: The pointed value, or None.
Raises:: JSONPointerException

‘S==x’¶

JSONData.__eq__(x)[source]¶

Compares this JSONData.data with x.

Args:: x: A valid JSONData.
Returns:: True or False
Raises:: JSONDataException

‘[]’¶

JSONData.__getitem__(key)[source]¶: Support of slices, for ‘iterator’ refer to self.__iter__.

‘S!=x’¶

JSONData.__ne__(x)[source]¶

Compares this JSONData with x.

Args:: x: A valid JSONData.
Returns:: True or False
Raises:: JSONDataException

‘+=’¶

JSONData.__iadd__(x)[source]¶: Adds the structure ‘x’ to ‘self’, performs deep-operation.

‘=&&’¶

JSONData.__iand__()[source]¶: Gets the intersection of ‘x’ and ‘self’, performs deep-operation.

‘S%x’¶

JSONData.__imod__(x)[source]¶: Returns the difference-modulo-set.

‘*=’¶

JSONData.__imul__(x)[source]¶: Duplicates the elements of ‘self’ ‘x’ times.

‘||=’¶

JSONData.__ior__(x)[source]¶: Returns the superset of branches and attributes.

‘-=’¶

JSONData.__isub__(x)[source]¶: Returns the residue of X after each present element of ‘x’ is removed.

‘^=’¶

JSONData.__ixor__(x)[source]¶: Returns the elements present in one only.

‘%’¶

JSONData.__mod__(x)[source]¶

Returns the difference-modulo-set.

The operations:

z = S % x

Returns the remaining subset of:

z = S - n * x

where ‘n*x’ is the maximum number of present branches ‘x’. When multiple exist, all matching are removed.

‘*’¶

JSONData.__mul__(x)[source]¶

Duplicates the elements of ‘self’ ‘x’ times.

The operations:

z = S * x

Returns the remaining subset of:

z = S - 1 * x

where ‘1*x’ is for each present element of ‘x’. When multiple exist ‘n-1’ remain.

‘||’¶

JSONData.__or__(x)[source]¶: Returns the superset of branches and attributes.

‘S+x’¶

JSONData.__radd__(x)[source]¶: Adds the structure ‘x’ to ‘self’, performs deep-operation.

‘S&&x’¶

JSONData.__rand__()[source]¶: Gets the intersection of ‘x’ and ‘self’, performs deep-operation.

‘S%x’¶

JSONData.__rmod__(x)[source]¶: Returns the difference-modulo-set.

‘S*x”¶

JSONData.__rmul__(x)[source]¶: Duplicates the elements of ‘self’ ‘x’ times.

‘S||x”¶

JSONData.__ror__(x)[source]¶: Returns the superset of branches and attributes.

‘S^x”¶

JSONData.__rxor__(x)[source]¶: Returns the elements present in one only.

‘-‘¶

JSONData.__sub__(x)[source]¶

Returns the residue of X after each present element of ‘x’ is removed.

The operations:

z = S - x

Returns the remaining subset of:

z = S - 1 * x

where ‘1*x’ is for each present element of ‘x’. When multiple exist ‘n-1’ remain.

‘^’¶

JSONData.__xor__()[source]¶: Returns the structure elements present in in one only.

Iterators¶

iter¶

JSONData.__iter__()[source]¶: Provides an iterator for data.

Exceptions¶

class jsondata.JSONData.JSONDataAmbiguity(requested, *sources)[source]¶: Error ambiguity of provided parameters.

i	target		source		move		type
i	t	k	s	sk	from	to	type
0	node	key	node	key	s[sk]	t[k]	any
1	node	None	node	key	s[sk]	t[sk]	match

‘jsondata.JSONData’ - Module¶

Constants¶

Compliance modes¶

Types of validator¶

Match Criteria¶

Return Sets¶

JSONData¶

Attributes¶

Methods¶

__init__¶

__repr__¶

__str__¶

branch_add¶

branch_copy¶

branch_create¶

branch_move¶

branch_remove¶

branch_replace¶

branch_test¶

getData¶

getPointerPath¶

getSchema¶

getTreeDiff¶

getValueNode¶

isApplicable¶

printData¶

printSchema¶

pop¶

setSchema¶

validate¶

Operators¶

‘+’¶

‘&&’¶

‘()’¶

‘S==x’¶

‘[]’¶

‘S!=x’¶

‘+=’¶

‘=&&’¶

‘S%x’¶

‘*=’¶

‘||=’¶

‘-=’¶

‘^=’¶

‘%’¶

‘*’¶

‘||’¶

‘S+x’¶

‘S&&x’¶

‘S%x’¶

‘S*x”¶

‘S||x”¶

‘S^x”¶

‘-‘¶

‘^’¶

Iterators¶

__iter__¶

Exceptions¶

init¶

repr¶

str¶

iter¶