‘filesysobjects.FileSysObjects’ - Module¶

The FileSysObjects package provides operations on paths, path parts and side branches.

For extended additional information refer to the manuals, offline, or online at “https://pythonhosted.org/pyfilesysobjects/”. Create manuals from sources by: “python setup.py build_sphinx” and “python setup.py build_epydoc”

The current version calls ‘os.path.normpath’ by default - when ‘raw’ is not selected. This is consistent for all path related parameters including search paths: start, top, plist, spath, etc.. Thus generally clears double slashes, but also replaces symbolic links, so later literal post processing e.g. for match based processing should be normalized too.

There is one exception due to for leading ‘//’ and ‘\’, see option ‘ias’ and IEEE Std 1003.1(TM), UNC, and SMB/CIFS for Pathname Resolution. Current supported URIs for filenames are: ‘file://‘, ‘smb://‘, and ‘cifs://’.

The following options are generic and common to multiple interfaces:

spath: An existing path to be added to an entry

from ‘plist’. The following cases are supported, for further specifics refer to the interfaces.

Independent path entry: spath is absolute, just added.

Subpath of current directory spath is relative and present in current working directory, added py prefixing ‘pwd’.

Arbitrary side-branch of a provided path spath is relative, searched in plist for an insertion hook, added when found as absolute.

3. Pattern matching - see manual ‘Semi-Literals’ and shortcut tables in manual:

regexpr: Regular expressions are applicable for

match on ‘plist’ only. Thus the part to be matched on the file system is required to be a literal.

glob: Glob expressions are applicable on the file system

itself only, thus the part to be matched on the ‘plist’ is required to be a literal.

Is absolute path:

Is checked to be a sub path of at least one of ‘plist’, than applied.

start: Start directory or file, when a file is provided the

directory portion is used as the starting pointer.

Each part is compared separately, but as a whole string.

top: The topmost path within a directory tree as an end point
for a search operation. This is defined by the end of a directory path name string. E.g. the the bottom-up search beginning at the start directory:
start=/a/b/c/d/e/f/g
is terminated by””:
top=d
at:
/a/b/c/d
This is used as a match string for processing literally on the parts of the provided start directory. The match is checked after application of ‘os.path.normpath’. Providing absolute paths still match, because of the string, but eventually match multiple times when equal sub paths exist and the match order is changed to bottom-up search.

The containment of ‘top’ within the absolute ‘start’ path is verified.

Each part is compared separately, but as a whole string.
plist: List of strings to be searched. By default first match

is used. Each is split into it’s components and matched separately.

default := sys.path

matchidx=#idx: Matches on the provided index count only
#idx==2 - ignores 0,1 and >2, matches idx==2
matchcnt=#num: The maximal number of matches returned when
multiple occur:
#num==0 - all
#num>0  - number of matches returned
matchlvl=#num: Increment of match for top node when multiple
are in the path. The counter starts at top, so #num(1) will match M(1) in:
/a/b/M/c/M/d/M/w/e/M/bottom
     0   1   2     3
     |   *
     |
     `-default
matchlvlupward=#num: Increment of match for top node when multiple
are in the path. The counter starts from the bottom, so #num(2) will match M(2) in:
/a/b/M/c/M/d/M/w/e/M/bottom
     3   2   1     0
         *
raw: Suppress normalization by call of ‘os.path.normpath’. The

caller has than to take care for appropriate measures for a feasible match.

ias: Treats for local file names any number of

subsequent ‘/’ only as one, also leading pattern ‘//[^/]+’. URI prefixes are treated correctly.

See also “IEEE Std 1003.1(TM), 2013 Edition; Chap. 4.12”.

Functions¶

addPathToSearchPath¶

filesysobjects.FileSysObjects.addPathToSearchPath(spath, plist=None, **kargs)[source]¶

Adds a path to ‘plist’.

In case of relative paths searches in provided ‘plist’, or ‘kargs[searchplist]’a hook, when found verifies the existence within file system, in case of success adds the completed path to ‘plist’ the list.

In case of ‘glob’ adds all entries.

Args:

spath:

A path to be added to ‘plist’. See common options for details. Valid scope types:

literal : X

re : -

blob : -

default := caller-file-position.

plist:

List to for the storage, and by default search list too. See common options for details.

default := sys.path

**kargs:

append:

Append, this is equal to pos=len(plist).

checkreal:

Checks redundancy by resolving real path, else literally.

exist:

Checks whether exists, else nothing is done.

pos=#pos:

A specific position for insertion within range(0,len(plist)).

prepend:

Prepend, this is equal to pos=0.

redundant:

Add relative, allow redundant when same is already present.

relative=<base>:

Add relative sub path to provided base.

searchplist:

Alternative list to search for checks.

Returns:

When successful returns insertion position, else a ‘value<0’. The insertion position in case of multiple items is the position of the last.

Raises:

passed through exceptions:

clearPath¶

filesysobjects.FileSysObjects.clearPath(plist=None, **kargs)[source]¶

Clears, splits and joins a list of path variables by various criteria.

Args:

plist:

List of paths to be cleared. See common options for details.

default := sys.path

**kargs:

abs:

Converts all entries into absolute pathnames.

existent:

Removes all existing items. For test and verification.

ias:

Treats for local file names any number of subsequent ‘/’ only as one.

See common options for details.

non-existent:

Removes all items which do not exist.

non-redundant:

Removes all items which are not redundant. Results e.g. in multiple incarnations of the same file/path type.

normpath:

Calls ‘os.path.normpath’ on each result.

redundant:

Clears all items from redundancies.

rel:

Converts all entries into relative pathnames.

reverse:

This reverses the resulting search order from bottom-up to top-down. Takes effect on ‘redundant’ only.

shrink:

Drops resulting empty items.

split:

Forces split of multiple paths items within one item into separate item entries.

default := noSplit

withinItemOnly:

Performs any action for each item of ‘plist’ only.

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

passed through exceptions:

delPathFromSearchPath¶

filesysobjects.FileSysObjects.delPathFromSearchPath(dellist, plist=None, **kargs)[source]¶

Deletes a list of paths from ‘plist’.

Args:

dellist:

A list of paths to be deleted from ‘plist’. Valid scope types:

literal : X

re : X

glob : X

see kargs[regexpr|glob].

default := None

plist:

List of search paths.

default := sys.path

**kargs:

The following keys are additional before comparison, on ‘dellist’ only when no match pattern is provided:

case:

Calls on both: os.path.normcase

esc:

Calls on both: escapeFilePath/unescapeFilePath

exist:

Calls on both: os.path.exists

noexist:

Calls on both: not os.path.exists

normX:

Calls on both: normpathX

norm:

Calls on both: os.path.normpath

real:

Calls on both: os.path.realpath

regexpr|glob:

Input is a list of

regexpr:

regular expressions, just processed by

‘re.match(dl,pl)’

glob:

process glob, and check containment in set

Returns:

When successful returns True, else False.

Raises:

passed through exceptions

escapeFilePath¶

filesysobjects.FileSysObjects.escapeFilePath(p, tps=None)[source]¶
Normalize ‘os.sep’ by re module.

Considers any os.sep seperated item as path part. Manages os.sep including flags of ‘re’ as special characters,e.g. ‘a’, ‘b’, and ‘x’.
Args:
p: A path.
tps:
Target path seperator:
s: slash

b: backslash

k: keep

keeps, also intermixed, just escapes backslash

cnp: cross native posix
Emulates native behavior of ‘os.path.normpath’ on posix platform(Linux, MacOS, ...), e.g.:
d:/  => d:
d:\  => d:\
cnw:cross native win
Emulates native behavior of ‘os.path.normpath’ on win platform, e.g.:
d:/  => d:\
d:\  => d:\
else: adapt os.sep, local os native
Returns:

Path.

findRelPathInSearchPath¶

filesysobjects.FileSysObjects.findRelPathInSearchPath(spath, plist=None, **kargs)[source]¶

Searches the provided path list for existing filesystem objects ‘spath’. The file system objects are searched as side-branches of the provided plist as a list of hooks(pathx[-1]) suitable to ‘spath’. The path items are treated as entry point hooks for specific extension branches.

The parameters of name type ‘is*’ are recommended when the iterator is applied. This enhances the performance vs. the post filtering.

Args:

spath:=(literal|glob): A path to be hooked into

‘plist[]’ when present. Could be either a literal, or a glob as an relative or absolute path. Valid scope types:

literal : X

re : -

glob : X

See common options for details.

plist:

List of potential hooks for ‘spath’. The following formats are provided:

list of single paths - used literally

list of search path strings - each search path is split

string with search path - split into it’s components

string with a single path - used literally

The default behavior is:

first: #1

second: #3, this contains #4

The case #2 has to be forced by the key-option: ‘subsplit’, or to be prepared by the call ‘clearPath(split=True,)’.

Due to performance the case #1 should be preferred in order to save repetitive automatic conversion.

See common options for further details.

default := sys.path

**kargs:

ias:

Treats for local file names any number of subsequent ‘/’ only as one.

isDir:

Is a directory.

isFile:

Is a file.

isLink:

Is a symbolic link.

isPathByLink:

Has a symbolic link in path.

matchidx=#idx:

Ignore matches ‘< #idx’, return match ‘== #idx’. Depends on ‘reverse’

default := 0 # first match

noglob:

Suppress application of ‘glob’.

not:

Inverts to does not matched defined criteria.

raw:

Suppress normalization by call of ‘os.path.normpath’.

reverse:

Reversed search order.

subsplit:

Splits each item part of a ‘plist’ option.

Returns:

When successful returns the absolute pathname, else ‘None’. For a list refer to iterator.

Raises:

passed through exceptions:

findRelPathInSearchPathIter¶

filesysobjects.FileSysObjects.findRelPathInSearchPathIter(spath, plist=None, **kargs)[source]¶

Iterates all matches in plist, see findRelPathInSearchPath.

findRelPathInUpperTree¶

getAppPrefixLocalPath¶

filesysobjects.FileSysObjects.getAppPrefixLocalPath(elems, **kargs)[source]¶

Joins app elements to a path for local access.

Args:

elems:

Elements as provided by ‘splitAppPrefix’

**kargs:

tpf:

Target platform for the file pathname, for details refer to ‘normpathX’ [see normpathX].

Returns:

When access path when successful, else None.

Raises:

passed through exceptions:

getHome¶

filesysobjects.FileSysObjects.getHome()[source]¶

Gets home directory with complete local file path name, eventually drive.

getDirUserData¶

filesysobjects.FileSysObjects.getDirUserData()[source]¶

Gets data directory with complete local file path name, eventually drive.

getDirUserConfigData¶

filesysobjects.FileSysObjects.getDirUserConfigData()[source]¶

Gets data directory for configuration.

getDirUserAppData¶

filesysobjects.FileSysObjects.getDirUserAppData()[source]¶

Gets data directory for applications.

getTopFromPathString¶

filesysobjects.FileSysObjects.getTopFromPathString(spath, plist=None, **kargs)[source]¶

Searches for a partial path in search paths from a provided list.

Searches for a given path component by various constraints on each string of provided ‘plist’ until the match of a break condition. The match is performed by default left-to-right, which results in top-down scan of a path hierarchy, or right-to-left as an upward bottom-up search.

Performs string operations only, the file system is neither checked, not utilized.

Args:

spath:

A path to be added to ‘plist’. See common options for details.

plist:

List of search strings for match. See common options for details.

default := sys.path

**kargs:

abs:

Return absolute path.

hook:

Returns the found part of the ‘plist’ entry only.

ias:

Treats for local file names any number of subsequent ‘/’ only as one.

includeapp:

Includes application specifics into search. E.g.

input : ‘//hostname/a/hostame/x/y/z’ not set => ‘//hostname/a/hostame’ set => ‘//hostname’

matchidx=#idx:

Ignore matches ‘< #idx’, return match ‘== #idx’.

default := 0 # first match

matchlvl=#num:

Increment of match for top node when multiple are in the path.

See common options for details.

matchlvlupward=#num:

Increment of match for top node when multiple are in the path.

See common options for details.

pattern:

Scope and type of match pattern. literal: Literal node by node match

regnode:

Match regular expression for individual nodes, implies no contained ‘os.sep’ and no ‘os.pathsep’

patternlvl:

Defines the level of sub nodes of the search path to be resolved. The value ‘full’ forces a full match of ‘spath’ within a ‘plist’ item.

raw:

Suppress normalization by call of ‘os.path.normpath’.

reverse:

This reverses the resulting search order from bottom-up to top-down. Takes effect on ‘redundant’ only.

split:

Returns the split path prefix matched on search list, and the relative sub path outside search list.

Returns:

When successful returns a path, else None.

Returns by default the expanded pathname including the searched.

Raises:

passed through exceptions:

getTopFromPathStringIter¶

filesysobjects.FileSysObjects.getTopFromPathStringIter(spath, plist=None, **kargs)[source]¶

Iterates all matches in plist,see getTopFromPathString.

normpathX¶

filesysobjects.FileSysObjects.normpathX(p, **kargs)[source]¶
Normalize path similar to os.path.normpath() function, but with optional extensions. This provides a simple interface for preparing valid pathnames across multiple platforms. The ‘normpathX’ processes a single path, when multiple are required either split them before or use ‘splitPathVar’ to do.

Considers all os.sep and os.pathsep of supported OS as valid path characters and treats them as reserved characters. In advance of os.path.normpath this supports the following simple URIs for file system paths:

smb, cifs, file [see normpathX].

The normpathX is in particular e.g. aware of special characters of the ‘re’ module like ‘\a’ and ‘\b’. These are masked by the escapeFilePath and unescapeFilePath functions.
Args:
p: A single path entry - no valid ‘os.pathsep’. In case of required

search path including semantic ‘os.pathsep’ use ‘splitPathVar()’.

**kargs:
tpf: Target platform for the file pathname.
Due to some deviations from the expected behavior in case of cross platform development the following options are defined:

tpf compatible behaviour

cnp to POSIX as os.path.normpath() on POSIX OS, transforms to ‘/’

cnw to Win as os.path.normpath() on Windows OS, transforms to ‘\’

keep n.a. keeps literal

local yes compatible to local os.path.normpath()

posix no keeps all separators as ‘/’

win no keeps all separators as ‘\’

<default> no adapts ‘win’ or ‘posix’ to local os

cnp: cross native posix
Emulates compatible native behavior of ‘os.path.normpath’ on POSIX platforms(Linux, MacOS, ...), e.g.:
d:/  => d:
d:\  => d:\
On POSIX OS:
Xposix = os.path.normpath(x)
On POSIX OS:
Xposix == filesysobjects.FileSysObjects(x,**{'tpf':'local'})
Xposix == filesysobjects.FileSysObjects(x,**{'tpf':'cnp'})
On Windows OS:
Xposix == filesysobjects.FileSysObjects(x,**{'tpf':'cnp'})
cnw:cross native win
Emulates compatible native behavior of ‘os.path.normpath’ on Windows platforms, e.g.:
d:/  => d:\
d:\  => d:\
On Windows OS:
Xwin = os.path.normpath(x)
On Windows OS:
Xwin == filesysobjects.FileSysObjects(x,**{'tpf':'local'})
Xwin == filesysobjects.FileSysObjects(x,**{'tpf':'cnw'})
On POSIX OS:
Xwin == filesysobjects.FileSysObjects(x,**{'tpf':'cnw'})
keep:

Keeps as provided, also intermixed styles of os.path.sep and os.pathsep, just escapes ‘’. The user is responsible for adaption to the local file system interfaces.

local:

Compatible to local ‘os.path.normpath()’.

posix: POSIX style
POSIX based style with os.sep = ‘/’. E.g. Linux, MacOS, BSD, Solaris, etc.:
d:/  => d:/
d:\  => d:/
This mode is literally compatible across all supported platforms.
win: Windows style
MS-Windows style with os.sep= ‘\’.:
d:/  => d:\
d:\  => d:\
This mode is literally compatible across all supported platforms.
else:

Adapt ‘os.path.sep’ and ‘os.pathsep’ to local native os, else wise the same behavior as the modes ‘posix’ or ‘win’.

This mode is in term of the structure including drives of Windows based file systems compatible across all supported platforms. But the os.path.sep, and the os.pathsep are adapted to the local platform.
Returns:

Normalized path.

setUpperTreeSearchPath¶

filesysobjects.FileSysObjects.setUpperTreeSearchPath(start=None, top=None, plist=None, **kargs)[source]¶
Extends the ‘plist’ based search by each subdirectory from ‘start’ on upward to ‘top’.

Prepends a set of search paths into plist. The set of search paths contains of each directory beginning with provided start position. The inserted path is normalized by default
Args:
start:

Start components of a path string. See common options for details. Valid scope types:

literal : X

re : -

blob : -

default := caller-file-position.

top:

End component of a path string. The node ‘top’ is included. Valid scope types:

literal : X

re : -

blob : -

default := <same-as-start>

plist:

List to for the storage. See common options for details.

default := sys.path

**kargs:
append:

Appends the set of search paths.

ias:

Treats for local file names any number of subsequent ‘/’ only as one.

matchidx=#idx:

Ignore matches ‘< #idx’, adds match ‘== #idx’ and returns.

default := 0 # all

matchcnt=#num:

The maximal number of matches returned when multiple occur.

matchlvl=#num:

Increment of match for top node when multiple are in the path.

See common options for details.

matchlvlupward=#num:

Increment of match for top node when multiple are in the path.

See common options for details.

noTypeCheck:
Suppress required identical types of ‘top’ and ‘start’. As a rule of thumb for current version, the search component has to be less restrictive typed than the searched. The default applicable type matches are:
 top    ¦ start
--------+---------------------
 LFSYS  ¦ LFSYS, LDSYS, SHARE
        | SMB, CIFS, IAS
 LDSYS  ¦ LDSYS
 SHARE  ¦ SHARE
 SMB    ¦ SMB
 CIFS   ¦ CIFS
 IAS    ¦ IAS
See common options for details.
prepend:

Prepends the set of search paths. This is default.

raw:

Suppress normalization by call of ‘os.path.normpath’.

relonly:

The paths are inserted relative to the top node only. This is mainly for test purposes. The intermix of relative and absolute path entries is not verified.

reverse:

This reverses the resulting search order

from bottom-up to top-down.

unique:

Insert non-present only, else present entries are not checked, thus the search order is changed in general for ‘prepend’, while for ‘append’ the present still covers the new entry.
Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception.

Raises:

passed through exceptions:

splitAppPrefix¶

filesysobjects.FileSysObjects.splitAppPrefix(apstr, **kargs)[source]¶
Splits application prefix from a single resource-path - IEEE-1003.1/SMB/CIFS. For a search path containing mulltiple single-path entries refer to ‘splitPathVar’ [see].

The supported application type prefixes are as follows, for detailed information refer to the chapter Syntax Elements [details]
PREFIXKEY := (
     'file:///' + 2SEP + SPECIALNODE + varSEP + share-name
   | 'file://'
   | 'smb://' + SPECIALNODE + varSEP + share-name
   | 2SEP  SPECIALNODE  [varSEP]
   | nSEP
)
SPECIALNODE := (
   <networknode>
 )
 share-name := (
   1*80pchar  # see [MS-DTYP]
 )
Args:
apstr:

A path containing an application part.

**kargs:
ias:
This option is foreseen for test purposes.

Ignore application separator, this normalizes the pathname by eliminating the PREFIXKEY property. This results in consequence into:
file://///hostname/share/a/b => /hostname/share/a/b
where the pathname is created without any application recognition on the base of:
/////hostname/share/a/b => /hostname/share/a/b
raw:

Displays the type prefix as provided raw sub string - see ‘rtype’ - and suppress any normalization of the resulting ‘pathname’ sub string.

rtype:

Displays the type prefix as provided raw sub string.

tpf:

Target platform for the file pathname, for details refer to ‘normpathX’ [see normpathX].
Returns:
When split successful returns a tuple containing:
(TYPE, host-name, share-name, pathname)

  TYPE := (RAW|CIFS|SMB|SHARE|LFSYS|LDSYS)
     SMB := ('file:///'+2SEP|'smb://')
     SHARE := 2SEP
     LFSYS := ('file://'|'')
     LDSYS := [a-z]':'
  host-name =: (host-name|'')
  share-name := (valid-share-name|'')
  valid-share-name := (
       raw
     | smb-share-name
     | cifs-share-name
     | win-drive-share-name
     | win-drive-os
     | win-special-share-name
  )
  pathname := "pathname on target"
else:
(LFSYS, '', '', apstr)
REMARK: The hostname may contain in current release

any suboption, but is not tested with options at all.
Raises:

passed through exceptions:

splitPathVar¶

filesysobjects.FileSysObjects.splitPathVar(pathvar, **kargs)[source]¶
Splits PATH variables which may include URI type prefixes into a list of single path entries.

The default behavior is to split a search path into a list of contained path entries. E.g:
p = 'file:///a/b/c:/d/e::x/y:smb://host/share/q/w'
Is split into:
px = [
    'file:///a/b/c',
    '/d/e',
    '',
    'x/y',
    'smb://host/share/q/w',
]
With ‘appsplit’
px = [
    ('LFSYS' ,'', '', '/a/b/c'),
    ('LFSYS' ,'', '', '/d/e'),
    ('LFSYS' ,'', '', ''),
    ('LFSYS' ,'', '', 'x/y'),
    ('SMB' ,'host', 'share', 'q/w'),
]
The supported application type prefixes are the same as by ‘splitAppPrefix’ [see]. For reserved prefix keywords parts of the name may be escaped in order to avoid filtering when required.
Args:

pathvar:

A search path compatible to ‘splitAppPrefix’ [see].

**kargs:

The provided key-options are also passed through transparently to

‘normpathX’ [see] if not ‘raw’.

appsplit:

Splits into tuples of application entries.

For further details refer to ‘splitAppPrefix’ [see].

appsplit = (True|False)

ias:

This option is foreseen for test purposes.

For further details refer to ‘splitAppPrefix’ [see].

ias = (True|False)

raw:

Displays a list of unaltered split path items, superposes ‘rpath’ and ‘rtype’.

raw = (True|False)

rpath:

Displays the path as provided raw sub string.

For further details refer to ‘splitAppPrefix’ [see].

rpath = (True|False)

rtype:

Displays the type prefix as provided raw sub string.

For further details refer to ‘splitAppPrefix’ [see].

rtype = (True|False)

tpf:

Target platform for the file pathname, for details refer to ‘normpathX’ [see normpathX].

Returns:
When split successful returns a list of tuples:
appsplit == False

    [
        <pathvar-item>,
        ...
    ]

appsplit == True

    [
        (TYPE, host-name, share-name, pathname),
        ...
    ]
The tuple contains:
(TYPE, host-name, share-name, pathname)

  TYPE := (RAW|CIFS|SMB|SHARE|LFSYS|LDSYS)
     RAW := (<raw-pathvar-item>)
     CIFS := ('cifs://')
     SMB := ('file:///'+2SEP|'smb://')
     SHARE := 2SEP
     LFSYS := ('file://'|'')
     LDSYS := [a-z]':'
  host-name := (host-name|'')
  share-name := (valid-share-name|'')
  valid-share-name := (
     smb-share-name
     | cifs-share-name
     | win-drive-share-name
     | win-drive-os
     | win-special-share-name
  )
  pathname := "pathname on target"

specials:

  ias := ('IAS', '', '', ias-pathvar-item)
  raw := ('RAW', '', '', raw-pathvar-item)
  rpath := (TYPE, host-name, share-name, raw-pathname>)
  rtype := (raw-type, host-name, share-name, pathname)
  rtype + rpath := (raw-type, host-name, share-name, raw-pathname>)
else:
(LFSYS, '', '', apstr)
REMARK: The hostname may contain in current release

any suboption, but is not tested with options at all.
Refer also to ‘splitAppPrefix’ [see].
Raises:

passed through exceptions:

unescapeFilePath¶

filesysobjects.FileSysObjects.unescapeFilePath(p)[source]¶

Unescape ‘os.pathsep’ by re module.

Considers any os.pathsep as path part. Manages os.pathsep only - thus flags of the re module itself are not considered and may omitted in regular expressions for pathnames, e.g. ‘a’, ‘b’, and ‘x’.

Args:

p: A path.

Returns:

Path.

Exceptions¶

FileSysObjectsException¶

class filesysobjects.FileSysObjects.FileSysObjectsException[source]¶