Contents

• browse
• codecs
• config
• console
• decorators
• dependencies
• doc
• easytest
• logging_tools
• multisetup
• Package
• paths
• progressbar
• sphinx themes
• tools
• url
• multicore

1. browse

Universal browser

This module provides a browser in 2 flavours: as a program to use in a Terminal, or as a Python function that can be used in other software. The underlying code is based on the standard python module webbrowser. With webbrowser module itself, you can already open a URL as follows in a command line interface:

python -m webbrowser -t "http://www.python.org"

However, with browse, you can simply type:

browse http://www.python.org

It does not seem to be a big improvments but it is a bit more flexible. First, there is no need to enter “http://” : it will be added if missing and if this is not a local file.:

browse docs.python.org
browse http://docs.python.org --verbose

Similarly, you can open an image (it uses the default image viewer):

browse image.png

Or a txt file (or any document provided there is a default executable to open it). It works like a charm under Linux. Under MAC, it uses the open command so this should also work.

When invoking browse, under MacOSX, it actually tries to call open first and then calls webbrowser, if unsuccessful only. Note tested under Windows but uses webbrowser is used and works for open HTML document and URLs.

You can also look at a directory (starts nautilus under Fedora):

browse ~/Pictures

See more examples below.

The interest of browse is that it can also be used programmatically:

from easydev.browser import browse
# open an image with the default image viewer:
browse("image.png")
# or a web page
browse("http://www.uniprot.org")

There is also an alias onweb:

from easydev import onweb

2. codecs

various type convertors (e.g., list to string)

to_list(data, verbose=True)

Transform an object into a list if possible

Parameters:data – a list, tuple, or simple type (e.g. int) Returns:a list

• if the object is list or tuple, do nothing
• if the object is not a list, we assume this is a primitive type and a list of length 1 is returned, the item being the parameter provided.

>>> from easydev import transform_to_list
>>> to_list(1)
[1]
>>> to_list([1,2])
[1,2]

transform_into_list(data)

deprecated. use to_list()

list2string(data, sep=', ', space=True)

Transform a list into a string

Parameters:

• data (list) – list of items that have a string representation. the input data could also be a simple object, in which case it is simply returned with a cast into a string
• sep (str) – the separator to be use

>>> list2string([1, 2]
"1, 2"
>>> list2string([1, 2], sep=;)
"1; 2"
>>> list2string([1, 2], sep=;, space=False)
"1;2"
>>> list2string(1, sep=;, space=False)
"1"

Note

the cast is performed with str()

3. config

class CustomConfig(name, verbose=False)

Base class to manipulate a config directory

user_config_dir

return directory of this configuration file

class DynamicConfigParser(config_or_filename=None, *args, **kargs)

Enhanced version of Config Parser

Provide some aliases to the original ConfigParser class and new methods such as save() to save the config object in a file.

>>> from easydev.config_tools import ConfigExample
>>> standard_config_file = ConfigExample().config
>>> c = DynamicConfigParser(standard_config_file)
>>>
>>> # then to get the sections, simply type as you would normally do with ConfigParser
>>> c.sections()
>>> # or for the options of a specific sections:
>>> c.get_options('General')

You can now also directly access to an option as follows:

>>> c.General.tag

Then, you can add or remove sections (remove_section(), add_section()), or option from a section remove_option(). You can save the instance into a file or print it:

>>> print(c)

Warning

if you set options manually (e.G. self.GA.test =1 if GA is a section and test one of its options), then the save/write does not work at the moment even though if you typoe self.GA.test, it has the correct value

Methods inherited from ConfigParser are available:

# set value of an option in a section
c.set(section, option, value=None)
# but with this class, you can also use the attribute
c.section.option = value

# set value of an option in a section
c.remove_option(section, option)
c.remove_section(section)

add_option(section, option, value=None)

add an option to an existing section (with a value)

>>> c = DynamicConfigParser()
>>> c.add_section("general")
>>> c.add_option("general", "verbose", True)

get_options(section)

Alias to get all options of a section in a dictionary

One would normally need to extra each option manually:

for option in config.options(section):
    config.get(section, option, raw=True)#

then, populate a dictionary and finally take care of the types.

Warning

types may be changed .For instance the string “True” is interepreted as a True boolean.

See also

internally, this method uses section2dict()

read(filename)

Load a new config from a filename (remove all previous sections)

save(filename)

Save all sections/options to a file.

Parameters:filename (str) – a valid filename

config = ConfigParams('config.ini') #doctest: +SKIP
config.save('config2.ini') #doctest: +SKIP

section2dict(section)

utility that extract options of a ConfigParser section into a dictionary

Parameters:

• config (ConfigParser) – a ConfigParser instance
• section (str) – the section to extract

Returns:

a dictionary where key/value contains all the options/values of the section required

Let us build up a standard config file:

>>> import ConfigParser
>>> c = ConfigParser.ConfigParser()
>>> c.add_section('general')
>>> c.set('general', 'step', str(1))
>>> c.set('general', 'verbose', 'True')

To access to the step options, you would write:

>>> c.get('general', 'step')

this function returns a dictionary that may be manipulated as follows:

>>> d_dict.general.step

Note

a value (string) found to be True, Yes, true, yes is transformed to True

Note

a value (string) found to be False, No, no, false is transformed to False

Note

a value (string) found to be None; none, “” (empty string) is set to None

Note

an integer is cast into an int

class ConfigExample

Create a simple example of ConfigParser instance to play with

>>> from easydev.pipeline.config import ConfigExample
>>> c = ConfigExample().config  # The ConfigParser instance
>>> assert 'General' in c.sections()
>>> assert 'GA' in c.sections()

This example builds up a ConfigParser instance from scratch.

This is equivalent to having the following input file:

[General]
verbose = True
tag = test

[GA]
popsize = 1

which can be read with ConfigParser as follows:

>>> from ConfigParser import ConfigParser
>>> config = ConfigParser()
>>> config.read("file.ini")

load_configfile(configpath)

Loads a JSON or YAML configfile as a dict.

4. console

Format colored consoled output. Modified from sphinx.util.console

from easydev.console import *
print(red("test"))

color_terminal()

Does terminal allows coloring

Returns:boolean

get_terminal_width()

Returns the current terminal width

term_width_line(text)

prints pruned version of the input text (limited to terminal width)

Parameters:text (str) – Return str text:  

darkblue(text)

turquoise(text)

red(text)

black(text)

teal(text)

lightgray(text)

standout(text)

white(text)

bold(text)

purple(text)

blue(text)

reset(text)

blink(text)

faint(text)

green(text)

yellow(text)

fuchsia(text)

darkred(text)

brown(text)

darkgreen(text)

underline(text)

darkgray(text)

5. decorators

Handy decorators

ifpylab(func)

check if pandas is available. If so, just return the function, otherwise returns dumming function that does nothing

requires(requires, msg='')

Decorator for class method to check if an attribute exist

>>> from easydev.decorators import requires
>>> class Test(object):
...     def __init__(self):
...         self.m = 1
...         self.x = 1
...     @requires(['m','x'], "set the m attribute first")
...     def printthis(self):
...         print(self.m+self.x)
>>> t = Test()
>>> t.printthis()
2

ifpandas(func)

check if pandas is available. If so, just return the function, otherwise returns dumming function that does nothing

6. dependencies

get_dependencies(pkgname)

Return dependencies of a package as a sorted list

Parameters:pkgname (str) – package name Returns:list (empty list if no dependencies)

7. doc

Module related to documentation

underline(text, symbol='=')

Underlines a string with a specific character

Parameters:

• text (str) – the text to underline
• symbol (str) – the character to be used to underline the text

Returns:

underlined text.

>>> print(underline("test"))
test
====

8. easytest

assert_list_almost_equal(first, second, places=None)

Combined version nose.tools.assert_almost_equal and assert_list_equal

This function checks that 2 lists contain identical items. The equality between pair of items is checked with assert_almost_equal function, which means you can check for the places argument

Note

there may be already some tools to check that either in nosetests or unittest but could not find.

>>> from easydev.easytest import assert_list_almost_equal as assert_list
>>> assert_list([0,0,1], [0,0,0.9999], places=3)

trysetattr(this, attrname, value, possible)

A common test pattern: try to set a non-writable attribute

class A(object):
    def __init__(self):
        self._a = 1
        self._b = 2
    def _get_a(self):
        return self._a
    def _set_a(self, value):
        self._a = value
    a = property(_get_a, _get_b)
    def _get_b(self):
        return self._b
    b = property(_get_b)

>>> o = A()
>>> trysetattr(A, "a", 1, possible=True)
>>> trysetattr(A, "b", 1, False)
AssertionError

class TempFile(suffix='')

A small wrapper around tempfile.NamedTemporaryFile function

f = TempFile(suffix="csv")
f.name
f.delete() # alias to delete=False and close() calls

delete()

name

9. logging_tools

class Logging(level)

logging utility.

When using the logging utility, it works like a singleton. So, once logging level is set, you cannot set it again easily. Here is a class that allows to do that.

Warning

this is a bit of a hack. Maybe this is not a proper solution but it seems to do the job.

>>> l = Logging("INFO")
>>> l.info("test")
INFO:root:test
>>> l.level = "WARNING"
>>> l.info("test")

constructor

Parameters:level (str) – valid levels are [“INFO”, “DEBUG”, “WARNING”, “CRITICAL”, “ERROR”]. If set to True, level is internally set to INFO. If set to False, level is seet internally to ERROR.

debugLevel

Read/Write access to the debug level. Must be one of INFO, DEBUG, WARNING, CRITICAL, ERROR

level

alias to debugLevel (Read-only access)

10. multisetup

Calling setup.py recursively and/or in multi python packages.

The commands are similar to those expected by setup.py. In addition, there are a few commands dedicated to multisetup (see –help).

Example:

>>> python multisetup install --quiet
>>> python multisetup install sdist --dist-dir ../dist
>>> python multisetup --keep-going install sdist --dist-dir ../dist

Based on OpenAlea.Misc http://openalea.gforge.inria.fr

class Multisetup(commands, packages=None, curdir='.', verbose=True)

The main base class to build Multisetup instances

In practice, you create a python script with this kind of code:

if __name__ == "__main__"
    from easydev.multisetup import Multisetup
    import sys
    packages = ['pkg1', 'pkg2']
    mysetup = Multisetup(commands=sys.argv[1:], packages=packages)
    mysetup.run()

Constructor

Parameters:

• commands (a string or list of strings) – list of user commands or command (see parse_commands()) accepted commands are –packages, –exclude-packages, -quiet, –keep-going
• packages (list) – list of packages to process
• curdir (str) – current directory default is .
• verbose (bool) – verbose option

The argument commands must be a list of strings combining arguments from multisetup and setup.

Examples:

>>> Multisetup("install --keep-going", ['pkg1', 'pkg2'], '.', verbose=True)
>>> Multisetup(["install","--keep-going"], ['pkg1', 'pkg2'], '.', verbose=True)

classmethod help()

help: to get more help and usage

parse_commands()

Search and remove multisetup options

Get the user command line arguments (self.commands) that are dedicated to multisetup such as –help, –quiet, –keep-going so that the remaining commands are fully comptatible with setuptools.

parse_packages()

Search and remove package(S) from multisetup command (e.g., –package)

Todo

known issue: python multisetup.py –packages with two packages will be confused by following commands. Must be put at the end of the command

run(color=True)

Executes ‘python setup.py’ with the user commands on all packages.

11. Package

class PackageBuilder(name, share=True, namespace=None)

simple class to automatically build a package layout

>>> from easydev import PackageBuilder
>>> p = PackageBuilder(name="testpackage")
>>> p.buildPackage()

For the time being, this example creates the following layout:

pkg/
|-- doc
|   |-- source
|   |-- _static
|-- README.txt
|-- setup.py
|-- share
|     |-- data
|-- src
|     |-- pkg
|         |-- __init__.py
|-- test

You can avoid the shared directory creation. The namespace is not implemented so far. The doc directory is empty so far.

The version is set to 0.0.1

Metainfo in the setup file need to be filed but the package should already be functional.

New modules can be added inside the src/pkgname directory.

Constructor

Parameters:

• name (str) –
• share (str) –

buildPackage(force=False)

Builds the entire package

This function removes the directory “pkgname” if it exists, to create it back (empty), and then calsl the methods starting with “create” word.

create_namespace()

create_readme()

Creates a README file

create_setup()

Creates a setup.py file

create_share_directory()

Create share/data directory if required

create_sphinx_directory()

Create layout for sphinx documentation

create_src_directory()

Create a source directory and empty __init__ file

create_test_directory()

Create a test directory

init(force=False)

12. paths

Utilities to ease access to share data paths

get_shared_directory_path(package)

Returns the share directory path of an installed package

sharedir = get_shared_directory_path("easydev")

get_shared_directories(package, datadir='data')

Returns all directory paths found in the package share/datadir directory

Parameters:datadir (str) – scans package/share/<datadir> where datadir is “data” by default. If it does not exists, the list returned is empty.

>>> from easydev import get_shared_directories
>>> shared_directories = get_shared_directories("easydev", "themes")
>>> len(shared_directories)>=2
True

get_share_file(package, datadir, filename)

Creates the full path of a file to be found in the share directory of a package

gsf(package, datadir, filename)

get_package_location(package)

Return physical location of a package

13. progressbar

A progress bar copied and adapted from pyMC code (dec 2014)

progress_bar(iters, interval=None)

A progress bar for Python/IPython/IPython notebook

Parameters:

• iters (int) – number of iterations (steps in the loop)
• interval – number of intervals to use to update the progress bar (20 by default)

from easydev import progress_bar
pb = progress_bar(10)
for i in range(1,10):
    import time
    time.sleep(0.1)
    pb.animate(i)

class TextProgressBar(iterations, printer, width=40, interval=None)

Use Progress

animate(i, dummy=None)

bar(percent)

elapsed

progbar(i)

class Progress(iters, interval=None)

Generic progress bar for python, IPython, IPython notebook

from easydev import Progress
pb = Progress(100, interval=1)
pb.animate(10)

animate(i)

elapsed

14. sphinx themes

Common tools to ease access to a easydev sphinx themes.

get_path_sphinx_themes()

Returns the path where the sphinx themes can be found

>>> from easydev import sphinx_themes
>>> themes_path = sphinx_themes.get_path_sphinx_themes()

get_sphinx_themes()

Returns the sphinx themes found in easydev

>>> from easydev import sphinx_themes
>>> themes = sphinx_themes.get_sphinx_themes()
>>> "standard" in themes
True

15. tools

toolkit to ease development

shellcmd(cmd, show=False, verbose=False, ignore_errors=False)

An alias to run system commands.

Based on subprocess.Popen.

Parameters:

• cmd (str) – the command to call
• show (bool) – print the command
• verbose (bool) – print the output

Returns:

the output as a string

checkParam(param, valid_values)

Warning

deprecated since 0.6.10 use check_param_in_list() instead

swapdict(dic, check_ambiguity=True)

Swap keys for values in a dictionary

>>> d = {'a':1}
>>> swapdict(d)
{1:'a'}

check_param_in_list(param, valid_values, name=None)

Checks that the value of param is amongst valid

Parameters:

• param – a parameter to be checked
• valid_values (list) – a list of values

check_param_in_list(1, [1,2,3])
check_param_in_list(mode, ["on", "off"])

check_range(value, a, b, strict=False)

Check that a value lies in a given range

Parameters:

• value – value to test
• a – lower bound
• b – upper bound

Returns:

nothing

>>> from easydev.tools import check_range
>>> check_range(1,0, 2)

precision(data, digit=2)

Return the value with only 2 digits

>>> precision(2.123)
2.12
>>> precision(2123, digit=-2)
2100

class AttrDict(**kwargs)

dictionary-like object that exposes its keys as attributes.

When you have dictionary of dictionaries with many levels e.g.:

d = {'a': {'a1': {'a2': 2}}}

to get/set a values, one has to type something like:

d['a']['a1']['a2'] = 3

The AttrDict allows the dictionary to work as attributes:

ad = AttrDict(**d)
ad.a.a1.a2 = 3

You can now add values as attribute, or with [‘key’] syntax

>>> from easydev import AttrDict
>>> a = AttrDict('value': 1)
>>> a.value
1
>>>
>>> a.unit = 'meter'
>>> a.keys()
['value', 'meter']

If you need to add new simple values after the creation of the instance, just use the setter:

>>> d['newa'] = 2
>>> d.newa = 2  # equivalent to the statement above

but if you want to set a dictionary (whichever recursive level), use the update() method:

>>> d.update({'newd': {'g': {'h':2}}})
>>> d.newd.g.h
2

Note that if you use the setter for a value that is a dictionary, e.g.:

ad.a = {'b':1}

then a is indeed a dictionary.

from_json(filename)

does not remove existing keys put replace them if already present

to_json(filename=None)

update(content)

See class/constructor documentation for details

Parameters:content (dict) – a valid dictionary

class DevTools

Aggregate of easydev.tools functions.

check_exists(filename)

Raise error message if the file does not exists

check_param_in_list(param, valid_values)

wrapper around easydev.check_param_in_list()

check_range(value, a, b)

wrapper around easydev.check_range()

list2string(query, sep=', ', space=False)

mkdir(dirname)

Create a directory if it does not exists; pass without error otherwise

shellcmd(cmd, show=False, verbose=False, ignore_errors=False)

See shellcmd()

swapdict(d)

wrapper around easydev.swapdict()

to_json(dictionary)

Transform a dictionary to a json object

to_list(query)

Cast to a list if possible

‘a’ ->[‘a’] 1 -> [1]

tolist(query)

execute(cmd, showcmd=True, verbose=True)

touch(fname, times=None)

16. url

Utilities related to the web

isurl_reachable(url, timeout=10)

Checks if an URL exists or nor

Parameters:url (str) – the url to look for Returns:True if it exists

17. multicore

class MultiProcessing(maxcpu=None, verbose=False, progress=True)

Class to run jobs in an asynchronous manner.

You would use this class to run several jobs on a local computer that has several cpus.

t = MultiProcessing(maxcpu=2)
t.add_job(func, func_args)
t.run()
t.results[0] # contain returned object from the function *func*.

Warning

the function must be a function, not a method. This is inherent to multiprocess in the multiprocessing module.

Warning

the order in the results list may not be the same as the list of jobs. see run() for details

Parameters:

• maxcpu – default returned by multiprocessing.cpu_count()
• verbose – print the output of each job. Could be very verbose so we advice to keep it False.
• progress – shows the progress

add_job(func, *args, **kargs)

add a job in the pool

reset()

remove joves and results

run(delay=0.1)

Run all the jobs in the Pool until all have finished.

Jobs that have been added to the job list in add_job() are now processed in this method by using a Pool. Here, we add all jobs using the apply_async method from multiprocess module.

In order to ensure that the jobs are run sequentially in the same order as in jobs, we introduce a delay between 2 calls to apply_async (see http://docs.python.org/2/library/multiprocessing.html)

A better way may be t use a Manager but for now, this works.