The cli package is a framework for making simple, correct command line applications in Python. With cli, you can quickly add standard command line parsing; logging; unit and functional testing; and profiling to your CLI apps. To make it easier to do the right thing, cli wraps all of these tools into a single, consistent application interface.
You can install the latest stable version of cli using pip:
$ pip install pyCLI
Public repositories for the project are hosted at github, so you can use either git to get a copy of the project’s code and history:
$ git clone git://github.com/whilp/cli.git
If you notice a problem with cli, please report it using the github issue tracker (or, if you have a fix, send a pull request).
cli is developed along two branches. The first, ‘default’ (or ‘master’ in git) contains new features and possible bugs – this branch is the active development branch. The second, ‘stable’, contains releases both major and minor as well as bugfixes. If you’d like to help improve cli, take a look at default/master. Otherwise, stick with stable.
Command line parsing:
#!/usr/bin/env python
import cli.app
@cli.app.CommandLineApp
def ls(app):
pass
ls.add_param("-l", "--long", help="list in long format", default=False, action="store_true")
if __name__ == "__main__":
ls.run()
When run, this script produces the following output:
$ python ls.py -h
usage: ls [-h] [-l]
optional arguments:
-h, --help show this help message and exit
-l, --long list in long format
Logging:
#!/usr/bin/env python
import time
import cli.log
@cli.log.LoggingApp
def sleep(app):
app.log.debug("About to sleep for %d seconds" % app.params.seconds)
time.sleep(app.params.seconds)
sleep.add_param("seconds", help="time to sleep", default=1, type=int)
if __name__ == "__main__":
sleep.run()
Which produces the following:
$ python sleep.py -h
usage: sleep [-h] [-l LOGFILE] [-q] [-s] [-v] seconds
positional arguments:
seconds time to sleep
optional arguments:
-h, --help show this help message and exit
-l LOGFILE, --logfile LOGFILE
log to file (default: log to stdout)
-q, --quiet decrease the verbosity
-s, --silent only log warnings
-v, --verbose raise the verbosity
$ python sleep.py -vv 3
About to sleep for 3 seconds
Daemonizing:
#!/usr/bin/env python
import cli.daemon
@cli.daemon.DaemonizingApp
def daemon(app):
if app.params.daemonize:
app.log.info("About to daemonize")
app.daemonize()
if __name__ == "__main__":
daemon.run()
And on the command line:
$ python daemon.py -h
usage: daemon [-h] [-l LOGFILE] [-q] [-s] [-v] [-d] [-u USER] [-p PIDFILE]
optional arguments:
-h, --help show this help message and exit
-l LOGFILE, --logfile LOGFILE
log to file (default: log to stdout)
-q, --quiet decrease the verbosity
-s, --silent only log warnings
-v, --verbose raise the verbosity
-d, --daemonize run the application in the background
-u USER, --user USER change to USER[:GROUP] after daemonizing
-p PIDFILE, --pidfile PIDFILE
write PID to PIDFILE after daemonizing
$ python daemon.py -d -vv
About to daemonize
While the cli modules provide a simple API for designing your own applications, the default implementations are intended to be flexible enough to cover most use cases. No matter which cli.app.Application you use, the basic pattern is the same: create a callable that does the work, wrap it in an cli.app.Application, add some parameters and call its run() method.
Your callable may be a simple function or a more complex class that implements the __call__() protocol. Either way, it should accept a single app instance as its only argument. It will use this object to interact with the application framework, find out what arguments were passed on the command line, log messages, etc.
You can wrap the callable in one of two ways. First, cli.app.Application can be thought of as a decorator (see PEP 318 for more information). For example:
@cli.app.Application
def yourapp(app):
do_stuff()
If you need to pass keyword arguments to the application, you can still use the decorator pattern:
@cli.app.CommandLineApp(argv=["-v"])
def yourapp(app):
do_stuff()
If you don’t like decorators (or your interpreter doesn’t support them), you can also simply pass your application callable to the cli.app.Application:
def yourapp(app):
do_stuff()
yourapp = cli.app.CommandLineApp(yourapp)
Some more complex scripts and applications may benefit from subclassing the cli.app.Application class itself. This approach can help make your code more reusable:
class YourApp(cli.app.CommandLineApp):
def main(self):
do_stuff()
When subclassing cli.app.Application, you’ll likely want to incorporate functionality from the other application classes (like cli.app.CommandLineApp). To do this, simply call methods from the appropriate mixin classes (like cli.app.CommandLineMixin) – in fact, this is how the application classes themselves work.
Most of the supplied cli.app.Application implementations support parameters. Parameters determine how your users interact with your program on the command line. To add parameters to your application, call add_param() after you’ve wrapped your callable (or in its setup() method):
yourapp.add_param("-v", "--verbose", default=0, action="count",
help="increase the verbosity", )
The interface here is the same as that implemented by argparse.ArgumentParser. In this case, an verbose attribute will be created on the app.params object with an integer representing the desired verbosity level.
Once you’ve added all the parameters you need (if any – the default implementations include sensible defaults), simply call the run() method on the wrapped callable. It’s best to do this only if your script is actually being run, so shield it with a conditional:
if __name__ == "__main__":
yourapp.run()
This will allow you to import your application and tweak it programmatically from another script without actually invoking it.
The following projects are known to use cli; please email willmaier@ml1.net if you’d like to see your project listed here.
cli is designed to encourage a simple set of best practices in script development. Following this simple guide will help you write robust, maintainable and user-friendly scripts.
A standard and clear coding style will make it easier for you to read your scripts months or years after you first wrote them. A good style will also help other people quickly understand your code so that they can use it, too and (hopefully) help you improve it.
Since you’re writing your scripts in Python, it is strongly recommended that you follow the coding style described in PEP 8. This style is easy to read and will be recognized by most people familiar with Python.
Monolithic code is hard to read. By breaking your script into functions, classes and maybe even separate modules, you help the reader (including your future self) navigate the code. Modular code can also be easily reused so that you don’t have to continually reimplement useful functions in all of your scripts.
When writing or refactoring your code to make it more modular, a good rule of thumb is to break large functions or methods into chunks that fit on a single screen (perhaps up to 50 lines of code or so). These chunks are large enough to get the job done but small enough for your reader to fully comprehend them.
In most cases, you can make your code more robust by adding tests to verify specific algorithms or even the macro behavior of the entire script. For testing small chunks of your code (unit tests), cli provides cli.test.AppTest, a unittest.TestCase with a few customizations to make it more useful for testing command line applications. To test larger behavior (functional tests), use cli.test.FunctionalTest. cli itself uses both of these classes for its own tests; for examples, see the tests/functional/ directory shipped with the cli package.
When packaging your scripts, use Titus’ SomePackage as a guide for integrating your tests.
The cli.app module establishes the basis for all of the other applications and is a good place to start when looking to extend Application functionality or to understand the basic API.
Bases: object
An application.
Application constructors should always be called with keyword arguments, though the main argument may be passed positionally (as when Application or its subclasses are instantiated as decorators). Arguments are:
main is the callable object that performs the main work of the application. The callable must accept an Application instance as its sole argument. If main is None, it is assumed that a valid callable will be passed to the __call__() method (when using an Application instance as a decorator). If main is not None, the setup() method will be called, allowing subclasses to customize the order in which certain setup steps are executed.
name is the name of the application itself. If name is not None, the name property will inspect the main callable and use its function or class name.
exit_after_main determines whether the application will call sys.exit() after main completes.
stdin, stderr and stdout are file objects that represent the usual application input and outputs. If they are None, they will be replaced with sys.stdin, sys.stderr and sys.stdout, respectively.
version is a string representing the application’s version.
description is a string describing the application. If description is None, the description property will use the main callable’s __doc__ attribute instead.
argv is a list of strings representing the options passed on the command line. If argv is None, sys.argv will be used instead.
profiler is a cli.profiler.Profiler instance, or None (default). If not None, the profiler will be available to the running application.
reraise is a tuple of exception classes: if an exception is raised by main and it is listed here, then it will be propagated upwards by post_run; otherwise it will just cause post_run to exit with return code 1.
In all but a very few cases, subclasses that override the constructor should call Application.__init__() at the end of the overridden method to ensure that the setup() method is called.
Wrap the main callable and return an Application instance.
This method is useful when it is necessary to pass keyword arguments to the Application constructor when decorating callables. For example:
@cli.Application(stderr=None)
def foo(app):
pass
In this case, setup() will occur during __call__(), not when the Application is first constructed.
A string describing the application.
Unless specified when the Application was created, this property will examine the main callable and use its docstring (__doc__ attribute).
A string identifying the application.
Unless specified when the Application was created, this property will examine the main callable and use its name (__name__ or func_name for classes or functions, respectively).
Clean up after the application.
After main has been called, run() passes the return value (or Exception instance raised) to this method. By default, post_run() decides whether to call sys.exit() (based on the value of the exit_after_main attribute) or pass the value back to run(). Subclasses should probably preserve this behavior.
Perform any last-minute configuration.
The pre_run() method is called by the run() method before main is executed. This is a good time to do things like read a configuration file or parse command line arguments. The base implementation does nothing.
Run the application, returning its return value.
This method first calls pre_run() and then calls main, passing it an instance of the Application itself as its only argument. The return value (or Exception instance raised) is then passed to post_run() which may modify it (or terminate the application entirely).
Configure the Application.
This method is provided so that subclasses can easily customize the configuration process without having to reimplement the base constructor. setup() is called once, either by the base constructor or __call__().
Bases: cli.app.CommandLineMixin, cli.app.Application
A command line application.
This class simply glues together the base Application and CommandLineMixin.
Actual functionality moved to CommandLineMixin.
Bases: object
A command line application.
Command line applications extend the basic Application framework to support command line parsing using the argparse module. As with Application itself, main should be a callable. Other arguments are:
usage is a string describing command line usage of the application. If it is not supplied, argparse will automatically generate a usage statement based on the application’s parameters.
epilog is text appended to the argument descriptions.
The rest of the arguments are passed to the Application constructor.
Add a parameter.
add_param() wraps argparse.ArgumentParser.add_argument(), storing the parameter options in a dictionary. This information can be used later by other subclasses when deciding whether to override parameters.
alias of ArgumentParser
alias of HelpFormatter
The params attribute is an object with attributes containing the values of the parsed command line arguments. Specifically, its an instance of argparse.Namespace, but only the mapping of attributes to argument values should be relied upon.
Parse command line.
During pre_run(), CommandLineMixin calls argparse.ArgumentParser.parse_args(). The results are stored in params.
..versionchanged:: 1.1.1
If argparse.ArgumentParser.parse_args() raises SystemExit but exit_after_main is not True, raise Abort instead.
Configure the CommandLineMixin.
During setup, the application instantiates the argparse.ArgumentParser and adds a version parameter (-V, to avoid clashing with -v verbose).
Update a parameter namespace.
The params instance will be updated with the names and values from newparams and then returned.
Changed in version 1.0.2: update_params() expects and returns argparse.Namespace instances; previously, it took keyword arguments and updated params itself. This is now left to the caller.
Logging applications use the standard library logging module to handle log messages.
Bases: cli.log.LoggingMixin, cli.app.CommandLineMixin, cli.app.Application
A logging application.
This class simply glues together the base Application, LoggingMixin and other mixins that provide necessary functionality.
Changed in version 1.0.4: Actual functionality moved to LoggingMixin.
Bases: object
A mixin for command-line applications that knows how to log.
The LoggingMixin requires cli.app.CommandLineMixin and allows command line configuration of the application logger. In addition to those supported by the standard cli.app.Application and cli.app.CommandLineMixin, arguments are:
stream is an open file object to which the log messages will be written. By default, this is standard output (not standard error, as might be expected).
logfile is the name of a file which will be opened by the logging.FileHandler.
message_format and date_format are passed directly to the CommandLineLogger and are interpreted as in the logging package.
If root is True, the LoggingMixin will make itself the root logger. This means that, for example, code that knows nothing about the LoggingMixin can inherit its verbosity level, formatters and handlers.
Set the verbosity level and configure the logger.
The application passes the params object to the CommandLineLogger‘s special CommandLineLogger.setLevel() method to set the logger’s verbosity and then initializes the logging handlers. If the logfile attribute is not None, it is passed to a logging.FileHandler instance and that is added to the handler list. Otherwise, if the stream attribute is not None, it is passed to a logging.StreamHandler instance and that becomes the main handler.
Configure the LoggingMixin.
This method adds the -l, q, -s and -v parameters to the application and instantiates the log attribute.
Bases: logging.Logger
Provide extra configuration smarts for loggers.
In addition to the powers of a regular logger, a CommandLineLogger can set its verbosity levels based on a populated argparse.Namespace.
An integer representing the default logging level.
Default: logging.WARN (only warning messages will be shown).
Set the logger verbosity level.
ns is an object with verbose, quiet and silent attributes. verbose and quiet may be positive integers or zero; silent is True or False. If silent is True, the logger’s level will be set to silent_level. Otherwise, the difference between quiet and verbose will be multiplied by 10 so it fits on the standard logging scale and then added to default_level.
An integer representing the silent logging level.
Default: logging.CRITICAL (only critical messages will be shown).
Daemonizing applications run in the background, forking themselves off.
Bases: cli.daemon.DaemonizingMixin, cli.log.LoggingMixin, cli.app.CommandLineMixin, cli.app.Application
A daemonizing application.
This class simply glues together the base Application, DaemonizingMixin and other mixins that provide necessary functionality.
Changed in version 1.0.4: Actual functionality moved to DaemonizingMixin.
Bases: object
A command-line application that knows how to daemonize.
The DaemonizingMixin requires the cli.log.LoggingMixin (for it’s not very helpful to daemonize without being able to log messages somewhere). In addition to those supported by the standard cli.app.Application, cli.app.CommandLineMixin and cli.log.LoggingMixin, arguments are:
pidfile is a string pointing to a file where the application will write its process ID after it daemonizes. If it is None, no such file will be created.
chdir is a string pointing to a directory to which the application will change after it daemonizes.
null is a string representing a file that will be opened to replace stdin, stdout and stderr when the application daemonizes. By default, this os.path.devnull.
Run in the background.
daemonize() must be called explicitly by the application when it’s ready to fork into the background. It forks, flushes and replaces stdin, stderr and stdout with the open null file and, if requested on the command line, writes its PID to a file and changes user/group.
Configure the DaemonizingMixin.
This method adds the -d, u, and -p parameters to the application.
The Profiler can help you quickly measure your application’s overall performance or focus on specific sections.
Bases: object
A profiling tool.
The Profiler provides two decorator methods which can help you improve the performance of your code. Arguments are:
stdout is a file-like object into which the profiling report will be written.
If anonymous is True, the profiling decorators will run in-place. Be careful when combining this option with count greater than 1 if the block of code in question may have side effects. This is useful for testing a specific block of code within a larger program; for example:
profiler = Profiler(anonymous=True)
biglist = []
# The profiler will measure the code defined in the anonymous
# block function and then proceed with the rest of the script.
# Note that you must take extra precautions to make sure that
# names defined in the anonymous block are valid outside of its
# scope.
@profiler.deterministic
def block():
global biglist
biglist = range(10**5)
length = len(biglist)
count and repeat control the number of iterations the code in question will be run in the statistical() profiler.
Deterministically evaluate func‘s performance.
func should be a callable. It will be decorated with a simple function that uses the standard library’s profile.Profile to trace and time each step of func‘s execution. After func is profiled, a report will be written to stdout. The profiling statistics are saved to the stats attribute after the run.
Return True if the name seems to be anonymous.
Callables whose names are “anonymous” or that start with “__profiler_” are considered anonymous.
Run func many times, reporting the best run time.
This profiling method wraps func with a decorator that performs repeat runs, executing func count times each run. The profiler will average the execution time for each loop and report the best time on stdout. The result is saved at result – divide this number by count to determine the average execution time.
This profiler is useful for comparing the speed of equivalent implementations of similar algorithms.
Wrap callable wrapped with wrapper.
wrap() calls functools.update_wrapper() (or an equivalent implementation if not available) to preserve the callable’s metadata. If anonymous is True or the wrapped callable’s name is anonymous (see isanon()), the wrapped callable will be executed in-place. Otherwise, the wrapped callable will simply be returned (like a decorator).
This class is used for creating reports from data generated by the Profile class. It is a “friend” of that class, and imports data either by direct access to members of Profile class, or by reading in a dictionary that was emitted (via marshal) from the Profile class.
The big change from the previous Profiler (in terms of raw functionality) is that an “add()” method has been provided to combine Stats from several distinct profile runs. Both the constructor and the add() method now take arbitrarily many file names as arguments.
All the print methods now take an argument that indicates how many lines to print. If the arg is a floating point number between 0 and 1.0, then it is taken as a decimal percentage of the available lines to be printed (e.g., .1 means print 10% of all available lines). If it is an integer, it is taken to mean the number of lines of data that you wish to have printed.
The sort_stats() method now processes some additional options (i.e., in addition to the old -1, 0, 1, or 2). It takes an arbitrary number of quoted strings to select the sort order. For example sort_stats(‘time’, ‘name’) sorts on the major key of ‘internal function time’, and on the minor key of ‘the name of the function’. Look at the two tables in sort_stats() and get_sort_arg_defs(self) for more examples.
Write the profile data to a file we know how to load back.
Expand all abbreviations that are unique.
This module provides support for easily writing both functional and unit tests for your scripts.
New in version 1.0.2.
Bases: unittest.case.TestCase
An application test, based on unittest.TestCase.
AppTest provides a simple setUp() method to instantiate app_cls, your application’s class. default_kwargs will be passed to the new application then.
Deprecated since version 1.1.1: Use AppMixin instead.
An application, usually descended from cli.app.Application.
Default keyword arguments that will be passed to the new cli.app.Application instance.
By default, the application won’t see any command line arguments and will not raise SystemExit when the main() function returns.
Set up the application.
setUp() instantiates app_cls and stores it at app. Test methods should call the application’s cli.app.Application.setup(), cli.app.Application.pre_run() and cli.app.Application.run() methods as necessary.
Bases: unittest.case.TestCase
A functional test, also based on unittest.TestCase.
Functional tests monitor an application’s ‘macro’ behavior, making it easy to spot regressions. They can also be simpler to write and maintain as they don’t rely on any application internals.
The FunctionalTest will look for scripts to run under scriptdir. It uses scripttest.TestFileEnvironment to provide temporary working areas for the scripts; these scratch areas will be created under testdir (and are created and removed before and after each test is run).
Fail if the result object’s stdout, stderr and returncode are unexpected.
result is usually a scripttest.ProcResult with stdout, stderr and returncode attributes.
Run a test script.
script, args and kwargs are passed to env. Default keyword arguments are specified in run_kwargs.
Changed in version 1.1.1: scriptdir is no longer prepended to script before passing it to env. Instead, it is added to the env’s script_path during setUp().
Prepare for the functional test.
setUp() creates the test’s working directory. If the unittest2 package is present, it also makes sure that differences in the test’s standard err and output are presented using unittest2.TestCase.assertMultiLineEqual. Finally, setUp() instantiates the scripttest.TestFileEnvironment and stores it at env.
Clean up after the test.
tearDown() removes the temporary working directory created during setUp().
cli ships with a number of unit tests that help ensure that the code runs correctly. To run the tests, use the tox automated testing tool:
$ tox
By default, tox will attempt to run the unittests on all of the platforms currently supported by cli. See the tox documentation for more information about how to run tests.
You can get a sense for how completely the unit tests exercise cli by running the unittests directly with nose and coverage:
$ nosetests --with-coverage --cover-package=cli
$ coverage report --omit "lib/cli/*ext*,lib/cli/tests/*"
All new code in cli should be accompanied by unit tests. Eventually, the unit tests should be complemented by a set of functional tests (especially to stress things like the daemon code).