Since one of the reason’s for exploring docopt is to find a way to simplify the argument parsing in the Ape, I’ll see if I can reproduce it here.
Contents:
| ape.main |
print subprocess.check_output('ape -h'.split())
usage: ape.interface.arguments [-h] [--debug] [-v] [--silent] [--pudb] [--pdb]
[--trace] [--callgraph]
{run,fetch,list,check,help} ...
optional arguments:
-h, --help show this help message and exit
--debug Sets the logging level to debug
-v, --version Display the version number and quit
--silent Sets the logging level to off (for stdout)
--pudb Enables the pudb debugger
--pdb Enables the pdb debugger
--trace Turn on code-tracing
--callgraph Create call-graph
Sub-Commands Help:
Available Subcommands
{run,fetch,list,check,help}
SubCommands
run Run the Ape
fetch Fetch a sample config file.
list List available plugins.
check Check your setup.
help Show more help
I’ll start by trying to reproduce the usage string, taking advantage of the greater flexibility of docopt.
usage = """The APE
The All-in-one Performance Evaluator.
Usage: ape -h|-v
Optional Arguments:
-h, --help Print a help message.
-v, --version Print the version number.
"""
help and version are special flags that are intercepted by docopt so you don’t have to handle them (using version requires you to pass in the version to the docopt function call). Since the docopt help and version calls force the program to exit, I created a function to trap the exception called catch_exit (otherwise Pweave will quit and this documentation won’t get built).
catch_exit(usage, argv=['-h'])
The APE
The All-in-one Performance Evaluator.
Usage: ape -h|-v
Optional Arguments:
-h, --help Print a help message.
-v, --version Print the version number.
Without the version set it thinks -v is just another option:
print docopt.docopt(doc=usage, argv=['-v'])
{'--help': False,
'--version': True}
With the version set -v tells it to print the version:
catch_exit(usage, argv=['-v'], version="alpha.beta.gamma")
alpha.beta.gamma
I defined the options as mutually exclusive (-h | -v), what happens if we pass in both flags?
catch_exit(usage, argv='--help --version'.split(), version='abc')
The APE
The All-in-one Performance Evaluator.
Usage: ape -h|-v
Optional Arguments:
-h, --help Print a help message.
-v, --version Print the version number.
It unexpectedly accepts the help option and ignores the version option. Is it the ordering that matters?
catch_exit(usage, argv='-v -h'.split())
The APE
The All-in-one Performance Evaluator.
Usage: ape -h|-v
Optional Arguments:
-h, --help Print a help message.
-v, --version Print the version number.
Apparently help intercepts the other options.
Now that help and version are out of the way, let’s add some options for the ape to interpret.
usage = """APE
Usage: ape -h | -v
ape --debug | --silent
Help Options:
-h, --help Display this help message and quit.
-v, --version Display the version number and quit.
Ape Options:
--debug Set logging level to DEBUG.
--silent Set logging level to ERROR.
"""
print docopt.docopt(doc=usage, argv=["--silent"])
{'--debug': False,
'--help': False,
'--silent': True,
'--version': False}
As expected, the options default to False and since we passed in the --silent only silent in the returned dictionary was set to True. What if we pass both silent and debug?
print catch_exit(usage, argv='--silent --debug'.split())
Usage: ape -h | -v
ape --debug | --silent
It looks like with the exception of cases where help is involved, docopt will enforce the 0 or 1 cardinality for the alternative options – you can either choose one or leave it out altogether, but you can’t pick more than one of them.
What about adding interactive debugging options (pudb or pdb)?
usage = """APE
Usage: ape -h | -v
ape [--debug | --silent] [--pudb | --pdb]
Help Options:
-h, --help Display this help message and quit.
-v, --version Display the version number and quit.
Ape Options:
--debug Set logging level to DEBUG.
--silent Set logging level to ERROR.
"""
I’ll try setting both logging and debugging, which are both optional and not mutually exclusive.
print docopt.docopt(doc=usage, argv="--silent --pudb".split())
{'--debug': False,
'--help': False,
'--pdb': False,
'--pudb': True,
'--silent': True,
'--version': False}
As expected, the dictionary entries for --silent and --pudb are True and and all the others are False.
The last of the top-level options are trace and callback (which I don’t remember as being particularly useful). Does it make sense to be able to set an interactive debugger when using trace or callback? Probably not, but what about the logging level? Maybe. I’ll say yes for now.
usage = """APE
Usage: ape -h | -v
ape [--debug | --silent] [--pudb | --pdb]
ape [--debug | --silent] [--trace] [--callback]
Help Options:
-h, --help Display this help message and quit.
-v, --version Display the version number and quit.
Ape Options:
--debug Set logging level to DEBUG.
--silent Set logging level to ERROR.
"""
print docopt.docopt(doc=usage, argv="--trace --debug --callback".split())
{'--callback': True,
'--debug': True,
'--help': False,
'--pdb': False,
'--pudb': False,
'--silent': False,
'--trace': True,
'--version': False}
If you look at the dictionary that was returned you can see that --trace, --debug and --callback were set to True and all the others to False, so it’s still working as expected. We now have three usage lines that each takes a different set of options, what happens if you mix up options from two different lines?
print catch_exit(usage, argv='--callback --pudb')
Usage: ape -h | -v
ape [--debug | --silent] [--pudb | --pdb]
ape [--debug | --silent] [--trace] [--callback]
It looks like it enforces the usage strings so your arguments have to match one of the usage-lines: –pudb isn’t offered on the same line that has –callback so it (docopt) prints the usage message and quits.
Setting the options above by themselves doesn’t really seem useful because the Ape expects sub-commands as well so we’ll have to add them. First we’ll tackle the run sub-command.
print subprocess.check_output('ape run -h'.split())
usage: ape.interface.arguments run [-h]
[<config-file list> [<config-file list>
...]]
positional arguments:
<config-file list> A list of config file name (default='['ape.ini']').
optional arguments:
-h, --help show this help message and exit
This looks easy enough, but first we have to add a <command> argument to the base usage-string so that we’ll know that the user wants to get the run sub-command. We’ll also need a list of optional <argument> inputs to pass to the sub-command.
APE
Usage: ape -h | -v
ape [--debug|--silent] [--pudb|--pdb] <command> [<argument>...]
ape [--debug|--silent] [--trace|--callgraph] <command> [<argument>..
.]
Help Options:
-h, --help Display this help message and quit.
-v, --version Display the version number and quit.
Logging Options:
--debug Set logging level to DEBUG.
--silent Set logging level to ERROR.
Debugging Options:
--pudb Enable the `pudb` debugger (if installed)
--pdb Enable the `pdb` (python's default) debugger
--trace Enable code-tracing
--callgraph Create a call-graph of for the code
Positional Arguments:
<command> The name of a sub-command (see below)
<argument>... One or more options or arguments for the sub-command
Available Sub-Commands:
run Run a plugin
fetch Fetch a sample configuration-file
help Display more help
list List known plugins
check Check a configuration
run_args = docopt.docopt(doc=usage, argv="run".split())
print run_args
{'--callgraph': False,
'--debug': False,
'--help': False,
'--pdb': False,
'--pudb': False,
'--silent': False,
'--trace': False,
'--version': False,
'<argument>': [],
'<command>': 'run'}
docopt allows you to name the variables (that hold the values users pass in to options and arguments) using either all uppercased letters (e.g. MODULES) or by lower-cased letters surrounded by angle brackets (e.g. <modules>). I looked at several commands on my machine and noticed the following:
iperf uses brackets (-o <filename>) unless it expects a number in which case it uses a pound sign (-i #)
nmap uses angle brackets (--max-retries <tries>)
man uses all caps and an equals sign (-L=LOCALE)
ls uses uppercase and = (--sort=WORD)
ping uses lower-cased variable names (-i interval)
- argparse (what the ape is using) uses UPPERCASE without an equal sign unless you set the metavar yourself in which case it uses angle brackets
- --modules [MODULES [MODULES...]]
- <config-file list>
The ping convention won’t work with docopt, since it uses either brackets or upper-cased letters to decide what’s a variable. I can’t really decide which of the remaining conventions is better. I think the UPPERCASE convention makes them stand out more and angle brackets introduce extra visual noise, but using equals signs seems confusing, since you don’t actually use them in the command (then again, you don’t use ellipses and square brackets either (oh, well)). I was thinking about using the argparse default convention of using uppercased letters without an equals sign, but it occurred to me that using the angle brackets allows you to create arbitrary strings with punctuation and whitespace (<config-file list> vs CONFIGFILELIST) so I think I’ll stick to the angle-brackets for now. (although see the Check Sub-Command section for a problem that I ran into)
The kind of disappointing part of docopt is that we don’t have a way to automatically pass things off to the sub-command. Instead we have to create a new parser or interpret the running ourselves.
from commons import run_usage
print catch_exit(run_usage, argv=['-h'])
`run` sub-command
Usage: ape run -h
ape run [<configuration>...]
Positional Arguments:
<configuration> 0 or more configuration-file names [default: ape.ini]
Options;
-h, --help This help message.
print docopt.docopt(doc=run_usage, argv=['run'])
{'--help': False,
'<configuration>': [],
'run': True}
It looks like it doesn’t allow you to set a default for positional arguments, so you’d have to check yourself or change the positional argument to an option. Let’s make sure that the <config> arguments are working at least.
print docopt.docopt(doc=run_usage, argv="run ape.ini man.ini".split())
{'--help': False,
'<configuration>': ['ape.ini', 'man.ini'],
'run': True}
Okay, but the idea for using this is that the run help would be reached from the base ape configuration. How does that work?
catch_exit(usage, argv="run -h".split())
APE
Usage: ape -h | -v
ape [--debug|--silent] [--pudb|--pdb] <command> [<argument>...]
ape [--debug|--silent] [--trace|--callgraph] <command> [<argument>..
.]
Help Options:
-h, --help Display this help message and quit.
-v, --version Display the version number and quit.
Logging Options:
--debug Set logging level to DEBUG.
--silent Set logging level to ERROR.
Debugging Options:
--pudb Enable the `pudb` debugger (if installed)
--pdb Enable the `pdb` (python's default) debugger
--trace Enable code-tracing
--callgraph Create a call-graph of for the code
Positional Arguments:
<command> The name of a sub-command (see below)
<argument>... One or more options or arguments for the sub-command
Available Sub-Commands:
run Run a plugin
fetch Fetch a sample configuration-file
help Display more help
list List known plugins
check Check a configuration
Okay, so that wasn’t what I wanted – the -h got caught before the sub-command was set and the top-level help got dumped to the screen. It turns out that there’s a docopt parameter called options_first which is False by default. When it’s True, the top-level options are only intrepreted before you get to the first positional argument and then rest are passed to the argument. So this would get the ape’s help and ignore everything else:
ape -h run
While this would pass the -h in as an argument for the <command> entry in the returned dictionary.
output = docopt.docopt(doc=usage, argv="run -h".split(),
options_first=True)
print output
{'--callgraph': False,
'--debug': False,
'--help': False,
'--pdb': False,
'--pudb': False,
'--silent': False,
'--trace': False,
'--version': False,
'<argument>': ['-h'],
'<command>': 'run'}
So now to make it work we would need to check the <command> entry and pass the arguments to docopt using the run-usage string instead.
if output['<command>'] == 'run':
arguments = ['run'] + output['<argument>']
catch_exit(run_usage, argv=arguments)
`run` sub-command
Usage: ape run -h
ape run [<configuration>...]
Positional Arguments:
<configuration> 0 or more configuration-file names [default: ape.ini]
Options;
-h, --help This help message.
Note
This fixes the inability to pass in the help option to the sub-command, but the behavior is now different from ArgParse – ArgParse keeps all the arguments at the top level (e.g. if args is the argparse namespace object, args.configuration would have the list of configuration files) but now the arguments specific to the sub-command are kept in a list (output['<argument>']) and need to be re-parsed using the sub-command’s usage string and docopt.
Presumably the run_usage string would be imported from the module where the run function is (the docopt documentation says that the intention is for the usage-string to be in the module’s docstring (__doc__)).
Now we should check the case where the user passed in some configuration file names. Although we would normally have to check for the command, I’ll just assume it’s working correctly here to save space.
# pretend we imported this
MESSAGE = "running '{config}'"
def run(configurations):
# empty lists evaluate to False
if not configurations:
configurations = ['ape.ini']
for configuration in configurations:
print MESSAGE.format(config=configuration)
return
output = docopt.docopt(doc=usage, argv='run cow.ini pie.ini'.split())
run(output['<argument>'])
running 'cow.ini'
running 'pie.ini'
The fetch sub-command should be similar to the run sub-command.
# python standard library
import subprocess
# third-party
import docopt
# this documentation
from commons import catch_exit, usage
print subprocess.check_output('ape fetch -h'.split())
usage: ape.interface.arguments fetch [-h] [--modules [MODULES [MODULES ...]
]]
[names [names ...]]
positional arguments:
names List of plugin-names (default=['Ape'])
optional arguments:
-h, --help show this help message and exit
--modules [MODULES [MODULES ...]]
Non-ape modules
from commons import fetch_usage
catch_exit(fetch_usage, ['--help'])
fetch subcommand
usage: ape fetch -h
ape fetch [<name>...] [--module <module> ...]
positional arguments:
<name> List of plugin-names (default=['Ape'])
optional arguments:
-h, --help Show this help message and exit
-m, --module <module> ... Non-ape modules
So it kind of looks like it works. I’m not sure if the help options should be optional or not, since all the arguments are optional. I guess it’s a matter of taste.
arguments = "fetch Dummy Sparky Iperf --module pig.thing --m cow.dog".split()
output = docopt.docopt(doc=usage, argv=arguments, options_first=True)
arguments = [output['<command>']] + output['<argument>']
print docopt.docopt(doc=fetch_usage, argv=arguments)
{'--help': False,
'--module': ['pig.thing', 'cow.dog'],
'<name>': ['Dummy', 'Sparky', 'Iperf'],
'fetch': True}
Warning
the arguments = [output['<command>']] + output['<argument>'] line in the above is necessary because the usage lines start with ape fetch but the fetch token is being assigned to the <command> key in the output dictionary so the <argument> list doesn’t have it. If you don’t re-add it to the arguments, docopt will print the usage string and quit. The run section above only worked because I cheated and assumed I was getting the list of file-names, not a list of arguments to pass to docopt.
Note
I mentioned it in the run section, but the first call to docopt has to set options_first to True or the options for the sub-command will get added to the top-level dictionary.
This lists the plugins, so it should be even simpler, I think.
print subprocess.check_output('ape list -h'.split())
usage: ape.interface.arguments list [-h] [--modules [MODULES [MODULES ...]]
]
optional arguments:
-h, --help show this help message and exit
--modules [MODULES [MODULES ...]]
Space-separated list of non-ape modules with plugin
s
from commons import list_usage
catch_exit(list_usage, ['-h'])
list subcommand
usage: ape list -h
ape list [<module> ...]
Positional Arguments:
<module> ... Space-separated list of importable module with plugins
optional arguments:
-h, --help Show this help message and exit
Note
The --help overrides everything, so even though that last call was missing list as the first argv element, it still works.
print docopt.docopt(list_usage, argv=['list'])
{'--help': False,
'<module>': [],
'list': True}
arguments = 'list man.dog bill.ted'.split()
base_output = docopt.docopt(doc=usage, argv=arguments, options_first=True)
arguments = [base_output['<command>']] + base_output['<argument>']
print docopt.docopt(list_usage, argv=arguments)
{'--help': False,
'<module>': ['man.dog', 'bill.ted'],
'list': True}
The inability to pass in a list to an option seems like a flaw. Either I can be consistent and require the ‘-m’ option when adding modules or use a positional argument. Maybe ‘-m’ would be better, since it would probably be a rare thing to use, and it would be better to be consistent, but I think since there’s no other options I’ll just leave it like this.
print subprocess.check_output('ape check -h'.split())
usage: ape.interface.arguments check [-h] [--modules [MODULES [MODULES ...]]]
[<config-file list> [<config-file list> ...]]
positional arguments:
<config-file list> List of config files (e.g. *.ini -
default='['ape.ini']').
optional arguments:
-h, --help show this help message and exit
--modules [MODULES [MODULES ...]]
Space-separated list of non-ape modules with plugins
from commons import check_usage
catch_exit(check_usage, argv=['-h'])
`check` sub-command
usage: ape check -h
ape check [<config-file-name> ...] [--module <module> ...]
Positional Arguments:
<config-file-name> ... List of config files (e.g. *.ini - default='['ape.ini']')
optional arguments:
-h, --help Show this help message and exit
-m, --module <module> Non-ape module with plugins
arguments = "check --module cow.dog.man ape.ini -m pip.ini".split()
output = docopt.docopt(doc=usage, argv=arguments, options_first=True)
arguments = [output['<command>']] + output['<argument>']
print docopt.docopt(doc=check_usage, argv=arguments)
{'--help': False,
'--module': ['cow.dog.man', 'pip.ini'],
'<config-file-name>': ['ape.ini'],
'check': True}
Warning
I originally used <config-file name> for the config files, but docopt couldn’t properly parse it. It might be safer to leave whitespace out of the names, especially when mixing positional arguments and options.
print subprocess.check_output('ape help -h'.split())
usage: ape.interface.arguments help [-h] [-w WIDTH]
[--modules [MODULES [MODULES ...]]]
[name]
positional arguments:
name A specific plugin to inquire about.
optional arguments:
-h, --help show this help message and exit
-w WIDTH, --width WIDTH
Number of characters to wide to format the page.
--modules [MODULES [MODULES ...]]
Space-separated list of non-ape modules with plugins
from commons import help_usage
catch_exit(help_usage, ["--help"])
`help` sub-command
usage: ape help -h
ape help [-w WIDTH] [--module <module>...] [<name>]
positional arguments:
<name> A specific plugin to inquire about [default: ape].
optional arguments:
-h, --help show this help message and exit
-w , --width <width> Number of characters to wide to format the page.
-m, --module <module> non-ape module with plugins
output = docopt.docopt(usage, argv="help bob -w 30 -m cow.pipe".split(), op
tions_first=True)
arguments = [output['<command>']] + output['<argument>']
print docopt.docopt(help_usage, arguments)
{'--help': False,
'--module': ['cow.pipe'],
'--width': '30',
'<name>': 'bob',
'help': True}
Since I refer to these while coding I thought I’d put them in one place.
APE
Usage: ape -h | -v
ape [--debug|--silent] [--pudb|--pdb] <command> [<argument>...]
ape [--debug|--silent] [--trace|--callgraph] <command> [<argument>..
.]
Help Options:
-h, --help Display this help message and quit.
-v, --version Display the version number and quit.
Logging Options:
--debug Set logging level to DEBUG.
--silent Set logging level to ERROR.
Debugging Options:
--pudb Enable the `pudb` debugger (if installed)
--pdb Enable the `pdb` (python's default) debugger
--trace Enable code-tracing
--callgraph Create a call-graph of for the code
Positional Arguments:
<command> The name of a sub-command (see below)
<argument>... One or more options or arguments for the sub-command
Available Sub-Commands:
run Run a plugin
fetch Fetch a sample configuration-file
help Display more help
list List known plugins
check Check a configuration
`run` sub-command
Usage: ape run -h
ape run [<configuration>...]
Positional Arguments:
<configuration> 0 or more configuration-file names [default: ape.ini]
Options;
-h, --help This help message.
fetch subcommand
usage: ape fetch -h
ape fetch [<name>...] [--module <module> ...]
positional arguments:
<name> List of plugin-names (default=['A
pe'])
optional arguments:
-h, --help Show this help message and exit
-m, --module <module> ... Non-ape modules
list subcommand
usage: ape list -h
ape list [<module> ...]
Positional Arguments:
<module> ... Space-separated list of importable module with plugins
optional arguments:
-h, --help Show this help message and exit
`check` sub-command
usage: ape check -h
ape check [<config-file-name> ...] [--module <module> ...]
Positional Arguments:
<config-file-name> ... List of config files (e.g. *.ini - default='[
'ape.ini']')
optional arguments:
-h, --help Show this help message and exit
-m, --module <module> Non-ape module with plugins
`help` sub-command
usage: ape help -h
ape help [-w WIDTH] [--module <module>...] [<name>]
positional arguments:
<name> A specific plugin to inquire about [default: ap
e].
optional arguments:
-h, --help show this help message and exit
-w , --width <width> Number of characters to wide to format the page.
-m, --module <module> non-ape module with plugins