The server recipe actually includes a general engine to install Python code that needs access to the Odoo API and build configuration-aware executables.
As usual, it tries and do so by bridging the standard Python packaging practice (setuptools-style console scripts, as in zc.recipe.egg:scripts) and Odoo specificities.
We call such scripts Odoo scripts to distinguish them among the more general concept of console scripts.
Warning
Odoo scripts are currently supported for versions ≥ 6.1 only.
Odoo scripts can do great in situations where an RPC script might not be powerful enough or not practical. Some examples:
There are several Python distributions that wrap the Odoo RPC APIs for easy use within Python code.
Using an RPC script for administrative tasks usually leads to wrap it in a shell script, with the admin password in clear text.
In this author’s experience of applicative maintainance, this always turns to be an easy source of breakage that may look to be trivial at first sight but has actually two nasty properties : it may stay unnoticed for a while, and it lies at the interface between responsibilities.
In case of password change, the persons who can do it in the database and in the system usually differ, and may not communicate on a regular basis. In enterprise hosting environments, you may have to explain stuff to several project managers with different responsabilities, go through crisis management meetings, etc. Who wants to waste hours of their life interacting with people under stress to try and persuade them that it’s only a matter of changing an obscure password ?
Because they are part of addons, Odoo cron jobs also have full unrestricted access to the internal API, and obviously don’t suffer from the password plague.
Some ideas to make a choice:
Perhaps, the best is not to choose : put the bulk of the logic in some technical addon, it’s easy to rewrap it in an Odoo script and as a cron job.
There are several cases, depending on the script authors intentions. Script authors should therefore state clearly in their documentation how to declare them.
Assume to fix ideas there is a Python distribution my.script that declares a console script entry point named my_script in its setup.py:
entry_points="""
[console_scripts]
my_script = my.script.main:run
"""
The first thing to do is to require that distribution using the eggs option:
[my-openerp]
(...)
eggs = my.script
How that distribution can be made available to buildout is a different question.
The following configuration:
[openerp-one]
(...)
openerp_scripts = my_script
Produces an executable bin/my_script-openerp-one, that can import Odoo server and addons code, and in which the Odoo configuration related to the appropriate buildout part (here, openerp-one) is loaded in the standard openerp.tools.config, for use in the script. The script has to take care of all database management operations.
Optionally, it’s possible to specify the name of the produced script:
[openerp-one]
(...)
openerp_scripts = my_script=wished_name
That would build the script as bin/wished_name.
This is good enough for scripts that’d take care of many bootstrapping details, but there is a more integrated way that script authors should be aware of: the special session argument.
Note
new in version 1.7.0
An arguments parameter, similar to the one of zc.recipe.egg:scripts can be specified:
[openerp-two]
(...)
openerp_scripts = my_script arguments=2,3
This is a raw string that will be used as the string of arguments for the callable specified in the entry point, as in main(2,3) in that example.
There is a special argument: session, which is an object provided by the recipe to expose Odoo API in a convenient manner for script authors. Check anybox.recipe.openerp.runtime.session.Session to learn what can be done with it.
Scripts written for these session objects must be declared as such:
[openerp-two]
(...)
openerp_scripts = my_script arguments=session
In some cases, it is useful to do some operations, such as preloading a database, before actual running of the script. This is intended for scripts which have no special knowledge of Odoo but may in turn call some code meant for Odoo, that’d need some preparations to already have been performed.
The main use-case is unit tests launchers.
For these, the command-line-options modifier tells the recipe to produce an executable that will implement some additional command-line options parsing and perform some actions accordingly. On the command-line -- is used as a separator between those additional options and the regular arguments expected by the script.
Example:
[openerp-three]
(...)
openerp_scripts = nosetests command-line-options=-d
This produces a bin/nosetests_openerp-three, which you can use like this:
bin/nosetests_openerp-three -d mydb -- [NOSE REGULAR OPTIONS & ARGUMENTS]
Currently available command-line-options:
-d DB_NAME: | preload the specified database |
---|
This is mostly meant for scripts with the command-line-options=-d modifier.
In some cases, one is not interested in the logs during the Odoo database load. The typical use-case this has been made for is the sphinx-build script, where any warning from Odoo would just make it harder to stop actual documentation warnings, or to limit the output of test launcher before actual testing begins.
The openerp_log_level modifier lets you specify the log level for the openerp logger, at the very start of the script, before any database loading is performed.
In the case of sphinx-build this has the advantage of not affecting the root logger nor the Sphinx dedicated ones.
Of course, the actual script can override that setting once it really starts, in which case the modifier is really only about the loading sequence.
One particularly interesting use of openerp_scripts is to enable the use of enhanced Python interactive interpreters, like IPython or bPython:
[buildout]
parts = odoo
find-links = http://download.gna.org/pychart/
[odoo]
version = git http://github.com/odoo/odoo.git odoo 8.0 depth=1
recipe = anybox.recipe.odoo:server
eggs =
ipython
bpython
openerp_scripts =
ipython arguments=user_ns=dict(session=session)
bpython arguments=locals_=dict(session=session)
The example buildout.cfg above will generate both a ipython_odoo and a bpython_odoo scripts in the bin directory, which can be used like the Python interpreter generated by interpreter_name, where the session object is available for interacting with your odoo application and database.
Keep in mind that bpython requires more system dependencies installed than plain odoo.
Note that Odoo forbids using the postgres user to connect to the database. But in some containerized environments (Docker), using postgres can be both safe and handy. In such case you would need to patch the Odoo server as of today. But for the interactives sessions of this buildout recipe, you can set the environment variable ENABLE_POSTGRES_USER=1 before opening the console to disable the default check_postgres_user() guard and enable the postgres user.
Script authors have to:
write their script as a callable within a setuptools distribution. Usually that’d be a function my_run at toplevel of a my/script/main.py file
declare that callable in setup.py like this:
entry_points="""
[console_scripts]
my_script = my.script.main:my_run
"""
(recommended) use the anybox.recipe.openerp.runtime.session.Session API. For that, let your callable accept a session argument, and tell users to pass it in their buildout configuration.
write the actual script! Here’s a silly example, that outputs the total of users in the database:
from argparse import ArgumentsParser
def my_run(session):
# command-line arguments handling is up to the script
parser = ArgumentsParser()
parser.add_argument('-d', '--database',
help="Database to work on", required=True)
arguments = parser.parse_args()
# loading the DB
session.open(arguments.database)
# using the models
users = session.registry('res.users').search(
session.cr, session.uid, [])
print("There are %d users in database %r" % (
len(users), arguments.database))
# Transaction control is up to the script
session.rollback() # we didn't write anything, but one never knows
In order to be used by the recipe, the distribution that holds the script code has to be required with the eggs option. But how can buildout retrieve it ? There’s nothing specific to the Odoo recipe about that, it works in the exact same way as for the standard zc.recipe.eggs recipe.
We list here some possibilities, as a convenience for readers without a more general buildout experience.
provide it locally and tell buildout to “develop” it:
[buildout]
develop = my_script_distribution_path
paths are interpreted relative to the buildout directory, but may be absolute.
put it on the Python Package Index
put it in a private index and use the index main buildout option
prebuild an egg and put it in the eggs directory (can be shared between several buildouts).
put a source distribution (tarball) or an egg on some HTTP server, and use the find-links global buildout option.
grab it and develop it from an external VCS, using the gp.vcsdevelop buildout extension.
use one of the other VCS-oriented buildout extensions (such as mr.developer
Note
the releasing features (freeze, extract) of the recipe are aware of gp.vcsdevelop and will control the revision it uses. There’s no such support of mr.developer right now.
Note
new in version 1.8.0
The recipe provides a toolkit for database management, including upgrade scripts generation, to fulfill two seemingly contradictory goals:
To accomodate these two needs, the installation-dependent flexibility is given back to the user (a project maintainer in that case) by letting her write the actual upgrade logic in the simplest way possible. The recipe rewraps it and produces the actual executable, with its command-line parsing, etc.
Project maintainers have to produce a callable using the high-level methods of anybox.recipe.openerp.runtime.session.Session. Here’s an example:
def run_upgrade(session, logger):
db_version = session.db_version # this is the state after
# latest upgrade
if db_version < '1.0':
session.update_modules(['account_account'])
else:
logger.warn("Not upgrading account_account, as we know it "
"to be currently a problem with our setup. ")
session.update_modules(['crm', 'sales'])
No need to set db_version, nor to commit: the recipe will do it for you in case of success (see below)
Such callables (source file and name) can be declared in the buildout configuration with the upgrade_script option:
upgrade_script = my_upgrade.py run_upgrade
The default is upgrade.py run. The path is interpreted relative to the buildout directory.
If the specified source file is not found, the recipe will initialize it with the simplest possible one : update of all modules. That is expected to work 90% of the time. The package manager can then modify it according to needs, and maybe track it in version control.
Note
about versions
the db_version settable property is meant to be really global for this precise project. The idea is that anything depending only on a module’s version number should be done in that module’s migration scripts (pre or post).
you’re supposed to provide and maintain a “package version” in a VERSION.txt file at the root of the buildout (the recipe will warn you if it’s missing). The recipe will use it to set db_version at the end of the process.
In truth, upgrade scripts are nothing but Odoo scripts, with the entry point console script being provided by the recipe itself, and in turn relaying to that user-level callable. See anybox.recipe.openerp.runtime.upgrade for more details on how it works.
For projects with a fixed number of modules to install at a given point of code history, upgrade scripts can be used to install a fresh database:
def upgrade(session, logger):
"""Create or upgrade an instance or my_project."""
if session.is_initialization:
logger.info("Installing modules on fresh database")
session.install_modules(['my_module'])
return
# now upgrade logic
Not having a command-line argument for modules ot install in the resulting script is a strength. It means that CI robots, deployment tools and the like will be able to install it with zero additional configuration.
The default script produced by the recipe also detects initializations and logs information on how to customize:
2013-10-14 17:16:17,785 WARNING Usage of upgrade script for initialization detected. You should consider customizing the present upgrade script to add modules install commands. The present script is at : /home/gracinet/openerp/recipe/testing-buildouts/upgrade.py (byte-compiled form)
2013-10-14 17:16:17,786 INFO Initialization successful. Total time: 22 seconds.
Note
the is_initialization attribute is new in version 1.8.1
Command-line parsing is done with argparse. If you have any doubt, use --help with the version you have. Here’s the current state:
$ bin/upgrade_openerp -h
usage: upgrade_openerp [-h] [--log-file LOG_FILE] [--log-level LOG_LEVEL]
[--console-log-level CONSOLE_LOG_LEVEL] [-q]
[-d DB_NAME]
optional arguments:
-h, --help show this help message and exit
--log-file LOG_FILE File to log sub-operations to, relative to the current
working directory, supports homedir expansion ('~' on
POSIX systems). (default: upgrade.log)
--log-level LOG_LEVEL
Main Odoo logging level. Does not affect the
logging from the main upgrade script itself. (default:
info)
--console-log-level CONSOLE_LOG_LEVEL
Level for the upgrade process console logging. This is
for the main upgrade script itself meaning that
usually only major steps should be logged (default:
info)
-q, --quiet Suppress console output from the main upgrade script
(lower level stages can still write) (default: False)
-d DB_NAME, --db-name DB_NAME
Database name. If ommitted, the general default values
from Odoo config file or libpq will apply.
--init-load-demo-data
Demo data will be loaded with module installations if
and only if this modifier is specified (default:
False)
Here’s the output of a run of the default upgrade script:
$ bin/upgrade_openerp -d testrecipe
Starting upgrade, logging details to /home/gracinet/openerp/recipe/testing-buildouts/upgrade.log at level INFO, and major steps to console at level INFO
2013-09-21 18:53:23,471 WARNING Expected package version file '/home/gracinet/openerp/recipe/testing-buildouts/VERSION.txt' does not exist. version won't be set in database at the end of upgrade. Consider including such a version file in your project *before* version dependent logic is actually needed.
2013-09-21 18:53:23,471 INFO Database 'testrecipe' loaded. Actual upgrade begins.
2013-09-21 18:53:23,471 INFO Default upgrade procedure : updating all modules.
2013-09-21 18:53:54,029 INFO Upgrade successful. Total time: 32 seconds.
The same with a version file:
$ bin/upgrade_openerp -d testrecipe
Starting upgrade, logging details to /home/gracinet/openerp/recipe/testing-buildouts/upgrade.log at level INFO, and major steps to console at level INFO
2013-09-22 19:23:17,908 INFO Read package version: 6.6.6-final from /home/gracinet/openerp/recipe/testing-buildouts/VERSION.txt
2013-09-22 19:23:17,908 INFO Database 'testrecipe' loaded. Actual upgrade begins.
2013-09-22 19:23:17,909 INFO Default upgrade procedure : updating all modules.
2013-09-22 19:23:48,626 INFO setting version 6.6.6-final in database
2013-09-22 19:23:48,635 INFO Upgrade successful. Total time: 32 seconds.
The familiar start_openerp, and its less pervasing siblings (gunicorn_openerp, test_openerp, …) are also special cases of Odoo scripts.
What is special with them amounts to the following:
In particular, you can control the names of the startup scripts with the openerp_scripts option. For instance, to replace bin/start_openerp with bin/oerp, just do:
[openerp]
(...)
openerp_scripts = openerp_starter=oerp
Here’s the list of currently available internal entry points.
openerp_starter: | |
---|---|
main Odoo startup script (dynamically added behing the scenes by the recipe) | |
openerp_tester: | uniform script to start Odoo, launch all tests and exit. This can be achieved with the main startup scripts, but options differ among Odoo versions. (also dynamically added behind the scenes). |
openerp_upgrader: | |
entry point for the upgrade script | |
openerp_cron_worker: | |
entry point for the cron worker script that gets built for gunicorn setups. | |
oe: | entry point declared by openerp-command and used by the recipe. |
gunicorn: | entry point declared by gunicorn and used by the recipe. |
Note
For these entry points, the command-line-options and arguments modifiers have no effect.