Welcome to the KForge Install Guide.
Contents
Using an account that has root privileges, install the following system packages (Debian package names).
$ sudo aptitude install python-pip sqlite3 libsqlite3-dev subversion \
python-subversion python-virtualenv virtualenvwrapper mercurial git \
gitweb mailman apache2 apache2-mpm-worker libapache2-svn viewvc libc-bin \
libapache2-mod-wsgi libapache2-mod-python build-essential python-dev \
locales hostname
Make sure the following Apache modules are enabled.
$ sudo a2enmod wsgi python rewrite dav dav_svn dav_fs env alias cgid
Install the KForge installer.
$ sudo pip install -U kforgeinstall
If you would like to copy and paste the bash shell commands below, please set KFORGEUSER in the environment, for example to ‘projects’.
$ KFORGEUSER=projects
Now, create the KForge system user.
$ sudo adduser $KFORGEUSER --disabled-password
Add the KForge system user to the Apache user group, and add the Apache user group to the KForge system user group. This is necessary because Subversion access via HTTP will run in mod_python (if it is enabled) so that repositories that are readable by unautheticated users don’t prompt for a password even though no password is needed. We haven’t been able to code for this is WSGI. Because this channel will run in mod_python, the process user will be the Apache user, and so new files created within the repository will be owned by the Apache user. To create such files, the Apache user will need to be in the KForge system user group. And then when accessing such files via SSH, the process user will be the KForge system user, and so the KForge system user will need to be in the Apache system user group. And both will need to run with a umask of 002 (see below). The Apache system user also needs to be able to check for .htaccess files in the wsgi directory.
$ sudo addgroup www-data $KFORGEUSER
$ sudo addgroup $KFORGEUSER www-data
Allow the KForge system user to restart Apache, to create and delete mailing lists, and to add and remove members from mailing lists.
$ echo "# KForge needs to restart Apache, and create and delete mailing lists.
%$KFORGEUSER ALL=(ALL) NOPASSWD: /etc/init.d/apache2
%$KFORGEUSER ALL=(ALL) NOPASSWD: /usr/sbin/newlist
%$KFORGEUSER ALL=(ALL) NOPASSWD: /usr/sbin/rmlist
%$KFORGEUSER ALL=(ALL) NOPASSWD: /usr/sbin/add_members
%$KFORGEUSER ALL=(ALL) NOPASSWD: /usr/sbin/remove_members
" | sudo tee -a /etc/sudoers
Edit the Apache ‘envvars’ file and set the umask to 002.
$ echo "
umask 002
" | sudo tee -a /etc/apache2/envvars
Decide which database management system will be used by KForge. The choices are SQLite (default), PostgreSQL, and MySQL.
If you want to use PostreSQL or MySQL, then follow the sections below. If you want to use SQLite, there is no need to anything further to prepare the database.
If you want to use PostgreSQL, then install the PostgreSQL packages (pick the version of the -dev- package according to the version of PostgreSQL on your system).
$ sudo aptitude install postgresql postgresql-server-dev-X.X
Make sure the KForge system user can access the database via TCP/IP sockets with a password. If necessary, please refer to the PostgreSQL manual.
Remember the database username and password, and the name of the new database.
If you want to use MySQL, then install the MySQL packages.
$ sudo aptitude install mysql-server libmysqlclient-dev
Make sure the KForge system user can access the database via TCP/IP sockets with a password. If necessary, please refer to the MySQL manual.
Remember the database username and password, and the name of the database.
Change to the KForge system user.
$ sudo su $KFORGEUSER --login
If you would like to copy and paste the bash shell commands below, then set the following environment variables: SITEHOST as the fully qualified domain name of your KForge site; RELEASE as the name of the release; and SERVICE as a combination of SITEHOST and RELEASE
(If you don’t know which version of KForge will be installed, please check with the Python Package Index.)
$ SITEHOST=$USER.`dnsdomainname`
$ RELEASE=kforge-0.20
$ SERVICE=$SITEHOST-$RELEASE
If you are creating a new site, please note that it is recommended to use a separate directory for project data. The project data directory will contain data created by project services for example Subversion repositories and Trac environments. When you migrate to a future version of KForge, you will need to setup a new service using the new version of KForge, but the project data directory will continue to be used by the new service. So it makes good sense to create a separate directory now.
$ mkdir $HOME/storage
$ mkdir $HOME/storage/$SITEHOST/
The KForge installer will create a new virtual Python environment and install KForge into it. Software that is needed to support different services and database systems will also be installed, according to the options you choose.
$ kforge-install --help
Usage: kforge-install [OPTIONS] [PATH]
Installs KForge software to current working directory, or PATH if provided.
...
It is possible to be selective of the different types of project services that will be availabe on your KForge site, but it is recommended to install KForge with all the software needed for all the services, by using the ‘–with-all-services’ option. You will still need to enable each plugin to make its functionality available to users as project services. But if you know you just want to create for example Trac and Git services, then just use the ‘–with-trac’ and ‘–with-git’ options instead of ‘–with-all-services’.
If you want to use PostgreSQL, then use the ‘–with-postgresql’ option. If you want to use MySQL, then use the ‘–with-mysql’ option.
To leave room for future versions of KForge software, install KForge to a directory called ‘software’ and a sub-directory named after the version of KForge being installed (RELEASE).
Now, run the KForge installer. If something fails to download, just try running the installer again.
$ kforge-install --with-all-services $HOME/software/$RELEASE
Activate the new virtualenv, and check the ‘kforge-makeconfig’ and ‘kforge-admin’ programs are available.
$ source $HOME/software/$RELEASE/bin/activate
$ which kforge-makeconfig
...
$ which kforge-admin
...
The KForge configuration file maker ‘kforge-makeconfig’ creates KForge configuration files. There are various options for overiding default values.
The ‘kforge-admin setup’ command will set up a KForge service based on the configuration file.
If you are migrating from an older version of KForge, then prepare the migration data file by activating the old vitualenv and dumping the old data. Before dumping the data, set the path to the dump file in the environment, so the path can be reused when setting up the new service (below).
$ DUMPPATH=$HOME/kforge-data-dump.json
$ source /path/to/old/kforge/bin/activate
$ kforge-admin --config /path/to/old/kforge/etc/kforge.conf migratedump $DUMPPATH
$ deactivate
Now, as above, make sure the new KForge virtualenv is activated, and both the new ‘kforge-makeconfig’ and the ‘kforge-admin’ programs are available.
$ source $HOME/software/$RELEASE/bin/activate
$ which kforge-makeconfig
...
$ which kforge-admin
...
Set and export the KForge environment variable KFORGE_SETTINGS as the filesystem path for the new configuration file.
It is recommended to keep the configuration file in a directory called ‘settings’, and a sub-directory named after both the site host and the new version of KForge (SERVICE).
$ export KFORGE_SETTINGS=$HOME/settings/$SERVICE/kforge.conf
You can read about the various options that are supported by the ‘kforge-makeconfig’ program.
$ kforge-makeconfig --help
Usage: kforge-makeconfig [OPTIONS] [PATH]
Generate new configuration file, see --help for options.
Writes file to PATH if given, otherwise prints to stdout.
...
To leave room for future services (setup from future releases of KForge software), it is recommended to use the ‘–master-dir’ option, so the service will be setup within a directory called ‘services’, and a sub-directory named after both the site host and the version of Kforge (SERVICE).
It is also recommended to locate the project data directory outside the service directory by using the ‘–project-data-dir’ option.
You may also wish to set the service name for your service by using the ‘–service-name’ option, so that your site can be recognised by users.
It is also recommended to enable reloading Apache, by using the ‘–enable-reload-apache’ option.
You may also wish to enable memoisation (‘–enable-memoization’) and the model cache (‘–enable-model-cache’).
It is also recommended to set the site host, by using the ‘–site-host’ option, so that for example email messages can correctly link back to the site.
And if you are using PostreSQL or MySQL, then you will need to use the ‘–db-type’, ‘–db-name’, ‘–db-user’ and ‘–db-pass’ options. (Otherwise, if you are using SQLite, then you do not need to use these options. But if you wish to specify the location of the SQLite file, then you can use the ‘–db-type’ to provide a filesystem path.)
Now, create the configuration file.
$ kforge-makeconfig --master-dir=$HOME/services/$SERVICE \
--project-data-dir=$HOME/storage/$SITEHOST --service-name="My KForge" \
--enable-reload-apache --enable-memoization --enable-model-cache \
--site-host=$SITEHOST $KFORGE_SETTINGS
Edit the new configuration file, and review the options. If you are upgrading from an existing service, then copy and paste any preferences for timezone, password digest secret, SMTP settings, and so on.
$ editor $KFORGE_SETTINGS
You can read about the various options that are supported by the ‘kforge-admin’ program.
$ kforge-admin --help
Usage: kforge-admin [options] [command]
Administer a KForge service, including its domain objects.
To obtain information about the commands available run the "help" command.
$ kforge-admin help
Now, setup a new service based on the configuration file created above. (The ‘kforge-admin’ program reads the path to the configuration file from the KFORGE_SETTTINGS environment variable.) If something goes wrong, you will need to move or remove the directories created under the “master directory” before trying again (and recreate an empty database if using PostgreSQL or MySQL).
$ kforge-admin setup
If you are upgrading, you can provide the path to the migration data as an additonal argument to the ‘setup’ command. This will initialise the database with migrated data, rather than default data.
$ kforge-admin setup $DUMPPATH
After the service has been setup, make sure the system settings are okay.
$ kforge-admin checksettings
If you are creating a new service (rather than migrating from an old service), then enable plugins for the different kinds of project services that are to be available. For example, if you installed KForge with all services, then you may wish to enable the ‘trac’, ‘git’, ‘svn’, ‘mercurial’ and ‘mailman’ plugins.
If there are any snags with a plugin’s dependencies or configuraton settings, they will need to be resolved before the plugin can be enabled. Plugin configuration options are set in the KForge configuration file.
For example, if the directory for “static” git files is not located at ‘/usr/share/gitweb/static’ then you may need to change this path to be ‘/usr/share/gitweb’ instead. This is one difference between Ubuntu and Debian.
$ kforge-admin plugin enable git
$ kforge-admin plugin enable mailman
$ kforge-admin plugin enable mercurial
$ kforge-admin plugin enable ssh
$ kforge-admin plugin enable svn
$ kforge-admin plugin enable trac
You can see which plugins are enabled by running the ‘plugin enabled’ command.
$ kforge-admin plugin enabled
You can see which plugins are available by running the ‘plugin available’ command.
$ kforge-admin plugin available
If you are upgrading and have enabled the trac plugin, then ‘sync’ the trac services.
$ kforge-admin plugin sync trac
You can use the ‘apacheconfig path’ command of the ‘kforge-admin’ program to locate the auto-generated KForge Apache configuration file.
$ kforge-admin apacheconfig path
The KForge Apache configuration file will be regenerated as project services change, so it is important to include the generated file within the main Apache configuration, rather than copy its contents elsewhere. It is recommended to run the WSGI process with the process user as the KForge system user, and the process group as the Apache system user group (so mailman doesn’t complain when new lists are created).
Now, create an Apache virtual host for the new KForge service. It is also recommended to create a separate directory for Apache virtual host files.
$ mkdir $HOME/sites
$ echo "<VirtualHost *:80>
ServerName $SITEHOST
Include `kforge-admin apacheconfig path`
WSGIDaemonProcess $SITEHOST user=$USER group=www-data threads=15 maximum-requests=10000
</VirtualHost>
" > sites/$SITEHOST
Exit the shell, return to the account that has root priveleges. Link to the Apache virtual host. Enable the Apache virtual host. Restart Apache.
$ exit
$ SITEHOST=$KFORGEUSER.`dnsdomainname`
$ sudo ln -s /home/$KFORGEUSER/sites/$SITEHOST /etc/apache2/sites-available/$SITEHOST
$ sudo a2ensite $SITEHOST
$ sudo /etc/init.d/apache2 restart
Configure your DNS so the site name can be resolved (or as a temporary fix, set the site host in your hosts file).
Check you can resolve the site name.
$ getent hosts $SITEHOST
...
Now, visit the KForge service in a browser.
$ www-browser http://$SITEHOST
For a new service, login with username ‘admin’, password ‘pass’. Please note, it is strongly recommended that you update this password immediately (see User Guide). If you are migrating, then login with previously established credentials.
Optionally, tail the Apache and KForge logs for errors.
$ sudo tail -f /var/log/apache2/*log
$ sudo tail -f /home/$KFORGEUSER/services/kforge-0.20-$SITEHOST/var/log/kforge.log
Configure a cronjob to dump the domain data, and then rsync all files to a remote machine.
Then, you will be able to restore the site by following the instructions for migrating, with an additional step of copying the backup of the project data into place. You may wish to test that your backup is working by building and maintaining a “fail-over” site on a different machine.
Please note, KForge should not be installed under DocumentRoot of the Apache installation, or any other directory exposed via Apache web server.
It is recommended to run a production service with PostgreSQL. You may also wish to enable both model cacheing and access control memoisation (see KForge configuration file). You may also wish to adjust Apache MPM parameters and the WSGI process parameters to match your hardware.
Please note, a production service should be created freshly and not by reusing a service that has been used to run the test suite. Do not run the test suite against a production service.
If you wish to run the KForge test suite, then set the KForge configuration option ‘system_mode’ to ‘development’ before setting up the KForge service.
It is recommended to repeat the above steps, starting from ‘Create System User’ perhaps with a system user name of ‘testprojects’ instead of ‘projects’.
Do not enable model cacheing, and do not enable access control memoisation.
The reason for creating a new user for each site is to make sure SSH access for each service is well separated (there can only be one authorized_users file per system user).
Now, the test suite can be executed by running the ‘kforge-test’ program.
$ kforge-test
The KForge software, and this install guide, have been tested with Python 2.6 on Debian 6, and with Python 2.7 on Ubuntu 12.10.
If you wish to create a user acceptance testing site for a new version of KForge, whilst leaving the current production service in operation, then repeat the above steps, Starting from ‘Create System User’ perhaps with a system user name of ‘acceptprojects’ instead of ‘projects’.