Welcome to GC3Pie documentation!

Introduction

GC3Pie is a Python package for running large job campaigns on diverse batch-oriented execution environments (for instance: a Sun/Oracle/Open Grid Engine cluster, or the Swiss National Distributed Computing Infrastructure SMSCG). It also provides facilities for implementing command-line driver scripts, in the form of Python object classes whose behavior can be customized by overriding specified object methods.

GC3Pie documentation is divided in three sections:

• User Documentation: info on how to install, configure and run GC3Pie applications.
• Programmer Documentation: info for programmers who want to use the GC3Pie libraries to write their own scripts and applications.
• Developer Documentation: detailed information on how to contribute to GC3Pie and get your code included in the main library.

Table of Contents

• User Documentation
  • Table of Contents
    • Installation of GC3Pie
      • Quick start
      • Non-standard installation options
      • Manual installation
      • Upgrade
      • MacOSX Installation
      • HTML Documentation
    • Configuration File
      • Location
      • Configuration file format
      • The DEFAULT section
      • auth sections
        • Authentication types
      • resource sections
        • arc0 resources
        • arc1 resources
        • sge resources
        • pbs resources
        • lsf resources
        • slurm resources
        • shellcmd resources
        • ec2+shellcmd resource
        • Example resource sections
      • Enabling/disabling selected resources
    • Environment Variables
    • The GC3Apps software
      • Introduction to session-based scripts
        • Basic command-line usage and options
          • Continuous execution
          • Verbose listing of jobs
          • Maximum number of concurrent jobs
          • Location of output files
        • Job control options
        • Session control options
        • Exit code
      • The ggamess script
        • How to run GAMESS on the Grid
        • Tweaking execution
        • Updating the GAMESS code
      • The ggeotop script
        • Introduction
        • Command-line invocation of ggeotop
        • Output files for ggeotop
        • Example usage
          • Manage a set of jobs from start to end
        • Using GC3Pie utilities
          • gkill: cancel a running job
          • ginfo: accessing low-level details of a job
      • The grosetta and gdocking scripts
        • Introduction
        • Command-line invocation of grosetta
        • Output files for grosetta
        • Output files for gdocking
        • Example usage
          • Manage a set of jobs from start to end
          • Managing a session by repeated grosetta invocation
      • The gcrypto script
        • Introduction
        • Command-line invocation of gcrypto
        • Output files for gcrypto
        • Example usage
          • Manage a set of jobs from start to end
        • Using GC3Pie utilities
          • gkill: cancel a running job
          • ginfo: accessing low-level details of a job
    • The GC3Utils software
      • gsession: manage sessions
      • gstat: monitor the status of submitted jobs
      • gtail: peeking at the job output and error report
      • gkill: cancel a running job
      • gget: retrieve the output of finished jobs
      • gclean: remove a completed job from the status list
      • gresub: re-submit a failed job
      • gservers: list available resources
      • ginfo: accessing low-level details of a job
      • gcloud: manage VMs created by the EC2 backend
    • Troubleshooting GC3Pie
      • What is a Python traceback?
      • Common errors using GC3Pie
        • AttributeError: args
        • AttributeError: module object has no attribute StringIO
        • DistributionNotFound
        • FTPControlError: Server responded: File unavailable.
        • ImportError: .../site-packages/_arclib.so: undefined symbol: Py_InitModule4
        • ImportError: No module named pstats
        • NoResources: Could not initialize any computational resource - please check log and configuration file.
        • Terminate called after throwing an instance of LdapQueryError
        • ValueError: I/O operation on closed file
• Programmer Documentation
  • Programming overview
    • Computational job lifecycle
    • Managing jobs with GC3Libs
    • Execution model of GC3Libs applications
      • Application exit codes
      • Application execution states
      • Computational job specification
      • UML Diagram
  • GC3Pie programming tutorials
    • Slides on specific topics
    • Example scripts
    • Warholize Tutorial
      • Introduction
      • From top to bottom
      • The script
      • The workflows
        • Main sequential workflow
        • Step one: convert to grayscale
      • Step two: parallel workflow to create colorized images
        • Step three: merge all them together
      • Making the script executable
      • Testing
  • GC3Libs programming API
    • gc3libs
    • gc3libs.application
    • gc3libs.application.apppot
    • gc3libs.application.codeml
    • gc3libs.application.demo
    • gc3libs.application.gamess
    • gc3libs.application.rosetta
    • gc3libs.application.turbomole
    • gc3libs.authentication
    • gc3libs.authentication.grid
    • gc3libs.authentication.ssh
    • gc3libs.backends
    • gc3libs.backends.arc0
    • gc3libs.backends.arc1
    • gc3libs.backends.batch
    • gc3libs.backends.lsf
    • gc3libs.backends.pbs
    • gc3libs.backends.sge
    • gc3libs.backends.shellcmd
    • gc3libs.backends.slurm
    • gc3libs.backends.transport
    • gc3libs.cmdline
    • gc3libs.config
    • gc3libs.core
    • gc3libs.debug
    • gc3libs.exceptions
    • gc3libs.optimizer
    • gc3libs.optimizer.dif_evolution
    • gc3libs.optimizer.drivers
    • gc3libs.optimizer.extra
    • gc3libs.persistence
      • A usage warning
    • gc3libs.persistence.accessors
    • gc3libs.persistence.filesystem
    • gc3libs.persistence.idfactory
    • gc3libs.persistence.serialization
    • gc3libs.persistence.sql
    • gc3libs.persistence.store
    • gc3libs.quantity
    • gc3libs.session
    • gc3libs.template
    • gc3libs.url
    • gc3libs.utils
    • gc3libs.workflow
    • gc3utils
    • gc3utils.commands
    • gc3utils.frontend
• Developer Documentation
  • Contributing to GC3Pie
    • Code repository organization
      • Repository structure
      • How can I get access to the SVN repository?
    • Package generation
    • Testing the code
      • Running the tests
        • Running tests with nosetests
        • Running multiple tests
        • Running tests on Python 2.5
      • Organizing tests
      • Writing tests
        • Testing for errors
        • Skipping tests
        • Generating tests
        • Grouping tests into classes
      • Opening the python debugger while running a test
    • Coding style
    • Documentation
    • Questions?
• List of contributors to GC3Pie
• Glossary

Indices and tables

• Index
• Module Index
• Search Page