The CROC Package

A package for calculating ROC curves and Concentrated ROC (CROC) curves written by Dr. S. Joshua Swamidass.

Citation

Please cite this paper when reporting any work which uses this software:

A CROC Stronger than ROC: Measuring, Visualizing, and Optimizing Early Retrieval

S. Joshua Swamidass, Chloe-Agathe Azencott, Kenny Daily and Pierre Baldi

Bioinformatics, April 2010, doi:10.1093/bioinformatics/btq140

Description

This pure-python package is designed to be a standardized implementation of performance curves and metrics for use either in python scripts or through a simple commandline interface. As a standardized implementation its output is robust enough to be using in publishable scientific work.

With this package, one can easily:

1. compute the coordinates of both Accumulation Curves and ROC curves.
2. handle ties appropriately using several methods.
3. compute the BEDROC metric.
4. vertically add and average the performance curves of several cross-validation folds.
5. focus on the early part of the ROC curve by using several x-axis transforms.

Documentation

The docstrings in this module are fairly complete and the scripts provide simple access to the most common functions. Further documentation can be found here:

• Installation Instructions
  • Requirements
  • From Source
  • Easy Install
  • Testing the Installation
• Scripts
  • croc-area
  • croc-average
  • croc-bedroc
  • croc-curve
• File Formats
  • Scored-Label Files
  • Rank Files
  • Curve Files
• R Interface
• The Croc API

R-Interface

Daniel Himmelstein has written up a basic R interface to CROC which is avialable on github (here).

Development Status

On 3/14/2010 this project officially moved out of beta and is now designated a stable release. Please email the author if you discover any bugs.

Future versions will include several enhancements, including:

1. the significance tests described in paper.
2. better performance and removal of dependence on Sympy.

The priority of these improvements will be, in part, directed by interest from users.