revscoring.scorer_models.test_statistics

A collection of statistics generators that can be applied to revscoring.ScorerModel.

class revscoring.scorer_models.test_statistics.accuracy(**kwargs)

Constructs an accuracy generator.

When applied to a test set, the score() method will return a float representing the proportion of correct predicitions.

class revscoring.scorer_models.test_statistics.precision(**kwargs)

Constructs an accuracy generator.

When applied to a test set, the score() method will return a float representing the proportion of correct predicitions.

class revscoring.scorer_models.test_statistics.recall(**kwargs)

Constructs an accuracy generator.

When applied to a test set, the score() method will return a float representing the proportion of correct predicitions.

class revscoring.scorer_models.test_statistics.roc(**kwargs)

Constructs a reciever operating characteristic statistics generator. See https://en.wikipedia.org/wiki/Receiver_operating_characteristic

When applied to a test set, the score() method will return a dictionary with four fields:

• auc: the area under the ROC curve
• fprs: a list of false-positive rates
• tprs: a list of true-positive rates
• thresholds: a list of probability thresholds

class revscoring.scorer_models.test_statistics.precision_recall(**kwargs)

Constructs a precision/recall statistics generator. See https://en.wikipedia.org/wiki/Precision_and_recall

When applied to a test set, the score() method will return a dictionary with four fields:

• auc: the area under the precision-recall curve
• precisions: a list of precisions
• recalls: a list of recalls
• thresholds: a list of probability thresholds

class revscoring.scorer_models.test_statistics.recall_at_fpr(max_fpr)

Constructs a statistics generator that measures the maximum recall that can be achieved at maximum false-positive rate. As a classifier gets better, the attainable recall at low false-positive rates should increase.

When applied to a test set, the score() method will return a dictionary with three fields:

• threshold: The probability threshold where recall was maximized
• recall: The recall at threshold
• fpr: The false-positive rate at threshold

Parameters:

max_fpr : float

Maximum false-positive rate that will be tolerated

class revscoring.scorer_models.test_statistics.filter_rate_at_recall(min_recall)

Constructs a statistics generator that measures the maximum filter rate that can be achieved at minum recalle. As a classifier gets better, the attainable filter rate at high recall values should go up.

When applied to a test set, the score() method will return a dictionary with three fields:

• threshold: The probability threshold where filter rate was maximized
• filter_rate: The filter rate at threshold
• recall: The recall at threshold

Parameters:

min_recall : float

The minimum recall proportion that will be tolerated

Abstract classes

class revscoring.scorer_models.test_statistics.TestStatistic(**kwargs)

Represents a test statistic.

class revscoring.scorer_models.test_statistics.ClassifierStatistic(**kwargs)

Represents a test statistic for classifier models.