faster_etapr.etapr.evaluate_from_ranges#

faster_etapr.etapr.evaluate_from_ranges(preds: list[tuple[int, int]], labels: list[tuple[int, int]], *, theta_p: float = 0.5, theta_r: float = 0.1) → dict[str, float | int][source]#

Calculates the enhanced time-aware (eTa), point-wise, and point-adjusted performance metrics (and some other miscellaneous metrics). To see how these metrics are calculated, check out the respective methods in eTaMetrics.

Parameters:

  • y_hat (list[tuple[int, int]]) – Predictions as list of ranges.

  • y (list[tuple[int, int]]) – Labels as list of ranges.

  • theta_p (float, optional) – Precision threshold. Only those predictions who overlap with at least theta_p with a detected anomaly are counted as correct. Defaults to 0.5.

  • theta_r (float, optional) – Recall threshold. Only those anomalies which overlap at least theta_r with an correct prediction are counted as detected. Defaults to 0.1.

Returns:

Returns a mapping with all metrics:

  • eta/recall: eTa recall score

  • eta/recall_detection: detection score of the recall

  • eta/recall_portion: portion score of the recall

  • eta/detected_anomalies: number of detected anomalies

  • eta/precision: eTa precision score

  • eta/precision_detection: detection score of the precision

  • eta/precision_portion: portion score of the precision

  • eta/correct_predictions: number of correct predictions

  • eta/f1: f1 score (harmonic mean of precision and recall)

  • eta/TP: number of true positives (points counted)

  • eta/FP: number of false positives (points counted)

  • eta/FN: number of false negatives (points counted)

  • eta/wrong_predictions: number of wrong predictions

  • eta/missed_anomalies: number of undetected anomalies

  • eta/anomalies: total number of anomalies

  • eta/segments: percentage of detected anomalies

  • point/recall: point-wise recall (TP / (TP + FN))

  • point/precision: point-wise precision (TP / (TP + FP))

  • point/f1: point-wise f1 score

  • point/TP: number of true positives, correctly classified as 1

  • point/FP: number of false positive, incorrectly classified as 1

  • point/FN: number of false negatives, incorrectly classified as 0

  • point/anomalies: total number of anomalies

  • point/detected_anomalies: number of detected anomalies (at least one point detected)

  • point/segments: percentage of detected anomalies

  • point_adjust/recall: point-adjusted recall

  • point_adjust/precision: point-adjusted precision

  • point_adjust/f1: point-adjusted f1

Return type:

dict[str, float | int]

Example

>>> import faster_etapr
>>> faster_etapr.evaluate_from_ranges(
...     y_hat=[(1, 1), (3, 4), (7, 9), (11, 13)],
...     y=    [(1, 2), (5, 7), (10, 14), (16, 16)],
...     theta_p=0.5,
...     theta_r=0.1,
... )
{
    'eta/recall': 0.3875,
    'eta/recall_detection': 0.5,
    'eta/recall_portion': 0.275,
    'eta/detected_anomalies': 2.0,
    'eta/precision': 0.46476766302377037,
    'eta/precision_detection': 0.46476766302377037,
    'eta/precision_portion': 0.46476766302377037,
    'eta/correct_predictions': 2.0,
    'eta/f1': 0.4226312395393011,
    'eta/TP': 4,
    'eta/FP': 5,
    'eta/FN': 7,
    'eta/wrong_predictions': 2,
    'eta/missed_anomalies': 2,
    'eta/anomalies': 4,
    'eta/segments': 0.499999999999875,
    'point/recall': 0.45454545454541323,
    'point/precision': 0.5555555555554939,
    'point/f1': 0.49999999999945494,
    'point/TP': 5,
    'point/FP': 4,
    'point/FN': 6,
    'point/anomalies': 4,
    'point/detected_anomalies': 3.0,
    'point/segments': 0.75,
    'point_adjust/recall': 0.9090909090909091,
    'point_adjust/precision': 0.7142857142857143,
    'point_adjust/f1': 0.7999999999995071
}