faster_etapr.etapr.evaluate_from_preds#

Calculates the enhanced time-aware (eTa), point-wise, and point-adjusted performance metrics (and some other miscellaneous metrics). To see how these metrics are calculated, check out the respective methods in eTaMetrics.

Parameters:

y_hat (npt.ArrayLike) – Predictions (point-wise).
y (npt.ArrayLike) – Labels (point-wise).
theta_p (float, optional) – Precision threshold. Only those predictions who overlap with at least theta_p with a detected anomaly are counted as correct. Defaults to 0.5.
theta_r (float, optional) – Recall threshold. Only those anomalies which overlap at least theta_r with an correct prediction are counted as detected. Defaults to 0.1.

Returns:

Returns a mapping with all metrics:

eta/recall: eTa recall score
eta/recall_detection: detection score of the recall
eta/recall_portion: portion score of the recall
eta/detected_anomalies: number of detected anomalies
eta/precision: eTa precision score
eta/precision_detection: detection score of the precision
eta/precision_portion: portion score of the precision
eta/correct_predictions: number of correct predictions
eta/f1: f1 score (harmonic mean of precision and recall)
eta/TP: number of true positives (points counted)
eta/FP: number of false positives (points counted)
eta/FN: number of false negatives (points counted)
eta/wrong_predictions: number of wrong predictions
eta/missed_anomalies: number of undetected anomalies
eta/anomalies: total number of anomalies
eta/segments: percentage of detected anomalies
point/recall: point-wise recall (TP / (TP + FN))
point/precision: point-wise precision (TP / (TP + FP))
point/f1: point-wise f1 score
point/TP: number of true positives, correctly classified as 1
point/FP: number of false positive, incorrectly classified as 1
point/FN: number of false negatives, incorrectly classified as 0
point/anomalies: total number of anomalies
point/detected_anomalies: number of detected anomalies (at least one point detected)
point/segments: percentage of detected anomalies
point_adjust/recall: point-adjusted recall
point_adjust/precision: point-adjusted precision
point_adjust/f1: point-adjusted f1

Return type:

dict[str, float | int]

Example

>>> import faster_etapr
>>> faster_etapr.evaluate_from_preds(
...     y_hat=[0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0],
...     y=    [0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1],
...     theta_p=0.5,
...     theta_r=0.1,
... )
{
    'eta/recall': 0.3875,
    'eta/recall_detection': 0.5,
    'eta/recall_portion': 0.275,
    'eta/detected_anomalies': 2.0,
    'eta/precision': 0.46476766302377037,
    'eta/precision_detection': 0.46476766302377037,
    'eta/precision_portion': 0.46476766302377037,
    'eta/correct_predictions': 2.0,
    'eta/f1': 0.4226312395393011,
    'eta/TP': 4,
    'eta/FP': 5,
    'eta/FN': 7,
    'eta/wrong_predictions': 2,
    'eta/missed_anomalies': 2,
    'eta/anomalies': 4,
    'eta/segments': 0.499999999999875,
    'point/recall': 0.45454545454541323,
    'point/precision': 0.5555555555554939,
    'point/f1': 0.49999999999945494,
    'point/TP': 5,
    'point/FP': 4,
    'point/FN': 6,
    'point/anomalies': 4,
    'point/detected_anomalies': 3.0,
    'point/segments': 0.75,
    'point_adjust/recall': 0.9090909090909091,
    'point_adjust/precision': 0.7142857142857143,
    'point_adjust/f1': 0.7999999999995071
}