compute_confusion_matrix#

DetectionEvaluator.compute_confusion_matrix(predictions_names: str | Iterable[str] | None = None, groups: str | ContinuousGroup | Sequence[str | ContinuousGroup] = (), min_iou: float = 0, min_confidence: float = 0) → DataFrame[source]#

Compute confusion matrix to evaluate object detection.

See also

related tutorial

Parameters:

  • predictions_names – name or collection of prediction names to compute the matches on. If set to None, will compute the matches with the prediction DataFrames contained in the self.predictions_dictionary attribute. Default to None

  • groups – Groups of image or annotation attributes to use to partition evaluation results to compute multiple confusion matrices. Must be a group_list . Defaults to ()

  • min_iou – IoU above which the detection is considered valid. Defaults to 0. Note that the lower bound of min_iou is not inclusive.

  • min_confidence – confidence threshold above which the detection is considered valid. Defaults to 0. Note that the lower bound of min_confidence is not inclusive.

Returns:

A Dataframe with confusion data for each group name(s) (if any) of each predictions_names.

Example

>>> from lours.utils.doc_utils import dummy_dataset
>>> groundtruth = dummy_dataset(
...     10,
...     1000,
...     label_map={0: "person", 1: "car"},
... )
>>> predictions1 = dummy_dataset(
...     10,
...     10000,
...     label_map=groundtruth.label_map,
...     images=groundtruth.images,
...     add_confidence=True,
...     seed=0,
... )
>>> predictions2 = dummy_dataset(
...     10,
...     10000,
...     label_map=groundtruth.label_map,
...     images=groundtruth.images,
...     add_confidence=True,
...     seed=1,
... )
>>> evaluator = DetectionEvaluator(
...     groundtruth=groundtruth, A=predictions1, B=predictions2
... )
>>> evaluator.compute_confusion_matrix().reset_index().set_index(
...     ["model", "label"]
... )
computing matches between groundtruth and A (category agnostic)
computing matches between groundtruth and B (category agnostic)
Processing confusion matrix for model=A
Processing confusion matrix for model=B
                   car    person  None
model label
A     car     0.487179  0.512821   0.0
      person  0.470588  0.529412   0.0
      None    0.500889  0.499111   0.0
B     car     0.495069  0.504931   0.0
      person  0.470588  0.529412   0.0
      None    0.505556  0.494444   0.0

You can also use a minimum iou and select a subset of all prediction sets.

>>> evaluator.compute_confusion_matrix(
...     min_iou=0.1, predictions_names="A"
... ).drop("model", axis=1)
Processing confusion matrix for model=A
             car    person      None
label
car     0.362919  0.374753  0.262327
person  0.312373  0.367140  0.320487
None    0.500377  0.499623  0.000000