Going deeper

To go deeper in the library here are some functions anc classes that aren’t necessary to use when using classic trainning and inference pipeline but allow to develop custom processes.

class detectools.inference.InferenceImage(image: Tensor, patch_size=typing.Tuple[int], overlap: float = 0.0)

Class to process patchification/unpatchification in inference pipeline. Pacthification is automatically done during construction.

Parameters:

• image (Tensor) – Image to predict on.

• patch_size (_type_, optional) – Patch size to predict with model forward pass. Defaults to Tuple[int].

• overlap (float, optional) – Proportion of overlap between patches. Defaults to 0.0.

Attributes:

patch_size

Size of patch to slice original image.

Type:

Tuple[int, int]

size

Size of original image.

Type:

Tuple[int, int]

overlap

List of to left corner patches coordinates.

Type:

List[Tuple[int, int]]

padded_size

Size of padded image to patchify.

Type:

Tuple[int, int]

Methods:

get_patches() → TensorDataset

Return patches as TensorDataset for batchification.

Returns:

• Patches Dataset.

Return type:

TensorDataset

rebuild_prediction(predictions: List[BaseFormat]) → BaseFormat

Merge predictions at corresponding positions the retroieve original image size by cropping predictions.

Parameters:

predictions (List[Format]) – Patches predictions.

Returns:

• Prediction rebuilt at original image size.

Return type:

Format

class detectools.models.BaseModel(*args, **kwargs)

Base Class for detectools models.

Attributes:

confidence_thr

Confidence score threshold to consider object as true prediction.

Type:

float

max_detection

Maximum number of object to predict on one image.

Type:

int

nms_threshold

IoU threshold to consider 2 boxes as overlapping for Non Max Suppression algorithm.

Type:

float

num_classes

Number of classes.

Type:

int

Methods:

classmethod build_results(raw_outputs: Any) → BatchedFormats

Transform model outputs into BaseFormat for results. This function also apply instances selection on results according to args:

• confidence_thr

• max_detection

• nms_threshold

Parameters:

raw_outputs (Any) – Model outputs.

Returns:

• Model output for batch.

Return type:

BatchedFormats

classmethod get_predictions(images: Tensor) → BatchedFormats

Prepare images, Apply model forward pass and build results.

Parameters:

images (Tensor) – RGB images Tensor.

Returns:

• Predictions for images as BatchedFormats.

Return type:

BatchedFormats

classmethod prepare(images: Tensor, targets: BatchedFormats | None = None) → Any | Tuple[Any]

Transform images and targets into model specific format for prediction & loss computation.

Parameters:

• images (Tensor) – Batch images.

• targets (BatchedFormats, optional) – Batched targets from DetectionDataset.

Returns:

• Images data prepared for model.

• If targets: images + targets prepared for model.

Return type:

Union[Any, Tuple[Any]]

classmethod run_forward(images: Tensor, targets: BatchedFormats, predict: bool = False) → Dict[str, Tensor] | Tuple[Dict[str, Tensor], BatchedFormats]

Compute loss from images and if target passed, compute loss & return both loss dict and results.

Parameters:

• images (Tensor) – Batch RGB images.

• targets (BatchedFormats) – Batch targets.

• predict (bool, optional) – To return predictions or not. Defaults to False.

Returns:

• Loss dict.

• If predict: Predictions.

Return type:

Union[Dict[str, Tensor], Tuple[Dict[str, Tensor], BatchedFormats]]

classmethod to_device(device: Literal['cpu', 'cuda'])

Send model to device.

Parameters:

device (Literal['cpu', 'cuda']) – Device to send model on.

class detectools.formats.BaseAnnotation

Abstract class for Annotation data container.

Attributes:

boxe

Boxe coordinates in XYWH format.

Type:

BoundingBoxes

label

Class label of object.

Type:

Tensor

spatial_size

Size of corresponding image (H, W)

Type:

Tuple[int, int]

score

Confidence score of the object (for prediction).

Type:

Tensor

Methods:

abstract object_to_coco(annotation_id: int = 1, image_id: int = 1) → Dict[str, Any]

Return Annotation data as COCO like dict.

Parameters:

• annotation_id (int, optional) – Id of the annotation. Defaults to 1.

• image_id (int, optional) – Id of the corresponding image. Defaults to 1.

Returns:

• COCO like dict with Annotation instance data.

Return type:

Dict[str, Any]

class detectools.formats.BaseFormat

Abstract class for detection data container classes in detectools. Store target and predictions data. This data format support basics and advanced operations (padding, cropping, NMS, etc.).

Attributes:

box_format

Format of bounding boxes.

Type:

Literal["XYWH", "XYXY", "CXCYWH"]

spatial_size

Size of corresponding image (H, W)

Type:

Tuple[int, int]

size

Number of objects in BaseFormat.

Type:

int

data

(Dict[str, Tensor]): Data dict that contains objects informations in it’s keys (labels, boxes, scores, masks).

Type:

Dict[str, torch.Tensor]

Methods:

clone() → BaseFormat

Return a clone of BaseFormat.

Returns:

• Cloned BaseFormat.

Return type:

BaseFormat

coco(image_id: int = 1, annotation_id: int = 1) → List[Dict[str, Any]]

Export data as COCO annotations.

Parameters:

• image_id (int, optional) – Id to write for “image_id” field in annotation dict. Defaults to 1.

• annotation_id (int, optional) – Id to write on the first annotation dict “id” field. Following ones have id indent from this one. Defaults to 1.

Returns:

• Coco annotations list for Format corresponding image.

Return type:

List[Dict[str, Any]]

confidence(confidence_threshold: float = 0.5) → BaseFormat

Keep only objects with confidence above confidence_threshold.

Parameters:

confidence_threshold (float, optional) – Minimum confidence to keep object. Defaults to 0.5.

Returns:

• Format with only objects with scores > confidence_thr.

Return type:

BaseFormat

convert_labels(convert_labels_dict: Dict[int, int])

Convert labels of Format.

Parameters:

convert_labels_dict (Dict[int, int]) – Dict of converion {old_labels:new_labels}.

abstract crop(top: int, left: int, height: int, width: int)

Crop boxes and mask from top corner pixel and update spatial size.

Parameters:

• top (int) – Position to crop from top border.

• left (int) – Position to crop from left border.

• height (int) – height of the crop.

• width (int) – Width of the crop.

abstract empty() → BaseFormat

Return an empty instance of BaseFormat (DetectionFormat or SegmentationFormat depending on the Task mode).

Parameters:

spatial_size (Tuple[int]) – Size (H, W) of the corresponding image.

Returns:

• BaseFormat instance.

Return type:

BaseFormat

from_coco(coco_annotations: List[Dict[str, Any]], spatial_size: Tuple[int]) → BaseFormat

Return BaseFormat from an image COCO data dictionnary.

Parameters:

• coco_annotations (List[Dict[str, Any]]) – Coco data dictionnary.

• spatial_size (Tuple[int]) – Size (H, W) of the corresponding image.

Returns:

• BaseFormat instance.

Return type:

BaseFormat

get(*keys: str) → Tensor | Tuple[Tensor]

Return tensor data values from data dict for each key in keys.

Parameters:

keys (str) – Key(s) of data dict.

Returns:

• Corresponding key’s values to gather from data dict.

Return type:

Union[Tensor, Tuple[Tensor]]

>>> format: BaseFormat
>>> labels, boxes = format.get("labels", "boxes")

get_device() → device

Verify that all tensors of data dict are on same device and return device.

Returns:

• Device that hold tensor values of data dict.

Return type:

torch.device

abstract get_object(indice: int) → BaseAnnotation

Return a BaseAnnotation object at position indice.

Parameters:

indice (int) – Position of object to gather.

Returns:

• Annotation instance.

Return type:

BaseAnnotation

match(format2: BaseFormat) → bool

Check if 2 Formats match for combinations: - check if both contains same keys on data dictionnary. - check if spatial size is equivalent.

Parameters:

• format1 (BaseFormat) – BaseFormat 1.

• format2 (BaseFormat) – BaseFormat 2.

Returns:

• True if BaseFormats matchs else False.

Return type:

bool

max_detections(maximum_objects: int) → BaseFormat

Retrieve N (maximum objects) with highest scores.

Parameters:

maximum_objects (int) – Number of object to keep.

Returns:

• Format with N objects with highest scores.

Return type:

BaseFormat

nms(iou_threshold=0.5) → BaseFormat

Apply non maximum suppression algorithm to format.

Parameters:

iou_threshold (float, optional) – Threshold to consider boxes as overlapping. Defaults to 0.5.

Returns:

• Format with objects selected by NMS

Return type:

BaseFormat

normalize()

Normalize boxes values between 0 & 1 by dividing by spatial_size.

abstract pad(left: int, top: int, right: int, bottom: int)

Pad boxes and mask and update spatial size.

Parameters:

• left (int) – Pad value on left border.

• top (int) – Pad value on top border.

• right (int) – Pad value on right border.

• bottom (int) – Pad value on bottom border.

rescale()

Rescale normalized boxes to true scale with spatial size.

sanitize(min_box_sides: float) → BaseFormat

Remove objects with boxes that have one of their sides smaller than min_box_sides.

Parameters:

min_box_sides (float) – Minimum size of border to keep boxes.

Returns:

• BaseFormat without small boxes.

Return type:

BaseFormat

set(key: str, value: Tensor)

Set a new pair of key/value. Value should be of shape (N, …) with N == self.size.

Parameters:

• key (str) – Key of value to set.

• value (Tensor) – Data as tensor.

set_boxes_format(box_format: Literal['XYWH', 'XYXY', 'CXCYWH'])

Change boxes format.

Parameters:

box_format (Literal['XYWH', 'XYXY', 'CXCYWH']) – Format to set for boxes.

set_device(device: device | Literal['cuda', 'cpu'])

Send all torch values of data dict on device.

Parameters:

device (Union[torch.device, Literal['cuda', 'cpu']]) – Device to send tensors on.

sort_by_scores(descending: bool = True) → BaseFormat

Sort objects by scores in decreasing order.

Parameters:

descending (bool, optional) – To sort objects in format with decreasing score order. Defaults to True.

Returns:

• Sorted BaseFormat.

Return type:

BaseFormat

class detectools.metrics.base.DetectMetric(func: Callable, iou_threshold: float = 0.5, name: str = 'DetectionMetric', **kwargs)

Base class for custom detection metric with torchmetrics engine.

compute() → Dict[str, Tensor]

Return metric computed with internal state.

Returns:

_description_

Return type:

Dict[str, Tensor]

update(prediction: Format, target: Format)

Update internal states of metric.

class detectools.metrics.base.ClassifMetric(func: Callable, num_classes: int = 1, name: str = 'ClassifMetric', **kwargs: Any)

Child class of torchmetrics metrics for classification. Allow to take Format as inputs and return dict of metric.

compute()

Comput metric with all averag strategy and return a dict with all values.

global_macro_compute() → Tuple[Tensor, Tensor]

Compute metric with global/macro averraging. Return also metric/class tensor.

global_micro_compute() → Tensor

Compute metric with global/micro averagging.

samplewise_macro() → Tensor

Compute metric with samplewise/macro averagging.

samplewise_micro() → Tensor

Compute metric with samplewise/micro averagging.

update(prediction: Format, target: Format)

Update internal states.

class detectools.metrics.base.SemanticSegmentationMetric(func: Callable, num_classes: int = 1, name: str = 'SegmentationMetric', **kwargs: Any)

Child class of classification metric. Move from instance to semantic segmentation paradigm to provide stats based on classes masks (instead of objects).

update(prediction: Format, target: Format)

Convert target & prediction to semantic mask to compute stats in semantic segmentation paradigm. Update internal state.

class detectools.metrics.base.MetricCompose(metrics: List[Metric], name='MetricCompose')

Wrap multiple DetectTM in a same object: - Merge metrics in same dictionnary output - Allow all metrics to be displayed on same graph.

compute()

Compute on each metrics and merge metric dicts together.

reset()

Reset all metrics.

update(prediction: Format, target: Format)

Update all metrics.