Object Tracking

Object tracking can be used to follow an object across a series of frames, or to determine whether a bounding box from a new detection is delineating the same object as a previous detection.

Following an Object Across Frames

A common usage of tracking is to reduce inference time overhead by running an object detector periodically and using a tracker to update bounding box positions for the intermediate frames. The CorrelationTracker class will update the position of bounding boxes based on the content of a newer frame. A simple example of that usage follows.

First, instantiate an ObjectDetection object and load the engine and accelerator using the load() function. The object detector will be used to create bounding boxes around the objects we’ll be tracking:

obj_detect = edgeiq.ObjectDetection("alwaysai/mobilenet_ssd")
obj_detect.load(engine=edgeiq.Engine.DNN)

Then instantiate the CorrelationTracker object:

tracker = edgeiq.CorrelationTracker()

Perform an object detection using the detect_objects() function, which will return a ObjectDetectionResults object containing a list of ObjectDetectionPrediction:

results = obj_detect.detect_objects(frame)

To start tracking each object that was detected, pass each ObjectDetectionPrediction directly to the tracker’s start() function, along with the frame used to originally detect the object.

for prediction in results.predictions:
  tracker.start(frame, prediction)

For each new frame, update the tracker using the update() function. The tracker uses correlation to follow an object as it moves frame-to-frame. The output format is of the same type that was originally provided in the start() function, which is ObjectDetectionPrediction:

tracker_predictions = tracker.update(frame)

It can be useful to visualize the detections. The image can be marked with the results of the detection using the markup_image() function:

new_image = edgeiq.markup_image(image, object_detection_predictions)

The markup_image() function takes as its input an image (the one used for object detection) and a list of ObjectDetectionPrediction objects and draws the bounding boxes and labels on the image.

When another detection is performed, tracking should be stopped for all currently tracked objects so that the new detections can be tracked. The count() attribute will return the number of objects currently tracked, and stop_all() will stop tracking all objects.:

# Perform another detection
results = obj_detect.detect_objects(frame)

# Stop tracking old objects and start tracking new ones
if tracker.count:
  tracker.stop_all()
for prediction in results.predictions:
  tracker.start(frame, prediction.box, prediction.label)

Following an Object Across Detections

Another common use of tracking is to identify unique objects between detections. The CentroidTracker class will match a new set of bounding boxes with a previous set of bounding boxes. A simple example of that usage follows.

First, instantiate an ObjectDetection object and load the engine and accelerator using the load() function. The object detector will be used to create bounding boxes around the objects we’ll be tracking:

obj_detect = edgeiq.ObjectDetection("alwaysai/mobilenet_ssd")
obj_detect.load(engine=edgeiq.Engine.DNN)

Then instantiate the CentroidTracker object:

tracker = edgeiq.CentroidTracker()

Perform an object detection using the detect_objects() function, which will return a ObjectDetectionResults object containing a list of ObjectDetectionPrediction:

results = obj_detect.detect_objects(frame)

Perform the initial tracker by calling the update() function to register all detected objects:

objects = tracker.update(results.predictions)

The update() function returns a dictionary with a unique key for each object, as well as the ObjectDetectionPrediction that goes with that object. After running another detection, the dictionary will match the new bounding boxes to the original object IDs, giving you the ability to perform processing for a specific object across multiple detections.