Tracking an Object Across Frames
A detector finds three cars in one frame and three in the next.
Tracking an Object Across Frames explained visually
A detector finds three cars in one frame and three in the next. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
A detector finds three cars in one frame and three in the next.
Connect observations over time
A detector finds three cars in one frame and three in the next. Which new box belongs to which earlier car? Tracking addresses that association problem. It combines observations across time and may use position, motion, class, and appearance. The goal is consistency, but the available evidence can become weak when objects overlap or disappear.
Prediction supports association
A tracker often predicts where an object should move, then compares that prediction with new detections. Motion models can bridge short gaps, while appearance features may help distinguish nearby objects. Fast Tracking and Stable Tracking in an annotation workflow provide different tracking choices, but neither removes the need to inspect the generated box. A plausible prediction can still drift onto another object.
Review uncertain estimates
Occlusion is especially important. If an object disappears behind an obstacle, the tracker has less direct evidence. An estimated box can support an efficient review workflow, but it should not be confused with a confirmed observation. Correct the location when necessary and preserve intentional annotations. Evaluate identity switches, lost tracks, localization quality, and response time together. A tracker that is fast but frequently swaps identities may create more annotation work than it saves.
A worked example
Two similar cars cross paths. Their predicted positions overlap, so location alone may associate the boxes incorrectly. Appearance can help, but identical-looking cars or heavy occlusion can still make the identity uncertain. Reviewing the crossing frames is more informative than checking only easy straight-line motion.
Worked example
Two similar cars cross paths. Their predicted positions overlap, so location alone may associate the boxes incorrectly. Appearance can help, but identical-looking cars or heavy occlusion can still make the identity uncertain. Reviewing the crossing frames is more informative than checking only easy straight-line motion.
Quick check: test your understanding
Try it yourself: Create a short tracking test with clear motion, partial occlusion, and two similar objects crossing. Record different error types separately.
Check the reasoning
Measure drift, lost observations, identity switches, and time per frame. Do not summarize all of them as a single success count.
Is a box predicted during occlusion a direct measurement of the hidden object?
No. It is an estimate based on the available history and model assumptions.
How would you solve the practice case?
Measure drift, lost observations, identity switches, and time per frame. Do not summarize all of them as a single success count.
What does the worked example show?
Two similar cars cross paths. Their predicted positions overlap, so location alone may associate the boxes incorrectly. Appearance can help, but identical-looking cars or heavy occlusion can still make the identity uncertain. Reviewing the crossing frames is more informative than checking only easy straight-line motion.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 8 transcript235 words · Video transcript · Matches the published narration and captionsOpen transcript
A detector finds three cars in one frame and three in the next. Which new box belongs to which earlier car? Tracking addresses that association problem. It combines observations across time and may use position, motion, class, and appearance. The goal is consistency, but the available evidence can become weak when objects overlap or disappear.
A tracker often predicts where an object should move, then compares that prediction with new detections. Motion models can bridge short gaps, while appearance features may help distinguish nearby objects. Fast Tracking and Stable Tracking in an annotation workflow provide different tracking choices, but neither removes the need to inspect the generated box. A plausible prediction can still drift onto another object.
Occlusion is especially important. If an object disappears behind an obstacle, the tracker has less direct evidence. An estimated box can support an efficient review workflow, but it should not be confused with a confirmed observation. Correct the location when necessary and preserve intentional annotations. Evaluate identity switches, lost tracks, localization quality, and response time together. A tracker that is fast but frequently swaps identities may create more annotation work than it saves.
Two similar cars cross paths. Their predicted positions overlap, so location alone may associate the boxes incorrectly. Appearance can help, but identical-looking cars or heavy occlusion can still make the identity uncertain. Reviewing the crossing frames is more informative than checking only easy straight-line motion.