Convert Stereo Disparity to Depth
A stereo rig sees a nearby bollard shifted strongly between images.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
A stereo rig sees a nearby bollard shifted strongly between images.
A practical question
Two cameras look at the same parking-lot bollard from slightly different positions. After we line up their images, the top of the bollard appears at one horizontal location in the left picture and another in the right. The gap between those locations is called disparity. Nearby points usually have a larger disparity than distant points in a properly calibrated stereo pair. This is the visual clue that allows two cameras to estimate depth, but only when we match the same physical point in both views.
The useful idea
To turn that gap into a distance, we need two measured camera properties. One is the distance between the cameras, called the baseline. The other is the focal length expressed in pixels. For rectified views, depth is approximately the focal length multiplied by the baseline, divided by the disparity. Notice the inverse relationship: if two points are observed with the same camera setup and one has twice the disparity, its estimated depth is roughly half as large. This does not mean every pixel shift is a valid depth measurement.
Work through the example
Try a nearby cone and a far wall in the same pair of pictures. Mark a distinctive point on each object in both views, then measure the horizontal shifts. The cone should generally shift more. Before you report meters, check the units: using a focal length in pixels with a baseline in meters gives a depth in meters under this model. If the cameras moved between pictures or the images were not properly rectified, the simple horizontal comparison is no longer enough. Save the calibration so another person can reproduce your estimate.
Limits and next step
The difficult case is far away. Disparity becomes small there, so a one-pixel matching error can change the depth estimate a great deal. A shiny surface or repeated pattern can also tempt the matcher to choose the wrong point. Look at the image pair and the match, not only the resulting depth map. When the correspondence is doubtful, report uncertain depth instead of a precise-looking number. Next, we will examine the parts of a stereo scene where matching commonly fails.
Worked example
Mark the same point on a nearby cone and a far wall in both rectified views. Compare their horizontal shifts and explain which object should have the smaller depth.
Quick check: test your understanding
Try it yourself: If a cone has twice the disparity of a wall under the same calibrated stereo setup, how do their estimated depths compare? Why might a far-away wall have a less stable estimate?
Check the reasoning
The cone's estimated depth is about half the wall's. Far-away points have small disparity, so a small pixel-matching error has a larger effect on their estimated depth.
What is the main idea behind convert stereo disparity to depth?
For rectified stereo, depth is approximately focal length times baseline divided by disparity.
How is it used in the example?
At fixed focal length and baseline, a point with twice the disparity has roughly half the depth.
What limitation should be checked?
Small disparity errors can cause large depth errors far away.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 37 transcript346 words · Narration ready for the upcoming videoOpen transcript
Two cameras look at the same parking-lot bollard from slightly different positions. After we line up their images, the top of the bollard appears at one horizontal location in the left picture and another in the right. The gap between those locations is called disparity. Nearby points usually have a larger disparity than distant points in a properly calibrated stereo pair. This is the visual clue that allows two cameras to estimate depth, but only when we match the same physical point in both views.
To turn that gap into a distance, we need two measured camera properties. One is the distance between the cameras, called the baseline. The other is the focal length expressed in pixels. For rectified views, depth is approximately the focal length multiplied by the baseline, divided by the disparity. Notice the inverse relationship: if two points are observed with the same camera setup and one has twice the disparity, its estimated depth is roughly half as large. This does not mean every pixel shift is a valid depth measurement.
Try a nearby cone and a far wall in the same pair of pictures. Mark a distinctive point on each object in both views, then measure the horizontal shifts. The cone should generally shift more. Before you report meters, check the units: using a focal length in pixels with a baseline in meters gives a depth in meters under this model. If the cameras moved between pictures or the images were not properly rectified, the simple horizontal comparison is no longer enough. Save the calibration so another person can reproduce your estimate.
The difficult case is far away. Disparity becomes small there, so a one-pixel matching error can change the depth estimate a great deal. A shiny surface or repeated pattern can also tempt the matcher to choose the wrong point. Look at the image pair and the match, not only the resulting depth map. When the correspondence is doubtful, report uncertain depth instead of a precise-looking number. Next, we will examine the parts of a stereo scene where matching commonly fails.