Measure End-to-End Latency
The camera captures a frame every second but the service responds too late.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
The camera captures a frame every second but the service responds too late.
A practical question
A model predicts occupancy in a fraction of a second, yet the sign at the garage entrance updates several seconds late. The model's runtime is only one part of the delay.
The useful idea
Measure capture, transfer, decode, preprocessing, inference, postprocessing, review, and display as one path. Use timestamps that make each stage comparable. Report typical delay and slow outliers, since a rare backlog can matter during peak traffic.
Work through the example
Follow one car entering a space and note when the displayed state changes. If the delay comes from video buffering, a faster neural model may not help. Decide which stages must be reliable even when frames are dropped. Start a timer when a vehicle is captured and stop it when the displayed count reflects that observation. Separate camera buffering, transfer, decoding, inference, tracking, and display. If the model runs quickly but the sign updates late, shortening model inference may not solve the visitor's problem. Measure slow outliers at busy load and decide how stale an occupancy number may be before the sign should mark it uncertain. Use this case to test your understanding: Measure the time from exposure to the displayed occupancy count at busy load. Here is the limitation to watch for: A fast network alone cannot repair a slow upstream pipeline. The answer follows from that evidence: End-to-end latency includes capture, transfer, inference, association, and output, not just model time. The driver sees the sign, not the model timer. Measure from camera exposure to the displayed count and include busy periods. If frames queue faster than they are processed, the sign may show a correct answer to an old scene. Show that age to the operator when delay passes the accepted limit, rather than displaying stale information as current.
Limits and next step
Next, we'll tune the confidence threshold with the human review capacity and the cost of mistakes in mind.
Worked example
Measure the time from exposure to the displayed occupancy count at busy load.
Quick check: test your understanding
Try it yourself: Consider this case: Measure the time from exposure to the displayed occupancy count at busy load. Explain how you would check the result. In your answer, address this limitation: A fast network alone cannot repair a slow upstream pipeline.
Check the reasoning
End-to-end latency includes capture, transfer, inference, association, and output, not just model time. A fast network alone cannot repair a slow upstream pipeline.
What is the main idea behind measure end-to-end latency?
End-to-end latency includes capture, transfer, inference, association, and output, not just model time.
How is it used in the example?
Measure the time from exposure to the displayed occupancy count at busy load.
What limitation should be checked?
A fast network alone cannot repair a slow upstream pipeline.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 73 transcript305 words · Narration ready for the upcoming videoOpen transcript
A model predicts occupancy in a fraction of a second, yet the sign at the garage entrance updates several seconds late. The model's runtime is only one part of the delay.
Measure capture, transfer, decode, preprocessing, inference, postprocessing, review, and display as one path. Use timestamps that make each stage comparable. Report typical delay and slow outliers, since a rare backlog can matter during peak traffic.
Follow one car entering a space and note when the displayed state changes. If the delay comes from video buffering, a faster neural model may not help. Decide which stages must be reliable even when frames are dropped. Start a timer when a vehicle is captured and stop it when the displayed count reflects that observation. Separate camera buffering, transfer, decoding, inference, tracking, and display. If the model runs quickly but the sign updates late, shortening model inference may not solve the visitor's problem. Measure slow outliers at busy load and decide how stale an occupancy number may be before the sign should mark it uncertain. Use this case to test your understanding: Measure the time from exposure to the displayed occupancy count at busy load. Here is the limitation to watch for: A fast network alone cannot repair a slow upstream pipeline. The answer follows from that evidence: End-to-end latency includes capture, transfer, inference, association, and output, not just model time. The driver sees the sign, not the model timer. Measure from camera exposure to the displayed count and include busy periods. If frames queue faster than they are processed, the sign may show a correct answer to an old scene. Show that age to the operator when delay passes the accepted limit, rather than displaying stale information as current.
Next, we'll tune the confidence threshold with the human review capacity and the cost of mistakes in mind.