Plan Safe Updates to the Vision System
A revised model improves recall but breaks several night scenes.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
A revised model improves recall but breaks several night scenes.
A practical question
A new model improves night occupancy estimates in one garage. Should we replace every camera's model immediately? First, compare it on all known conditions and record what changed.
The useful idea
Version the model, data split, bay geometry, thresholds, and annotation policy together. Test on held-out cameras and difficult cases, then release gradually while comparing predictions with reviewed outcomes. Keep a rollback path to the last working version.
Work through the example
If a daylight error rises while night performance improves, decide whether the trade-off meets the real policy. An update is a measurement-backed change to a running system, not just a new weight file. Run the current and proposed versions on the same held-out scenes, including night, rain, and empty frames. Compare error categories and latency, not only the average count. Keep the old version available for rollback if the new one harms a critical condition. State who approves the change and how long the trial lasts. A version update is an operational decision supported by evidence, not just a new model file. Now check the idea on a concrete example: Run old and new versions on the same held-out conditions before switching. A possible trap is worth naming: Do not use live users as the only test for a high-impact change. This supports a careful conclusion: A safe update plan includes versioning, staged evaluation, rollback, and monitoring. A version that improves one garage may fail another with different lighting. Run both versions on shared held-out scenes, document errors that changed, and switch only with a rollback plan. A single average can conceal the losing site. Keep the exact old model and threshold available, and name the condition that would send the service back to them. Test the night scenes before switching every camera.
Limits and next step
Now we'll assemble a final parking-space prototype and define exactly what each decision means.
Worked example
Run old and new versions on the same held-out conditions before switching.
Quick check: test your understanding
Try it yourself: Consider this case: Run old and new versions on the same held-out conditions before switching. Explain how you would check the result. In your answer, address this limitation: Do not use live users as the only test for a high-impact change.
Check the reasoning
A safe update plan includes versioning, staged evaluation, rollback, and monitoring. Do not use live users as the only test for a high-impact change.
What is the main idea behind plan safe updates to the vision system?
A safe update plan includes versioning, staged evaluation, rollback, and monitoring.
How is it used in the example?
Run old and new versions on the same held-out conditions before switching.
What limitation should be checked?
Do not use live users as the only test for a high-impact change.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 76 transcript302 words · Narration ready for the upcoming videoOpen transcript
A new model improves night occupancy estimates in one garage. Should we replace every camera's model immediately? First, compare it on all known conditions and record what changed.
Version the model, data split, bay geometry, thresholds, and annotation policy together. Test on held-out cameras and difficult cases, then release gradually while comparing predictions with reviewed outcomes. Keep a rollback path to the last working version.
If a daylight error rises while night performance improves, decide whether the trade-off meets the real policy. An update is a measurement-backed change to a running system, not just a new weight file. Run the current and proposed versions on the same held-out scenes, including night, rain, and empty frames. Compare error categories and latency, not only the average count. Keep the old version available for rollback if the new one harms a critical condition. State who approves the change and how long the trial lasts. A version update is an operational decision supported by evidence, not just a new model file. Now check the idea on a concrete example: Run old and new versions on the same held-out conditions before switching. A possible trap is worth naming: Do not use live users as the only test for a high-impact change. This supports a careful conclusion: A safe update plan includes versioning, staged evaluation, rollback, and monitoring. A version that improves one garage may fail another with different lighting. Run both versions on shared held-out scenes, document errors that changed, and switch only with a rollback plan. A single average can conceal the losing site. Keep the exact old model and threshold available, and name the condition that would send the service back to them. Test the night scenes before switching every camera.
Now we'll assemble a final parking-space prototype and define exactly what each decision means.