Split by Camera, Time, and Scene
Frames from the same camera appear in both training and test sets.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
Frames from the same camera appear in both training and test sets.
A practical question
A camera records a parking bay every second. If neighboring frames land in both training and test sets, the test may look easy because it has nearly seen the same car already.
The useful idea
Split by camera, location, time period, or continuous video group according to the claim we want to make. If deployment will use a new pole, reserve a camera. If it will face winter lighting, reserve a later period. Keep every frame from one near-duplicate sequence together.
Work through the example
Compare random-frame accuracy with performance on a held-out camera. A drop is useful evidence about generalization, not a reason to hide the harder split. Record the split definition so it can be reproduced. Frames from one camera on adjacent seconds can be nearly identical. Putting such frames in both training and evaluation makes the test easier than a future week or new camera. Choose the split according to the deployment question and keep related scenes together. State what is held out and why. If performance falls on a new camera, that is useful evidence about generalization, not a reason to quietly mix its examples into the training split. Now check the idea on a concrete example: Hold out a later week or a new camera when that is the deployment question. A possible trap is worth naming: A random frame split can leak location-specific cues across sets. This supports a careful conclusion: Splitting by camera, time, and scene prevents near-duplicate context from making the test too easy. A later week tests whether the occupancy rule works after lighting and parking patterns change. Keep adjacent frames together so the evaluation is not filled with near copies of training images. State the held-out period clearly.
Limits and next step
Next, we'll inspect errors by condition and subgroup rather than trusting one overall accuracy number.
Worked example
Hold out a later week or a new camera when that is the deployment question.
Quick check: test your understanding
Try it yourself: Consider this case: Hold out a later week or a new camera when that is the deployment question. Explain how you would check the result. In your answer, address this limitation: A random frame split can leak location-specific cues across sets.
Check the reasoning
Splitting by camera, time, and scene prevents near-duplicate context from making the test too easy. A random frame split can leak location-specific cues across sets.
What is the main idea behind split by camera, time, and scene?
Splitting by camera, time, and scene prevents near-duplicate context from making the test too easy.
How is it used in the example?
Hold out a later week or a new camera when that is the deployment question.
What limitation should be checked?
A random frame split can leak location-specific cues across sets.
Further reading
Programming Computer Vision with Python — Jan Erik Solem.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 70 transcript296 words · Narration ready for the upcoming videoOpen transcript
A camera records a parking bay every second. If neighboring frames land in both training and test sets, the test may look easy because it has nearly seen the same car already.
Split by camera, location, time period, or continuous video group according to the claim we want to make. If deployment will use a new pole, reserve a camera. If it will face winter lighting, reserve a later period. Keep every frame from one near-duplicate sequence together.
Compare random-frame accuracy with performance on a held-out camera. A drop is useful evidence about generalization, not a reason to hide the harder split. Record the split definition so it can be reproduced. Frames from one camera on adjacent seconds can be nearly identical. Putting such frames in both training and evaluation makes the test easier than a future week or new camera. Choose the split according to the deployment question and keep related scenes together. State what is held out and why. If performance falls on a new camera, that is useful evidence about generalization, not a reason to quietly mix its examples into the training split. Now check the idea on a concrete example: Hold out a later week or a new camera when that is the deployment question. A possible trap is worth naming: A random frame split can leak location-specific cues across sets. This supports a careful conclusion: Splitting by camera, time, and scene prevents near-duplicate context from making the test too easy. A later week tests whether the occupancy rule works after lighting and parking patterns change. Keep adjacent frames together so the evaluation is not filled with near copies of training images. State the held-out period clearly.
Next, we'll inspect errors by condition and subgroup rather than trusting one overall accuracy number.