Cross-Validation with Groups and Time
Several records share a driver and repeated route conditions.
Cross-Validation with Groups and Time explained visually
Several records share a driver and repeated route conditions. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
Several records share a driver and repeated route conditions.
A practical question
One driver runs the same route every weekday. If Monday's trip is in training and Tuesday's nearly identical trip is in validation, our score may exaggerate how well the model handles a new driver.
The useful idea
Keep related trips together when the deployment claim is about new drivers or routes. For forecasts into the future, train on earlier periods and validate on later ones. Fit scaling and imputation only within each training fold; otherwise the held-out records leak into preprocessing even when their labels stay hidden.
Work through the example
Compare a random split with a driver-grouped or chronological split. Ask which one resembles the actual prediction setting. Lower but honest performance is useful for planning.
Limits and next step
Document the split and every fitted preprocessing step. Next, we'll see why a score measured on a limited sample needs uncertainty attached.
Worked example
Assign each driver or future period to a held-out fold without leaking duplicates.
Quick check: test your understanding
Try it yourself: In the situation described above, explain the method and one reason its output might be misleading.
Check the reasoning
Cross-validation must keep related groups together; time-based tasks should preserve chronology. Random folds can overstate performance on genuinely new drivers or future days.
What is the main idea behind cross-validation with groups and time?
Cross-validation must keep related groups together; time-based tasks should preserve chronology.
How is it used in the example?
Assign each driver or future period to a held-out fold without leaking duplicates.
What limitation should be checked?
Random folds can overstate performance on genuinely new drivers or future days.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 34 transcript132 words · Video transcript · Matches the published narration and captionsOpen transcript
One driver runs the same route every weekday. If Monday's trip is in training and Tuesday's nearly identical trip is in validation, our score may exaggerate how well the model handles a new driver.
Keep related trips together when the deployment claim is about new drivers or routes. For forecasts into the future, train on earlier periods and validate on later ones. Fit scaling and imputation only within each training fold; otherwise the held-out records leak into preprocessing even when their labels stay hidden.
Compare a random split with a driver-grouped or chronological split. Ask which one resembles the actual prediction setting. Lower but honest performance is useful for planning.
Document the split and every fitted preprocessing step. Next, we'll see why a score measured on a limited sample needs uncertainty attached.