Separate Classes with a Margin
Late and on-time trips overlap in feature space.
Read the full lesson and open its transcript below. A narrated video will be added after production.
What you will learn
Late and on-time trips overlap in feature space.
A practical question
Plot past deliveries using two informative features. Some late and on-time trips fall on opposite sides of a possible line. Many lines could separate the easy examples. A margin-based classifier favors a boundary with room around it.
The useful idea
The examples nearest that boundary help determine where it sits; these are support vectors in a support vector machine. Compare two candidate lines that make the same training predictions. One passes close to several points; the other has more clearance. More clearance can make the decision less sensitive to small changes, under the chosen feature geometry.
Work through the example
That geometry depends on scaling. If one feature is measured in hundreds and another between zero and one, the apparent margin changes. Plot the standardized features and inspect the border cases. Draw two candidate separating lines between on-time and late deliveries in a small feature plot. One line may classify every training point while passing close to them; another leaves more clearance. A margin method favors useful separation, but the picture changes when features are scaled or the data contain outliers. Examine the points nearest the boundary and test later trips before calling the wider margin safer. The separation is a model rule, not a natural law of traffic. Now check the idea on a concrete example: Compare two candidate lines and prefer the one with useful clearance on training points. A possible trap is worth naming: A wide training margin alone does not prove performance on new trips. The answer follows from that evidence: A margin-based classifier looks for a separating boundary with room around it. Draw the deliveries closest to the separating line, then change one input slightly. A warning that flips around the boundary may need review even if the classifier has a wide training margin elsewhere. Report behavior on later routes, not just the geometry of the training plot.
Limits and next step
Real deliveries often overlap. Next, we'll allow some points on the wrong side rather than demanding a perfect dividing line.
Worked example
Compare two candidate lines and prefer the one with useful clearance on training points.
Quick check: test your understanding
Try it yourself: Consider this case: Compare two candidate lines and prefer the one with useful clearance on training points. Explain how you would check the result. In your answer, address this limitation: A wide training margin alone does not prove performance on new trips.
Check the reasoning
A margin-based classifier looks for a separating boundary with room around it. A wide training margin alone does not prove performance on new trips.
What is the main idea behind separate classes with a margin?
A margin-based classifier looks for a separating boundary with room around it.
How is it used in the example?
Compare two candidate lines and prefer the one with useful clearance on training points.
What limitation should be checked?
A wide training margin alone does not prove performance on new trips.
Further reading
Machine Learning — Tom M. Mitchell.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 49 transcript326 words · Narration ready for the upcoming videoOpen transcript
Plot past deliveries using two informative features. Some late and on-time trips fall on opposite sides of a possible line. Many lines could separate the easy examples. A margin-based classifier favors a boundary with room around it.
The examples nearest that boundary help determine where it sits; these are support vectors in a support vector machine. Compare two candidate lines that make the same training predictions. One passes close to several points; the other has more clearance. More clearance can make the decision less sensitive to small changes, under the chosen feature geometry.
That geometry depends on scaling. If one feature is measured in hundreds and another between zero and one, the apparent margin changes. Plot the standardized features and inspect the border cases. Draw two candidate separating lines between on-time and late deliveries in a small feature plot. One line may classify every training point while passing close to them; another leaves more clearance. A margin method favors useful separation, but the picture changes when features are scaled or the data contain outliers. Examine the points nearest the boundary and test later trips before calling the wider margin safer. The separation is a model rule, not a natural law of traffic. Now check the idea on a concrete example: Compare two candidate lines and prefer the one with useful clearance on training points. A possible trap is worth naming: A wide training margin alone does not prove performance on new trips. The answer follows from that evidence: A margin-based classifier looks for a separating boundary with room around it. Draw the deliveries closest to the separating line, then change one input slightly. A warning that flips around the boundary may need review even if the classifier has a wide training margin elsewhere. Report behavior on later routes, not just the geometry of the training plot.
Real deliveries often overlap. Next, we'll allow some points on the wrong side rather than demanding a perfect dividing line.