Resize and Crop Without Breaking Annotations
A parcel's box must follow the image when the image is resized or cropped.
Resize and Crop Without Breaking Annotations explained visually
A parcel's box must follow the image when the image is resized or cropped. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
A parcel's box must follow the image when the image is resized or cropped.
Pixels and labels travel together
Look at a red parcel on our conveyor. The inspection image has a box around it. Now shrink the image but leave the box coordinates alone. The box lands in the wrong place even though every number still looks valid. That's why an image transformation and its annotations have to travel together. The same rule applies to masks, keypoints, and anything else tied to pixel positions.
Scale the coordinates
For a simple resize, multiply horizontal positions by the new width divided by the old width. Do the same vertically using the heights. If the two scale factors differ, the parcel stretches. Sometimes we preserve its shape by resizing uniformly and adding padding. When we do that, the labels need both the scale and the padding offset. Recording the final dimensions alone isn't enough to reconstruct where the box went.
Handle cropping explicitly
A crop changes the origin of our coordinates. Subtract the crop's left edge from horizontal positions and its top edge from vertical positions. Then clip a box to the visible area. Decide how to label a partly visible parcel; remove a visible box when its object is completely outside. For a class mask, use an interpolation choice that preserves class IDs, commonly nearest neighbor. Overlay the results and actually look at them: valid coordinates can still mark the wrong thing.
A worked example
Let's check the arithmetic. In an eight-hundred-by-four-hundred image, a box begins at left one hundred, top fifty, and measures two hundred by one hundred. Resize the image to four hundred by two hundred. Every coordinate and size is halved, so the new box begins at fifty, twenty-five, and measures one hundred by fifty. Next we'll inspect how image brightness can change what we see without recovering lost detail.
Worked example
Suppose a box has left 100, top 50, width 200, and height 100 in an 800-by-400 image. Resizing to 400 by 200 multiplies every coordinate and size by one half. The new box is left 50, top 25, width 100, and height 50.
Quick check: test your understanding
Try it yourself: A crop begins at x equals 40 and y equals 20. Where does an original point at 70, 50 move?
Check the reasoning
It moves to 30, 30 before any later resize or padding.
Why avoid ordinary smooth interpolation for a class-ID mask?
It can create intermediate values that are not valid class IDs.
How would you solve the practice case?
It moves to 30, 30 before any later resize or padding.
What does the worked example show?
Suppose a box has left 100, top 50, width 200, and height 100 in an 800-by-400 image. Resizing to 400 by 200 multiplies every coordinate and size by one half. The new box is left 50, top 25, width 100, and height 50.
Further reading
Digital Image Processing, 4th edition — Rafael C. Gonzalez and Richard E. Woods.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 4 transcript284 words · Video transcript · Matches the published narration and captionsOpen transcript
Look at a red parcel on our conveyor. The inspection image has a box around it. Now shrink the image but leave the box coordinates alone. The box lands in the wrong place even though every number still looks valid. That's why an image transformation and its annotations have to travel together. The same rule applies to masks, keypoints, and anything else tied to pixel positions.
For a simple resize, multiply horizontal positions by the new width divided by the old width. Do the same vertically using the heights. If the two scale factors differ, the parcel stretches. Sometimes we preserve its shape by resizing uniformly and adding padding. When we do that, the labels need both the scale and the padding offset. Recording the final dimensions alone isn't enough to reconstruct where the box went.
A crop changes the origin of our coordinates. Subtract the crop's left edge from horizontal positions and its top edge from vertical positions. Then clip a box to the visible area. Decide how to label a partly visible parcel; remove a visible box when its object is completely outside. For a class mask, use an interpolation choice that preserves class IDs, commonly nearest neighbor. Overlay the results and actually look at them: valid coordinates can still mark the wrong thing.
Let's check the arithmetic. In an eight-hundred-by-four-hundred image, a box begins at left one hundred, top fifty, and measures two hundred by one hundred. Resize the image to four hundred by two hundred. Every coordinate and size is halved, so the new box begins at fifty, twenty-five, and measures one hundred by fifty. Next we'll inspect how image brightness can change what we see without recovering lost detail.