Monitoring, Corrections, and Retraining
A deployed model does not keep learning merely because people use it.
Monitoring, Corrections, and Retraining explained visually
A deployed model does not keep learning merely because people use it. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
A deployed model does not keep learning merely because people use it.
Watch behavior after release
A deployed model does not keep learning merely because people use it. New examples and corrected annotations become training information only when a training process explicitly uses them. This distinction matters in an annotation tool: saving more reviewed images improves the available dataset, but it does not silently change the active model.
Collect reviewed corrections
Monitor the kinds of errors users correct, the conditions where they occur, and changes in incoming data. More examples are valuable when they add trustworthy labels and useful variation. Hundreds of nearly identical frames may contribute less than a smaller set covering new viewpoints, lighting, and difficult cases. Preserve evaluation examples that remain independent of training, and review labeling consistency before retraining. Data drift means a change in the distribution of incoming inputs; it is an investigation signal, not a direct measurement of accuracy.
Compare before replacing
Track which dataset version each model used. During retraining, retain the previous successful model and expose the new run's real status. Compare the candidate with the current version on representative held-out data before activation. If it fails validation or training, keep the earlier model. This creates a controlled improvement cycle in which changes can be traced, measured, and reversed when necessary.
Worked example
Suppose the active model used 100 reviewed images and the project now contains 160. That means 60 additional images are available for consideration; it does not mean the model has already learned from them. A new training run must prepare the dataset, train, validate, and activate a successful result.
Quick check: test your understanding
Try it yourself: Why should retraining data include corrected difficult cases rather than only easy successful frames?
Check the reasoning
Difficult cases reveal gaps in the current model and can broaden coverage. They still need reliable labels and an independent evaluation design.
Does a larger training set guarantee that a retrained model will be better?
No. Label quality, data relevance, training settings, and evaluation all affect the result.
Further reading
Dive into Deep Learning — Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola.
Deep Learning — Ian Goodfellow, Yoshua Bengio, and Aaron Courville.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 81 transcript234 words · Video transcript · Matches the published narration and captionsOpen transcript
You have corrected sixty more images. Has the model already learned from those corrections? Unless a training process used them, the answer is no.
Saving annotations improves the available dataset. It does not automatically change the active model. Keeping those two ideas separate makes retraining much easier to understand.
Suppose the current model used one hundred reviewed images, and the project now contains one hundred and sixty. There are sixty additional images to consider. A new run still needs to prepare the data, train, validate, and activate a successful result.
Before retraining, look at what the new examples add. Do they include different lighting, viewpoints, objects, or difficult situations? Many nearly identical frames may add less useful information than a smaller, varied collection with reliable labels.
Also watch for data drift: changes in the kinds of inputs arriving after deployment. A new camera or darker environment may change that distribution. Drift is a reason to investigate; it does not, by itself, tell us the model's accuracy.
Review labeling consistency and keep evaluation examples independent of training. Record which dataset and model versions belong together. During retraining, the previous successful model can remain available.
Then compare the candidate against the current version using representative held-out data. If the new run fails or performs worse, keep the earlier model. Improvement should be measured and deliberate, with a clear path from user corrections to a validated update.