The model is becoming more certain about wrong predictions, often due to overfitting or distribution shift. This is especially common after retraining or fine-tuning on narrow datasets. Measure calibration metrics like expected calibration error (ECE) and inspect confidence histograms. Techniques such as temperature scaling or label smoothing can restore better alignment between confidence and correctness.

Common mistakes:

1. Equating confidence with correctness

2. Monitoring accuracy without calibration

3. Deploying fine-tuned models without recalibration

A trustworthy model knows when it might be wrong.