A PyTorch inference script produces different outputs on every run.
The model weights are loaded from the same file and the input tensor never changes.
This only happens after moving from training to deployment.
There are no errors or warnings.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
A PyTorch inference script produces different outputs on every run.
The model weights are loaded from the same file and the input tensor never changes.
This only happens after moving from training to deployment.
There are no errors or warnings.
This happens because your model is still running in training mode, which keeps randomness active inside layers like dropout and batch normalization.
PyTorch layers behave differently depending on whether the model is in training or evaluation mode. If
model.eval()is not called before inference, dropout will randomly disable neurons and batch normalization will update running statistics, which makes predictions change on every run even with identical input.The fix is simply to switch the model to evaluation mode before inference:
model.eval()
with torch.no_grad():
output = model(input_tensor)
torch.no_grad()is important because it prevents PyTorch from tracking gradients, which also reduces memory usage and avoids subtle state changes during inference.