You currently set: model.train() # in validation That keeps dropout + batchnorm updating. Validation should be: model.eval() Otherwise val loss can behave misleadingly.