Seq2seq Transformer Tutorial best model saving

[micahcarroll]

In this tutorial, it says "Save the model if the validation loss is the best we’ve seen so far.", and then the following code follows (also here).

if val_loss < best_val_loss:
        best_val_loss = val_loss
        best_model = model

However, my understanding is that this kind of checkpointing won't work, as best_model will contain a pointer to the same set of parameters as model (which will be updated). I tried to verify this by checking that next(model.parameters()) and next(best_model.parameters()) are identical, and it seemed like that was the case (although admittedly I did not check that the last model was indeed not the best one).

cc @pytorch/team-text-core @Nayef211

[Nayef211]

This should have been fixed by #2181