How to run evaluation in PyTorch

Model evaluation is the point where a trained PyTorch model is measured on held-out data instead of optimized on the training set. Evaluation mode matters whenever a model contains layers such as Dropout or BatchNorm, because those layers behave differently while training.

model.eval() changes the module's training flag and switches supported submodules into their evaluation behavior. It does not disable autograd, so the validation loop also needs torch.inference_mode() or torch.no_grad() around forward passes that do not feed a later backward pass.

A small TensorDataset and DataLoader are enough to prove the evaluation loop shape before it moves into a larger project. Keep the model and validation batches on the same device in real projects, and switch back to model.train() before resuming optimizer updates.

1. Create a minimal evaluation smoke script.

   evaluation_run_demo.py

   import torch
   from torch import nn
   from torch.utils.data import DataLoader, TensorDataset
    
    
   features = torch.tensor(
       [
           [0.0, 0.1],
           [0.2, 0.0],
           [1.0, 1.1],
           [1.2, 0.9],
       ],
       dtype=torch.float32,
   )
   labels = torch.tensor([0, 0, 1, 1])
   validation_data = TensorDataset(features, labels)
   validation_loader = DataLoader(validation_data, batch_size=2)
    
   model = nn.Sequential(
       nn.Linear(2, 2),
       nn.Dropout(p=0.75),
   )
    
   with torch.no_grad():
       model[0].weight.copy_(torch.tensor([[1.0, -1.0], [1.0, 1.0]]))
       model[0].bias.copy_(torch.tensor([0.25, -1.0]))
    
   loss_fn = nn.CrossEntropyLoss(reduction="sum")
    
   model.eval()
   loss_total = 0.0
   correct = 0
   sample_count = 0
   gradient_flags = []
    
   with torch.inference_mode():
       for batch_features, batch_labels in validation_loader:
           logits = model(batch_features)
           loss = loss_fn(logits, batch_labels)
    
           loss_total += loss.item()
           predictions = logits.argmax(dim=1)
           correct += (predictions == batch_labels).sum().item()
           sample_count += batch_labels.size(0)
           gradient_flags.append(logits.requires_grad)
    
   average_loss = loss_total / sample_count
   accuracy = correct / sample_count
    
   print(f"model training mode: {model.training}")
   print(f"dropout training mode: {model[1].training}")
   print(f"logits require grad: {any(gradient_flags)}")
   print(f"validation loss: {average_loss:.4f}")
   print(f"validation accuracy: {correct}/{sample_count} ({accuracy:.0%})")

   model.eval() sets the module and its children to evaluation mode. torch.inference_mode() disables autograd tracking because this validation loop does not call backward().

2. Run the evaluation smoke script.

   $ python evaluation_run_demo.py
   model training mode: False
   dropout training mode: False
   logits require grad: False
   validation loss: 0.3336
   validation accuracy: 4/4 (100%)

   model training mode: False and dropout training mode: False confirm evaluation mode. logits require grad: False confirms the forward passes ran without autograd tracking.

3. Check the accumulated metric lines.

   validation loss: 0.3336 is the average loss across all four validation samples. validation accuracy: 4/4 (100%) confirms the prediction count and sample count were accumulated across both batches.

4. Move the same pattern into the project validation function.

   def evaluate(model, validation_loader, loss_fn, device):
       model.eval()
       loss_total = 0.0
       correct = 0
       sample_count = 0
    
       with torch.inference_mode():
           for inputs, targets in validation_loader:
               inputs = inputs.to(device)
               targets = targets.to(device)
               logits = model(inputs)
               loss = loss_fn(logits, targets)
    
               loss_total += loss.item() * targets.size(0)
               predictions = logits.argmax(dim=1)
               correct += (predictions == targets).sum().item()
               sample_count += targets.size(0)
    
       return {
           "loss": loss_total / sample_count,
           "accuracy": correct / sample_count,
           "correct": correct,
           "total": sample_count,
       }

   Multiplying loss.item() by targets.size(0) keeps the epoch average correct when the loss function returns a batch mean.

5. Call the evaluation function between training epochs.

   validation_metrics = evaluate(model, validation_loader, loss_fn, device)
   print(
       f"val_loss={validation_metrics['loss']:.4f} "
       f"val_accuracy={validation_metrics['accuracy']:.0%}"
   )
   model.train()

   Call model.train() after evaluation when another training pass follows. model.eval() is sticky until the mode changes again.

6. Remove the temporary smoke script.

   $ rm evaluation_run_demo.py