How to run evaluation in PyTorch

A validation split is useful only when its scores are computed without changing trained parameters or using training-time layer behavior. PyTorch evaluation puts a model in the state used to compare checkpoints, detect overfitting, and decide whether another epoch improves performance on held-out data.

The model.eval() method switches modules such as Dropout and BatchNorm to their evaluation behavior. It does not disable automatic differentiation, so forward passes that will not feed backward() also belong inside torch.inference_mode().

When a loss function returns a batch mean, the epoch loss must weight each batch by its actual sample count before dividing by the total. This matters when the last validation batch is smaller than the others; accuracy likewise uses total correct predictions divided by total samples.

1. Define the evaluation function with sample counters.

   def evaluate(model, data_loader, loss_fn, device):
       model.eval()
       loss_total = 0.0
       correct = 0
       sample_count = 0

   model.eval() changes the module and its children to evaluation behavior until a later model.train() call.

2. Add the inference-only batch loop below the counters.

       with torch.inference_mode():
           for inputs, targets in data_loader:
               inputs = inputs.to(device)
               targets = targets.to(device)
               logits = model(inputs)
               batch_size = targets.size(0)
    
               loss_total += loss_fn(logits, targets).item() * batch_size
               correct += (logits.argmax(dim=1) == targets).sum().item()
               sample_count += batch_size

   Multiplying by batch_size is correct when loss_fn uses the usual reduction="mean". A loss with reduction="sum" already includes every sample in its batch.

3. Return a sample-weighted metric dictionary below the loop.

       return {
           "loss": loss_total / sample_count,
           "accuracy": correct / sample_count,
           "correct": correct,
           "total": sample_count,
       }

   The validation DataLoader must yield at least one sample. The forward pass requires model parameters, inputs, and targets on the same device.

4. Call the evaluation function after each completed training epoch.

   validation_metrics = evaluate(model, validation_loader, loss_fn, device)
   print(
       f"val_loss={validation_metrics['loss']:.4f} "
       f"val_accuracy={validation_metrics['accuracy']:.0%}"
   )

   Related: How to run a training loop in PyTorch

5. Restore training mode before the next optimizer update.

   model.train()

6. Save the consolidated CPU smoke test as evaluation_run_demo.py.

   evaluation_run_demo.py

   import torch
   from torch import nn
   from torch.utils.data import DataLoader, TensorDataset
    
    
   def evaluate(model, data_loader, loss_fn, device):
       model.eval()
       loss_total = 0.0
       correct = 0
       sample_count = 0
    
       with torch.inference_mode():
           for inputs, targets in data_loader:
               inputs = inputs.to(device)
               targets = targets.to(device)
               logits = model(inputs)
               batch_size = targets.size(0)
    
               loss_total += loss_fn(logits, targets).item() * batch_size
               correct += (logits.argmax(dim=1) == targets).sum().item()
               sample_count += batch_size
    
       return {
           "loss": loss_total / sample_count,
           "accuracy": correct / sample_count,
           "correct": correct,
           "total": sample_count,
       }
    
    
   device = torch.device("cpu")
   features = torch.tensor(
       [
           [0.0, 0.1],
           [0.2, 0.0],
           [1.0, 1.1],
           [1.2, 0.9],
           [0.1, 0.2],
       ],
       dtype=torch.float32,
   )
   targets = torch.tensor([0, 0, 1, 1, 0])
   validation_loader = DataLoader(
       TensorDataset(features, targets),
       batch_size=2,
       shuffle=False,
   )
    
   model = nn.Sequential(
       nn.Linear(2, 2),
       nn.Dropout(p=0.75),
   ).to(device)
    
   with torch.no_grad():
       model[0].weight.copy_(torch.tensor([[1.0, -1.0], [1.0, 1.0]]))
       model[0].bias.copy_(torch.tensor([0.25, -1.0]))
    
   loss_fn = nn.CrossEntropyLoss()
   metrics = evaluate(model, validation_loader, loss_fn, device)
    
   with torch.inference_mode():
       full_logits = model(features.to(device))
       full_loss = loss_fn(full_logits, targets.to(device)).item()
       full_correct = (full_logits.argmax(dim=1) == targets.to(device)).sum().item()
    
   metrics_match = (
       abs(metrics["loss"] - full_loss) < 1e-7
       and metrics["correct"] == full_correct
   )
    
   print(f"model training mode: {model.training}")
   print(f"evaluated samples: {metrics['total']}")
   print(f"validation loss: {metrics['loss']:.4f}")
   print(
       f"validation accuracy: {metrics['correct']}/{metrics['total']} "
       f"({metrics['accuracy']:.0%})"
   )
   print(f"full-batch metrics match: {metrics_match}")

   The five samples produce batch sizes of two, two, and one. Comparing the accumulated result with a single full-batch calculation checks the weighting of the short final batch.

7. Run the smoke test to compare batched metrics with a full-batch calculation.

   $ python evaluation_run_demo.py
   model training mode: False
   evaluated samples: 5
   validation loss: 0.3381
   validation accuracy: 5/5 (100%)
   full-batch metrics match: True

   model training mode: False confirms evaluation behavior. full-batch metrics match: True confirms that the batched loss and accuracy match the same model evaluated over all five samples at once.