How to run evaluation in PyTorch

Model evaluation is the point where a trained PyTorch model is measured on held-out data instead of optimized on the training set. Evaluation mode matters whenever a model contains layers such as Dropout or BatchNorm, because those layers behave differently while training.

model.eval() changes the module's training flag and switches supported submodules into their evaluation behavior. It does not disable autograd, so the validation loop also needs torch.inference_mode() or torch.no_grad() around forward passes that do not feed a later backward pass.

A small TensorDataset and DataLoader are enough to prove the evaluation loop shape before it moves into a larger project. Keep the model and validation batches on the same device in real projects, and switch back to model.train() before resuming optimizer updates.

Steps to run PyTorch model evaluation:

Create a minimal evaluation smoke script.

evaluation_run_demo.py

import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset
 
 
features = torch.tensor(
    [
        [0.0, 0.1],
        [0.2, 0.0],
        [1.0, 1.1],
        [1.2, 0.9],
    ],
    dtype=torch.float32,
)
labels = torch.tensor([0, 0, 1, 1])
validation_data = TensorDataset(features, labels)
validation_loader = DataLoader(validation_data, batch_size=2)
 
model = nn.Sequential(
    nn.Linear(2, 2),
    nn.Dropout(p=0.75),
)
 
with torch.no_grad():
    model[0].weight.copy_(torch.tensor([[1.0, -1.0], [1.0, 1.0]]))
    model[0].bias.copy_(torch.tensor([0.25, -1.0]))
 
loss_fn = nn.CrossEntropyLoss(reduction="sum")
 
model.eval()
loss_total = 0.0
correct = 0
sample_count = 0
gradient_flags = []
 
with torch.inference_mode():
    for batch_features, batch_labels in validation_loader:
        logits = model(batch_features)
        loss = loss_fn(logits, batch_labels)
 
        loss_total += loss.item()
        predictions = logits.argmax(dim=1)
        correct += (predictions == batch_labels).sum().item()
        sample_count += batch_labels.size(0)
        gradient_flags.append(logits.requires_grad)
 
average_loss = loss_total / sample_count
accuracy = correct / sample_count
 
print(f"model training mode: {model.training}")
print(f"dropout training mode: {model[1].training}")
print(f"logits require grad: {any(gradient_flags)}")
print(f"validation loss: {average_loss:.4f}")
print(f"validation accuracy: {correct}/{sample_count} ({accuracy:.0%})")

model.eval() sets the module and its children to evaluation mode. torch.inference_mode() disables autograd tracking because this validation loop does not call backward().

Run the evaluation smoke script.
```
$ python evaluation_run_demo.py
model training mode: False
dropout training mode: False
logits require grad: False
validation loss: 0.3336
validation accuracy: 4/4 (100%)
```
model training mode: False and dropout training mode: False confirm evaluation mode. logits require grad: False confirms the forward passes ran without autograd tracking.
Check the accumulated metric lines.

validation loss: 0.3336 is the average loss across all four validation samples. validation accuracy: 4/4 (100%) confirms the prediction count and sample count were accumulated across both batches.

Move the same pattern into the project validation function.

def evaluate(model, validation_loader, loss_fn, device):
    model.eval()
    loss_total = 0.0
    correct = 0
    sample_count = 0
 
    with torch.inference_mode():
        for inputs, targets in validation_loader:
            inputs = inputs.to(device)
            targets = targets.to(device)
            logits = model(inputs)
            loss = loss_fn(logits, targets)
 
            loss_total += loss.item() * targets.size(0)
            predictions = logits.argmax(dim=1)
            correct += (predictions == targets).sum().item()
            sample_count += targets.size(0)
 
    return {
        "loss": loss_total / sample_count,
        "accuracy": correct / sample_count,
        "correct": correct,
        "total": sample_count,
    }

Multiplying loss.item() by targets.size(0) keeps the epoch average correct when the loss function returns a batch mean.

Call the evaluation function between training epochs.

validation_metrics = evaluate(model, validation_loader, loss_fn, device)
print(
    f"val_loss={validation_metrics['loss']:.4f} "
    f"val_accuracy={validation_metrics['accuracy']:.0%}"
)
model.train()

Call model.train() after evaluation when another training pass follows. model.eval() is sticky until the mode changes again.

Remove the temporary smoke script.
```
$ rm evaluation_run_demo.py
```