An inference release depends on both its saved model and the code that loads it. Replacing TorchScript with torch.export in PyTorch therefore requires a new artifact and Python loader, with output comparisons before the old release can be retired.
torch.export captures the original nn.Module with its trained weights; it does not convert a TorchScript archive into an equivalent .pt2 file. The original model definition, matching weights, preprocessing, and representative inputs must still be available. Tensor-dependent Python branches may need changes for export, even when TorchScript accepted them.
The replacement here runs through Python and PyTorch's exported-program runtime. A TorchScript C++ loader cannot consume the new file directly, and saved export compatibility is not guaranteed across PyTorch versions. Producer and consumer need compatible versions and their own inference checks before a production switch.
Steps to migrate a PyTorch model from TorchScript to torch.export:
- Retain the working TorchScript release and its Python environment for rollback.
The retained release includes the .pt artifact, loader, source, weights, dependencies, and preprocessing.
Loading model artifacts can execute malicious code. Only trusted model source, weights, TorchScript files, and exported programs belong in this process. The original artifact and loader remain unchanged, so recovery means returning to the retained release.
- Activate a separate Python environment that has PyTorch 2.14 or later and the application's other dependencies.
A new environment from How to install PyTorch with pip holds only the PyTorch packages, so install the application's own packages there too. PyTorch 2.13 raises its TorchScript deprecation warnings as DeprecationWarning, which the FutureWarning checks in later steps miss. Tested with PyTorch 2.15.0 on CPU.
Reference: PyTorch 2.14 release notes - Confirm that the project directory holds model.py, score_model_weights.pth and score_model_torchscript.pt, or your own model source, state dictionary and TorchScript file in their place.
ScoreModel in model.py takes a CPU float32 tensor with four features per row and returns two class probabilities per row. Each later step names the values to replace for a different model.
Related: How to save and load a PyTorch modelA project built with How to export a PyTorch model to TorchScript keeps ScoreModel inside its script and saves no state dictionary. In that project, copy the class and its torch and nn imports into model.py, then save the archive's weights with this command, which also prints an expected torch.jit.load warning.
$ python -c "import torch; \ archive = torch.jit.load('score_model_torchscript.pt'); \ torch.save(archive.state_dict(), 'score_model_weights.pth')" - Create migrate_export.py in the model project using your text editor, replacing ScoreModel, its model module and the weights file with your own.
- migrate_export.py
import torch from torch.export import Dim from model import ScoreModel model = ScoreModel() state = torch.load( "score_model_weights.pth", map_location="cpu", weights_only=True, ) model.load_state_dict(state) model.eval()
The original class and trained weights must describe the same model as the retained TorchScript artifact. Evaluation mode fixes dropout and batch-normalization behavior before capture.
- Append the export code at the end of migrate_export.py, replacing the example input, the features argument name and the batch limits with your model's values.
- migrate_export.py
example_inputs = (torch.randn(2, 4),) batch = Dim("batch", min=1, max=8) exported = torch.export.export( model, example_inputs, dynamic_shapes={"features": {0: batch}}, ) torch.export.save(exported, "score_model.pt2") print(f"PyTorch: {torch.__version__}") print(f"saved: score_model.pt2 ({type(exported).__name__})")
The features key matches the argument name in the original model's forward method. Only its batch dimension varies; the feature count remains four. The declared batch range is one through eight. A constraint or capture error means the model or its declared input contract needs attention before the loader can change. There is no automatic TorchScript fallback.
Reference: PyTorch export documentation - Save migrate_export.py in your text editor.
- Run the exporter with future warnings treated as errors.
$ python -W error::FutureWarning migrate_export.py PyTorch: 2.15.0+cpu saved: score_model.pt2 (ExportedProgram)
This export path does not call the deprecated TorchScript exporter. A warning promoted to an exception must be investigated before proceeding.
- Create the replacement loader in run_export.py using your text editor, replacing torch.ones(3, 4) with an input your model accepts.
- run_export.py
import torch def load_model(): return torch.export.load("score_model.pt2").module() if __name__ == "__main__": model = load_model() with torch.inference_mode(): output = model(torch.ones(3, 4)) print(f"output shape: {tuple(output.shape)}") print(f"sums: {output.sum(dim=-1)}")
The loaded ExportedProgram supplies a callable module through .module(). This loader needs the .pt2 file and PyTorch, without importing the original Python model class. Evaluation behavior was captured during export, and the loaded module raises NotImplementedError on eval(), so remove any model.eval() call kept from the old loader.
- Save run_export.py in your text editor.
- Create compare_migration.py using your text editor, replacing the .pt file name with your retained archive, the sample rows with at least as many preprocessed application rows as your upper batch limit, including the application's largest and smallest values, and the cases sizes with your own batch limits and example batch size.
- compare_migration.py
import warnings import torch from run_export import load_model legacy = torch.jit.load( "score_model_torchscript.pt", map_location="cpu" ) legacy.eval() rows = torch.tensor( [ [0.4, 1.2, -0.3, 0.8], [12.5, 3.1, 7.7, 0.2], [0.0, 0.0, 0.0, 0.0], [5.6, 48.0, 2.4, 19.3], [200.0, 10.0, 30.0, 120.0], [1.1, 0.6, 9.8, 4.5], [33.0, 2.7, 0.9, 61.2], [7.2, 15.4, 88.0, 3.3], ] ) cases = [rows[:size] for size in (1, 2, 5, 8)] rtol, atol = 1e-5, 1e-6
Real application rows matter because a value-dependent branch can match on random noise and fail on real data. They need the application's preprocessing, device, and dtype. The cases sizes cover both batch limits, the example batch size, and another supported size. The tolerances shown suit this float32 classifier; application accuracy requirements determine acceptable tolerances.
- Append reload, warning capture, and output comparison at the end of compare_migration.py.
- compare_migration.py
with warnings.catch_warnings(record=True) as captured: warnings.simplefilter("always") replacement = load_model() with torch.inference_mode(): new_outputs = [replacement(batch) for batch in cases] messages = [str(item.message).lower() for item in captured] deprecations = [ text for text in messages if "torchscript" in text or "torch.jit" in text ] if deprecations: raise RuntimeError("\n".join(deprecations)) print(f"TorchScript warnings: {len(deprecations)}") with torch.inference_mode(): for features, actual in zip(cases, new_outputs): expected = legacy(features) torch.testing.assert_close( actual, expected, rtol=rtol, atol=atol ) print(f"input {tuple(features.shape)}: outputs match") print(f"rtol={rtol}, atol={atol}")
The retained TorchScript loader emits its own warning before the comparison, which is expected. It loads before warning capture, so only the replacement loading and inference path contributes to the TorchScript warnings count. An assertion failure stops the comparison instead of printing a passing result.
- Save compare_migration.py in your text editor.
- Compare the reloaded export with the retained TorchScript model.
$ python compare_migration.py ##### snipped ##### TorchScript warnings: 0 input (1, 4): outputs match input (2, 4): outputs match input (5, 4): outputs match input (8, 4): outputs match rtol=1e-05, atol=1e-06
A TorchScript warnings count of 0 and outputs match on every input line mean the replacement passed. A mismatch means the replacement is not ready to replace the retained release, while a Guard failed error means a size in cases exceeds the exported batch limit or a row has the wrong feature count. Common boundaries to inspect are the checkpoint, evaluation mode, preprocessing, and shape constraints. Warning-free loading alone does not establish output parity.
- Run the replacement Python loader in a fresh process to verify inference from the saved export.
$ python -W error::FutureWarning run_export.py output shape: (3, 2) sums: tensor([1., 1., 1.])
The saved .pt2 file now supplies inference through the replacement loader at a batch size different from capture, and each row of ScoreModel probabilities sums to 1. A larger batch or a different feature count requires a new export with suitable constraints. The retained TorchScript release remains available if the application's broader compatibility checks fail.