How to load a Sentence Transformers model offline

Embedding applications can run on servers that cannot reach the Hugging Face Hub. Sentence Transformers can load a model offline when its weights, tokenizer, configuration, and pooling modules are available together in a local directory.

A model name normally tells the library which repository to download. Saving the loaded model as a standalone directory keeps its components together for transfer, without depending on the connected machine's cache layout or copying only the weights.

Both machines need a Python environment with Sentence Transformers and its dependencies, preferably at the same versions. The model archive does not include Python packages, so an air-gapped server that lacks them needs their wheels moved with it. The shell syntax is for Linux.

1. Create save_model.py on the internet-connected machine in your text editor with the model load.

   save_model.py

   from sentence_transformers import SentenceTransformer
    
   model = SentenceTransformer(
       "sentence-transformers/paraphrase-MiniLM-L3-v2"
   )

   The public paraphrase-MiniLM-L3-v2 model is small enough to run on a CPU and requires no account.

2. Append the save call to save_model.py.

   model.save_pretrained("./offline-model")

   The saved directory includes the tokenizer, weights, configuration, and Sentence Transformers modules.
   Related: Model saving reference

3. Save save_model.py.
4. Run save_model.py to save the model in the offline-model directory.

   $ python3 save_model.py
   Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
   Loading weights: 100%|██████████| 55/55 [00:00<00:00, 7869.78it/s]
   Writing model shards: 100%|██████████| 1/1 [00:00<00:00, 16.65it/s]

   The unauthenticated-requests warning is harmless, because this public model needs no token.

5. Create check_offline_model.py in your text editor with the local model load.

   check_offline_model.py

   from sentence_transformers import SentenceTransformer
    
   model = SentenceTransformer("./offline-model")

   The local path selects the transferred model instead of a Hub repository. Writing the script on the connected machine lets it travel in the archive to a server that cannot reach this page.

6. Append the similarity check to check_offline_model.py.

   embeddings = model.encode(
       [
           "I forgot my login password.",
           "How do I reset my password?",
           "The train leaves at noon.",
       ]
   )
   print(model.similarity(embeddings[:1], embeddings[1:]))

   The script compares the first sentence with a related sentence and an unrelated one, so the result depends on what the transferred model understands.

7. Save check_offline_model.py.
8. Archive the complete offline-model directory with the check script.

   $ tar -czf offline-model.tar.gz \
       offline-model check_offline_model.py

   The whole directory has to travel. A missing config.json, weights file, or 1_Pooling folder raises an error on the offline machine, but a missing tokenizer.json does not: the model still loads and produces meaningless embeddings.

9. Transfer offline-model.tar.gz to a working directory on the offline machine using an approved removable drive or file-transfer channel.

   The archive contains model files rather than the Hugging Face cache or login credentials.

   If the offline machine lacks the Python packages, run the first command below on a connected machine with the same Python version and platform, and move wheelhouse with the archive. The extra index selects the CPU PyTorch wheel, as the install guide does, for a folder of about 250 MB. The second command installs the wheels in the offline machine's Python environment.
   Related: pip download reference
   Related: Install a local Python wheel with pip

   $ python3 -m pip download sentence-transformers \
       --extra-index-url https://download.pytorch.org/whl/cpu \
       --dest wheelhouse

   $ python3 -m pip install --no-index \
       --find-links=wheelhouse sentence-transformers

10. Extract the archive in a working directory without an existing offline-model folder.

    $ tar -xzf offline-model.tar.gz

11. Confirm that the transferred model scores the related sentence clearly higher than the unrelated one with Hugging Face offline mode enabled.

    $ HF_HUB_OFFLINE=1 python3 check_offline_model.py
    Loading weights: 100%|██████████| 55/55 [00:00<00:00, 8813.25it/s]
    tensor([[0.6882, 0.0722]])

    The first score compares the related sentences and the second the unrelated ones, using Sentence Transformers 6.2.0 on Python 3.12 on Linux. A transfer without tokenizer.json gives nearly equal high scores instead, because every word becomes an unknown token.

    The environment variable is set before Python imports the libraries. It disables Hugging Face Hub requests, while an air gap or network policy enforces the machine's actual network isolation. A notebook needs the variable set before imports, with a kernel restart if the libraries are already loaded.
    Related: Hugging Face offline mode