How to train with multiple negatives ranking loss in Sentence Transformers

Retrieval fine-tuning starts with pairs of text that should land close together in embedding space. In Sentence Transformers, the multiple negatives ranking loss class turns the other positives in the same batch into implicit negatives, so anchor-positive rows can train a bi-encoder without a separately labeled negative column.

The Python workflow uses SentenceTransformerTrainer, a small Dataset with anchor and positive columns, and InformationRetrievalEvaluator to compare the base model against the trained model on a tiny retrieval check. The saved directory is then loaded again to confirm that the output can be used like any other Sentence Transformers model.

Batch contents matter more for this loss than for ordinary pairwise objectives. Keep duplicate or near-duplicate positives out of the same batch, use the no-duplicates batch sampler when possible, and switch to hard-negative triplets or the cached loss variant when larger data or larger effective batches require it.

1. Install Sentence Transformers with training dependencies in the active Python environment.

   $ python -m pip install --upgrade "sentence-transformers[train]"

   The training extra installs the datasets and accelerate packages used by the current trainer workflow.

2. Create the training script with anchor-positive rows and a retrieval evaluator.

   train_mnrl.py

   from pathlib import Path
   import shutil
    
   from datasets import Dataset
   from sentence_transformers import (
       SentenceTransformer,
       SentenceTransformerTrainer,
       SentenceTransformerTrainingArguments,
   )
   from sentence_transformers.sentence_transformer.evaluation import InformationRetrievalEvaluator
   from sentence_transformers.sentence_transformer.losses import MultipleNegativesRankingLoss
   from sentence_transformers.sentence_transformer.training_args import BatchSamplers
    
    
   output_dir = Path("support-mnrl-model")
   training_dir = Path("mnrl-training-output")
    
   shutil.rmtree(output_dir, ignore_errors=True)
   shutil.rmtree(training_dir, ignore_errors=True)
    
   train_dataset = Dataset.from_dict(
       {
           "anchor": [
               "reset a locked account",
               "reset a user password",
               "enable two-factor authentication",
               "rotate an API token",
               "restore a deleted project",
               "invite a new team member",
               "export audit logs",
               "change the billing contact",
           ],
           "positive": [
               "Unlock the account from the admin users page.",
               "Open the user profile and send a password reset email.",
               "Open account security and enroll an authenticator app.",
               "Revoke the old API token and create a replacement token.",
               "Open deleted projects and restore the selected project.",
               "Open team settings and send an invitation email.",
               "Open compliance reports and export the audit log CSV.",
               "Open billing settings and update the primary contact.",
           ],
       }
   )
    
   queries = {
       "q1": "How do I send a password reset?",
       "q2": "How do I replace an API token?",
       "q3": "How do I recover a deleted project?",
   }
   corpus = {
       "d1": "Open the user profile and send a password reset email.",
       "d2": "Revoke the old API token and create a replacement token.",
       "d3": "Open deleted projects and restore the selected project.",
       "d4": "Open billing settings and update the primary contact.",
   }
   relevant_docs = {
       "q1": {"d1"},
       "q2": {"d2"},
       "q3": {"d3"},
   }
    
   model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
   evaluator = InformationRetrievalEvaluator(
       queries=queries,
       corpus=corpus,
       relevant_docs=relevant_docs,
       name="support-faq",
       accuracy_at_k=[1],
       precision_recall_at_k=[1, 3],
       map_at_k=[3],
       ndcg_at_k=[3],
       show_progress_bar=False,
   )
    
   before = evaluator(model)
    
   args = SentenceTransformerTrainingArguments(
       output_dir=str(training_dir),
       num_train_epochs=1,
       per_device_train_batch_size=4,
       learning_rate=2e-5,
       warmup_steps=0.0,
       batch_sampler=BatchSamplers.NO_DUPLICATES,
       save_strategy="no",
       eval_strategy="no",
       logging_steps=1,
       disable_tqdm=True,
       report_to=[],
   )
    
   loss = MultipleNegativesRankingLoss(model)
   trainer = SentenceTransformerTrainer(
       model=model,
       args=args,
       train_dataset=train_dataset,
       loss=loss,
   )
    
   trainer.train()
   after = evaluator(model)
   model.save_pretrained(output_dir)
    
   reloaded = SentenceTransformer(str(output_dir))
   embedding = reloaded.encode(["How do I replace an API token?"], convert_to_numpy=True)
    
   metric_key = "support-faq_cosine_ndcg@3"
   print(f"train rows: {len(train_dataset)}")
   print(f"loss: {loss.__class__.__name__}")
   print(f"retrieval metric: {metric_key}")
   print(f"before score: {before[metric_key]:.4f}")
   print(f"after score: {after[metric_key]:.4f}")
   print(f"saved model path: {output_dir}")
   print(f"reloaded embedding shape: {embedding.shape}")

   MultipleNegativesRankingLoss treats the other positives in the current batch as negatives for each anchor. BatchSamplers.NO_DUPLICATES reduces false negatives caused by repeated or equivalent pairs in the same batch.

3. Run the training script.

   $ python train_mnrl.py
   {'loss': '0.0005047', 'grad_norm': '0.04047', 'learning_rate': '2e-05', 'epoch': '0.5'}
   {'loss': '0.005828', 'grad_norm': '0.699', 'learning_rate': '1e-05', 'epoch': '1'}
   ##### snipped #####
   train rows: 8
   loss: MultipleNegativesRankingLoss
   retrieval metric: support-faq_cosine_ndcg@3
   before score: 1.0000
   after score: 1.0000
   saved model path: support-mnrl-model
   reloaded embedding shape: (1, 384)

   A tiny retrieval set can already score perfectly before training. For this smoke run, the important signals are completed loss logging, the retrieval metric, the saved model path, and the reload shape.

4. Confirm that the saved model reloads and returns the expected embedding width.

   reloaded embedding shape: (1, 384)