Semantic search compares text through numeric vectors rather than literal word matches. Ollama can generate these embeddings directly in the terminal with an embedding model such as all-minilm.

Embedding output is a JSON array rather than prose. The vector length depends on the model, so the same model should be used for both indexing documents and embedding search queries.

The all-minilm:latest model produces 384 values per input. A running Ollama server, an installed copy of that model, Bash or a compatible shell, and Python 3 are required.

Steps to generate embeddings with Ollama:

  1. Inspect the installed all-minilm:latest model's embedding capability.
    $ ollama show all-minilm:latest
      Model
        architecture        bert
        parameters          23M
        context length      512
        embedding length    384
        quantization        F16
     
      Capabilities
        embedding
    ##### snipped #####

    The embedding capability identifies a model that returns vectors; an embedding length field alone does not establish that capability. If the model is missing, it needs to be downloaded as all-minilm:latest before continuing. A connection error means the CLI cannot reach the server.
    Related: How to pull an Ollama model
    Related: How to start the Ollama server

  2. Generate an embedding for the library's closing time.
    $ ollama run all-minilm:latest "The library closes at six."
    [0.053679496,-0.04358265,-0.04447335,##### snipped #####]

    The displayed vector is shortened. The complete JSON array contains 384 numbers; truncated text containing snipped cannot be parsed as JSON.

  3. Save the embedding for piped text in museum-embedding.json.

    Shell redirection with > replaces an existing museum-embedding.json file in the current directory.

    $ printf '%s\n' 'The museum opens at nine.' | ollama run all-minilm:latest > museum-embedding.json

    The full vector is written to the file, so successful execution produces no vector on the terminal.

  4. Verify that the saved vector contains 384 finite numbers.
    $ python3 - <<'PY'
    import json
    import math
     
    with open("museum-embedding.json") as source:
        vector = json.load(source)
    assert isinstance(vector, list) and len(vector) == 384
    assert all(type(value) in (int, float) and math.isfinite(value) for value in vector)
    print(f"Dimensions: {len(vector)}")
    print("All values finite: True")
    PY
    Dimensions: 384
    All values finite: True

    Invalid JSON, a different dimension count, or a nonnumeric or nonfinite value causes the check to fail. The complete vector remains in museum-embedding.json for later use.

Share this guide