Running Embedding Models with sentence-transformers

1. Overview

Embedding is a way to convert a sentence into a numerical vector. Two sentences with similar meanings will produce two similar vectors. This is the foundation of semantic search, RAG, text clustering, and content recommendations.

In this article, you will use the sentence-transformers library to run an embedding model from Hugging Face directly on your machine. After reading, you will know how to create embeddings, calculate similarity, perform semantic search, write tests, and expose the model as an HTTP service for a Spring Boot application to call.

2. Installation

sentence-transformers is a Python library that runs on PyTorch. You should create a separate virtual environment:

python -m venv .venv
source .venv/bin/activate
pip install sentence-transformers

This command pulls in torch, transformers, and huggingface_hub. If your machine has an NVIDIA GPU, install the CUDA version of torch first, then install sentence-transformers.

The first time you load a model, the library will download it from Hugging Face Hub to a cache directory (default is ~/.cache/huggingface). After that, the model is read directly from the cache.

3. Creating your first embedding

We start with all-MiniLM-L6-v2. This is a small model (around 90 MB), fast, and suitable for experimentation with English text:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

sentences = [
    "The order was shipped yesterday.",
    "Your package left our warehouse last night.",
    "I would like to change my password.",
]

embeddings = model.encode(sentences)

print(embeddings.shape)
print(model.get_sentence_embedding_dimension())

Explanation:

• SentenceTransformer(...) loads the model and tokenizer from Hub, or from cache if already present.
• model.encode(sentences) takes a list of strings and returns a numpy.ndarray. Each sentence corresponds to one row.
• get_sentence_embedding_dimension() tells you the number of dimensions of the vector.

Output:

(3, 384)
384

You have three sentences, each becomes a 384-dimensional vector. Different models will have different dimensions, for example 768 or 1024.

3.1. Vietnamese text

all-MiniLM-L6-v2 was trained primarily on English. For Vietnamese, you should use a multilingual model:

model = SentenceTransformer(
    "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
)

vectors = model.encode([
    "Đơn hàng đã được giao cho đơn vị vận chuyển.",
    "Gói hàng của bạn đã rời kho tối qua.",
])
print(vectors.shape)

(2, 384)

This model also produces 384-dimensional vectors. It understands around 50 languages, including Vietnamese.

4. Calculating similarity

Vectors are only useful when you compare them with each other. The most common measure is cosine similarity. From version 3, the library has model.similarity:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

sentences = [
    "The order was shipped yesterday.",
    "Your package left our warehouse last night.",
    "I would like to change my password.",
]
embeddings = model.encode(sentences)

scores = model.similarity(embeddings, embeddings)
print(scores)

model.similarity(a, b) returns a torch.Tensor matrix where cell [i][j] is the similarity between sentence i and sentence j. The output looks something like this (numbers may vary slightly depending on version):

tensor([[1.0000, 0.6123, 0.0412],
        [0.6123, 1.0000, 0.0587],
        [0.0412, 0.0587, 1.0000]])

The diagonal is always 1 because each sentence is identical to itself. The two sentences about shipping have a high score, even though they hardly share any words. The sentence about password change is essentially unrelated.

4.1. Normalizing vectors

If you plan to store vectors in a vector database, normalize them to unit length:

embeddings = model.encode(sentences, normalize_embeddings=True)
score = embeddings[0] @ embeddings[1]
print(round(float(score), 4))

0.6123

When vectors are normalized, the dot product (@) gives the exact cosine similarity value. This operation is cheaper, and many databases like pgvector, Qdrant, and Elasticsearch are optimized for it.

5. Semantic search

A practical problem is finding the best matching answer from an FAQ database for a customer’s question. The util.semantic_search function does this:

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer(
    "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
)

faq = [
    "Làm sao để theo dõi đơn hàng?",
    "Chính sách hoàn tiền như thế nào?",
    "Tôi quên mật khẩu đăng nhập.",
    "Phí vận chuyển ra nước ngoài là bao nhiêu?",
]
faq_embeddings = model.encode(faq, convert_to_tensor=True, normalize_embeddings=True)

query = "Hàng của tôi đang ở đâu rồi?"
query_embedding = model.encode(query, convert_to_tensor=True, normalize_embeddings=True)

hits = util.semantic_search(query_embedding, faq_embeddings, top_k=2)[0]
for hit in hits:
    print(f"{hit['score']:.3f}  {faq[hit['corpus_id']]}")

Explanation:

• You encode the entire FAQ once. In a real application, this step runs at startup or when data changes.
• convert_to_tensor=True keeps the result as a tensor, so you don’t have to convert back and forth with numpy.
• semantic_search returns a list for each query. Each element is a dict with corpus_id and score, sorted in descending order.

Sample output:

0.721  Làm sao để theo dõi đơn hàng?
0.388  Phí vận chuyển ra nước ngoài là bao nhiêu?

This approach works well for tens of thousands of documents kept in memory. With millions of documents, you should switch to a vector database with ANN index.

Note: Some models like the E5 or BGE families require a prefix, for example "query: " for questions and "passage: " for documents. Omitting the prefix can significantly reduce search quality. Check the model card before using it. You can pass prompt="query: " to encode.

6. Performance optimization

encode has several parameters that directly affect speed:

import torch
from sentence_transformers import SentenceTransformer

device = "cuda" if torch.cuda.is_available() else "cpu"
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device=device)

documents = [f"Customer review number {i}" for i in range(10_000)]

embeddings = model.encode(
    documents,
    batch_size=64,
    show_progress_bar=True,
    normalize_embeddings=True,
)
print(embeddings.shape, device)

Batches: 100%|██████████| 157/157 [00:09<00:00, 16.8it/s]
(10000, 384) cpu

• device chooses CPU or GPU. On GPU, speed can be several times faster.
• batch_size defaults to 32. On GPU, increasing it to 64–256 often helps run faster, as long as you don’t run out of VRAM.
• Always pass the entire list to encode. Calling encode for each sentence in a for loop is much slower.

Each model also has a maximum input length, which you can check via model.max_seq_length. For all-MiniLM-L6-v2, this is 256 tokens. Text exceeding this is silently truncated with no warning. Therefore, long documents need to be chunked before encoding.

7. Writing tests for embeddings

Embeddings are floating-point numbers, so you can’t compare individual values against fixed numbers. Test properties instead of specific values. Install pytest:

pip install pytest

# test_embeddings.py
import numpy as np
import pytest
from sentence_transformers import SentenceTransformer


@pytest.fixture(scope="module")
def model():
    return SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")


def test_embedding_has_expected_dimension(model):
    vectors = model.encode(["Order #42 is delayed"])
    assert vectors.shape == (1, 384)


def test_normalized_embedding_has_unit_length(model):
    vector = model.encode("Refund request", normalize_embeddings=True)
    assert np.linalg.norm(vector) == pytest.approx(1.0, abs=1e-5)


def test_similar_sentences_score_higher(model):
    shipping = model.encode("The order was shipped yesterday.")
    package = model.encode("Your package left our warehouse last night.")
    password = model.encode("I would like to change my password.")

    related = model.similarity(shipping, package).item()
    unrelated = model.similarity(shipping, password).item()
    assert related > unrelated

The fixture with scope="module" means the model is loaded only once for the entire file. The third test catches errors if someone accidentally switches to an unsuitable model.

8. Calling the model from a Spring Boot application

Most of your backend might be written in Java. The simplest way is to wrap the model as a small HTTP service using FastAPI, then call it from Spring Boot.

8.1. Embedding service with FastAPI

pip install fastapi uvicorn

# app.py
from fastapi import FastAPI
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer

app = FastAPI()
model = SentenceTransformer(
    "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
)


class EmbedRequest(BaseModel):
    texts: list[str]


@app.post("/embed")
def embed(request: EmbedRequest):
    vectors = model.encode(request.texts, normalize_embeddings=True)
    return {
        "embeddings": vectors.tolist(),
        "dimension": model.get_sentence_embedding_dimension(),
    }

uvicorn app:app --port 8000

The model is loaded once when the module starts, not reloaded for each request. The endpoint uses def instead of async def. The reason is that encode is a blocking operation, and FastAPI will run the def function in a separate thread pool instead of blocking the event loop.

8.2. Client in Spring Boot

Maven dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

embedding.base-url=http://localhost:8000

public record EmbedRequest(List<String> texts) {}

public record EmbedResponse(List<List<Float>> embeddings, int dimension) {}

@Service
public class EmbeddingClient {

    private final RestClient restClient;

    public EmbeddingClient(RestClient.Builder builder,
                           @Value("${embedding.base-url}") String baseUrl) {
        this.restClient = builder.baseUrl(baseUrl).build();
    }

    public List<List<Float>> embed(List<String> texts) {
        EmbedResponse response = restClient.post()
                .uri("/embed")
                .body(new EmbedRequest(texts))
                .retrieve()
                .body(EmbedResponse.class);
        return response.embeddings();
    }
}

RestClient (available since Spring Framework 6.1) sends EmbedRequest as JSON, then maps the response into the record EmbedResponse using Jackson. You can use the result to store in pgvector or compare with other vectors.

8.3. Testing the client with MockRestServiceServer

import static org.assertj.core.api.Assertions.assertThat;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.*;
import static org.springframework.test.web.client.response.MockRestResponseCreators.withSuccess;

class EmbeddingClientTest {

    @Test
    void shouldReturnEmbeddingsFromService() {
        RestClient.Builder builder = RestClient.builder();
        MockRestServiceServer server = MockRestServiceServer.bindTo(builder).build();

        server.expect(requestTo("http://localhost:8000/embed"))
              .andExpect(method(HttpMethod.POST))
              .andExpect(jsonPath("$.texts[0]").value("Đơn hàng bị trễ"))
              .andRespond(withSuccess("""
                      {"embeddings": [[0.1, 0.2, 0.3]], "dimension": 3}
                      """, MediaType.APPLICATION_JSON));

        EmbeddingClient client = new EmbeddingClient(builder, "http://localhost:8000");

        List<List<Float>> result = client.embed(List.of("Đơn hàng bị trễ"));

        assertThat(result).hasSize(1);
        assertThat(result.getFirst()).containsExactly(0.1f, 0.2f, 0.3f);
        server.verify();
    }
}

MockRestServiceServer.bindTo(builder) must be called before the builder creates the RestClient. In this test, that’s always true because the client only calls build() inside its constructor. This way, the test runs without needing an actual Python service.

9. Common mistakes

• Mixing vectors from different models. Vectors from two different models lie in two different spaces, even if they have the same dimension. Switching models means re-encoding all your data.
• Forgetting to normalize on one side. If documents are normalized but queries are not, the dot product will be incorrect. Use the same encode configuration on both sides.
• Text that’s too long. Parts exceeding max_seq_length are silently truncated. Chunk your documents before encoding.
• Loading the model in every request. This takes several seconds and a lot of RAM. Load it only once at startup.
• Container has no network access. Production environments often block Hub access. Download the model beforehand into the image, then set HF_HUB_OFFLINE=1.

Note: Embedding does not replace exact search. For order codes, phone numbers, or SKUs, full-text search or regular SQL queries give better results. In practice, combining both (hybrid search) is often most effective.

10. Conclusion

You’ve used sentence-transformers to create embeddings with just a few lines of code. You’ve also calculated cosine similarity, built semantic search for an FAQ database, and tuned batch_size and device for faster performance. Additionally, you’ve learned how to test embeddings through their properties, as well as how to wrap the model as an HTTP service for Spring Boot to call via RestClient.

The logical next step is to store vectors in a vector database like pgvector, then use them as the retrieval component in a RAG pipeline.