Top 20 AI Concepts in LLM: Detailed Explanations with Examples

1. Overview

If you’re a Java developer starting to integrate LLM (ChatGPT, Gemini, Claude…) into your system, you’ll encounter many unfamiliar terms: token, embedding, RAG, fine-tuning, hallucination… This article summarizes the 20 most important AI concepts related to LLM, each with a brief explanation and illustrative examples (mostly in Java/Spring AI). After reading, you’ll have enough vocabulary to read OpenAI, Anthropic API documentation or design a basic RAG system.

2. Token and Tokenization

LLM doesn’t read “characters” but reads tokens — units smaller than words, could be a word, part of a word, or punctuation. Tokenization is the step of breaking text into these tokens before feeding it into the model.

Input:  "Spring Boot is very powerful"
Tokens: ["Spring", " Boot", " is", " very", " powerful"]  (5 tokens)

The number of tokens determines the cost of calling the API (billing per token) and the input/output length limit.

Note: Vietnamese with diacritics is usually split into more tokens than English for the same sentence, so API call costs for Vietnamese content are typically higher.

3. Embedding

Embedding is the process of converting a text passage into a vector of real numbers (e.g., 1536 dimensions), such that two passages with similar meaning have vectors close to each other. This is the foundation for semantic search.

EmbeddingModel embeddingModel = ...; // bean auto-configured by Spring AI
EmbeddingResponse response = embeddingModel.embedForResponse(
        List.of("Customer wants to cancel order"));
float[] vector = response.getResult().getOutput();
System.out.println(vector.length); // 1536

embedForResponse calls the embedding model and returns a vector. Two sentences “cancel order” and “hủy đơn hàng” will have similar vectors even though they’re in different languages.

To compare semantic similarity, use cosine similarity:

double cosineSimilarity(float[] a, float[] b) {
    double dot = 0, normA = 0, normB = 0;
    for (int i = 0; i < a.length; i++) {
        dot += a[i] * b[i];
        normA += a[i] * a[i];
        normB += b[i] * b[i];
    }
    return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

@Test
void similarVectorsShouldScoreHigh() {
    float[] a = {1f, 0f, 1f};
    float[] b = {1f, 0.1f, 0.9f};
    assertThat(cosineSimilarity(a, b)).isGreaterThan(0.95);
}

A value close to 1 means the two vectors (and therefore the two sentences) are very similar in meaning.

4. Context Window

Context window is the maximum number of tokens a model can “see” in one call — including both the prompt and the response. Exceeding this limit will cause the model to truncate the beginning or return an error.

GPT-4o:        128,000 tokens
Claude 3.5:    200,000 tokens
Gemini 1.5 Pro: 1,000,000 tokens

With applications like chatbots with long conversation history, you must proactively trim/summarize old history before the context window fills up, otherwise you’ll lose information from the beginning of the conversation without clear warning.

5. Prompt and Prompt Engineering

Prompt is the text you send to the LLM to request what it should do. Prompt engineering is the technique of designing the prompt so the model answers correctly and in the desired format.

First, add the Spring AI dependency:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
    <version>1.0.0</version>
</dependency>

@Service
class OrderSummaryService {
    private final ChatClient chatClient;

    OrderSummaryService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    String summarize(String orderNote) {
        return chatClient.prompt()
                .user("Summarize the customer's request in 1 sentence: " + orderNote)
                .call()
                .content();
    }
}

ChatClient is Spring AI’s wrapper for calling models without manually handling HTTP. .user(...) sets the prompt content, .call().content() returns the result string.

6. System Prompt vs User Prompt

System prompt is “implicit” instructions that define the role and rules of the model throughout the conversation; user prompt is the specific question/request from the user.

chatClient.prompt()
        .system("You are a customer support assistant for a shoe shop. Always answer briefly and politely.")
        .user("When will my order #1023 be delivered?")
        .call()
        .content();

System prompt only needs to be set once, user prompt changes with each request. Separating these two helps you control the model’s “personality” without repeating instructions in every question.

7. Zero-shot, One-shot, Few-shot Learning

This is how you provide (or don’t provide) example samples in the prompt to teach the model how to answer in the desired format, without needing to fine-tune the model again.

Zero-shot: No examples, just description of requirements.
Few-shot:  2-5 example input -> output samples before the actual question.

String fewShotPrompt = """
    Classify review sentiment (positive/negative):
    Review: "Fast delivery, good quality" -> positive
    Review: "Defective product, delivered a week late" -> negative
    Review: "%s" ->
    """.formatted(userReview);

Few-shot usually gives more stable results than zero-shot for classification/format-specific tasks, but uses more tokens with each call.

8. Chain-of-Thought (CoT)

CoT is a technique to ask the model to “think step by step” before giving the final answer, helping increase accuracy for problems that require reasoning (math, logic).

chatClient.prompt()
        .user("""
            An order has 3 products priced at 120k, 85k, 45k, with 10% discount.
            Calculate step by step then provide the total final amount.
            """)
        .call()
        .content();

When asked to go “step by step”, the model tends to break down the problem into smaller calculations, reducing the chance of miscalculation compared to asking directly “what’s the total?”.

9. Temperature and Top-p

Temperature and top-p are two parameters that adjust the “randomness” when the model chooses the next word. Low temperature → consistent, repetitive answers; high → more creative but likely to drift off-topic.

ChatOptions options = OpenAiChatOptions.builder()
        .withTemperature(0.2)
        .withTopP(0.9)
        .build();

temperature = 0.0  -> suitable for: data extraction, factual answers
temperature = 0.8  -> suitable for: creative writing, brainstorming

Note: For tasks requiring reproducible results (e.g., automated testing), set temperature = 0 to reduce variation between calls.

10. Attention Mechanism

Attention is the mechanism that allows the model, when processing one word, to “look at” all other words in the sentence and automatically calculate weights for how relevant each word is. This is why the model can understand distant context, for example a pronoun “it” referring to which noun.

Sentence: "The order was canceled because the customer changed their mind, it has been refunded."
Attention helps the model link "it" -> "order", not "customer".

You don’t code attention directly when using the API, but understanding it helps explain why the model sometimes “forgets” context that’s too far away or the sentence is too long.

11. Transformer Architecture

Transformer is the neural network architecture underlying almost all modern LLMs, using attention to process the entire sentence simultaneously (in parallel) instead of sequentially like old RNNs, thus enabling faster training on massive data.

Input tokens -> Embedding -> [Attention + Feed-Forward] x N layers -> Output probabilities

The number of layers (N) and the number of parameters decide the “size” of the model — GPT-4, Claude, Llama are all variations of Transformer with different layer counts and training methods.

12. Fine-tuning

Fine-tuning is continuing the training of an already-trained (pretrained) model on your own data to specialize it for a specific task (e.g., answering in your brand’s tone).

{"messages": [
  {"role": "user", "content": "When will my order #203 be delivered?"},
  {"role": "assistant", "content": "Your order #203 is on the way, expected to arrive today."}
]}

This JSONL file is one line of training data for fine-tuning — each line is a sample conversation pair, you send thousands of such lines to the provider’s fine-tuning API.

When NOT to use: if you only need the model to follow a few simple rules, prompt engineering or RAG is much cheaper and faster than fine-tuning.

13. LoRA (Low-Rank Adaptation)

LoRA is a cost-efficient fine-tuning technique: instead of updating all the trillions of parameters of the original model, LoRA only trains small matrices “inserted” into layers, keeping the original model unchanged.

Original model: 7 billion parameters (frozen, not changed)
LoRA adapter: ~10-50 million parameters (trained)

This way, LoRA allows you to fine-tune large models on a regular GPU, and you can “plug/unplug” multiple adapters for different use cases on the same base model.

14. Quantization

Quantization is reducing the numerical precision of model parameters (e.g., from float32 down to int8/int4) to reduce size and increase inference speed, trading off a bit of accuracy.

FP32: 7B params x 4 bytes = ~28 GB RAM
INT4: 7B params x 0.5 byte = ~3.5 GB RAM

Quantization is why you can run a 7-billion-parameter LLM on a regular laptop instead of needing a dedicated GPU server — at the cost of slightly reduced answer quality.

15. RAG (Retrieval-Augmented Generation)

RAG is a technique combining retrieval of internal data with LLM: before answering, the system finds the most relevant text passages (via embedding) and includes them in the prompt as “context”, helping the model answer based on your own data without needing fine-tuning.

List<Document> relevantDocs = vectorStore.similaritySearch(
        SearchRequest.query("return policy").withTopK(3));

String context = relevantDocs.stream()
        .map(Document::getContent)
        .collect(Collectors.joining("\n---\n"));

String answer = chatClient.prompt()
        .system("Only answer based on the provided context.")
        .user("Context:\n" + context + "\n\nQuestion: How many days for return?")
        .call()
        .content();

similaritySearch finds the 3 most semantically similar document passages to the question, concatenates them into the prompt, then sends it to the model. This way the answer sticks to your actual policy, not “general knowledge” the model learned during pretraining.

16. Vector Database

Vector database is storage for embeddings with metadata, supporting “k nearest neighbors” (k-NN) search fast across millions of vectors — a mandatory component for RAG to scale (Pinecone, Milvus, PGVector…).

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT,
    embedding VECTOR(1536)
);

SELECT content FROM documents
ORDER BY embedding <-> '[0.01, -0.02, ...]'
LIMIT 3;

The <-> operator (provided by the pgvector extension) calculates distance between the question vector and stored vectors, returning the nearest records — this is exactly the similaritySearch step from section 15.

17. Function Calling / Tool Use

Function calling allows LLM to “request” your application to call a specific function (check price, check weather, query DB) when it sees the need, instead of making up data.

@Tool(description = "Get order status by order code")
String getOrderStatus(String orderId) {
    return orderRepository.findStatusById(orderId);
}

chatClient.prompt()
        .user("What status is order #1023 in?")
        .tools(this)
        .call()
        .content();

When you send the question, the model itself recognizes it needs to call getOrderStatus, Spring AI executes the actual Java function, gets the result, and passes it back to the model to compose the final answer — the model no longer has to “guess” the order status.

18. AI Agent

Agent is a system using LLM as a “brain” to auto-plan, call multiple tools in sequence, observe results and iterate until a goal is reached — different from chatbot that only responds once.

User: "Check order #1023, if it's delayed more than 3 days send an apology email to customer."

Agent automatically performs:
1. Call getOrderStatus("1023") tool
2. Calculate days delayed
3. If > 3 -> call sendEmail(...) tool
4. Return report to user

Agents are typically built with “think → act → observe” loops (ReAct pattern), combining multiple function calls in sequence to handle multi-step tasks.

19. Hallucination

Hallucination is when the model answers confidently but the information is wrong or completely made up — for example citing a law that doesn’t exist, or misremembering a non-existent API.

User: "How does the API `OrderService.cancelAndRefund()` work?"
Model: "This method automatically refunds within 24 hours..." (but this method doesn't exist in the codebase)

The most common way to reduce hallucination is using RAG (section 15) to ground the model in real documents, and require the model to answer “I don’t know” when the context has insufficient information, instead of inferring on its own.

20. Guardrails

Guardrails are a set of control layers (filter input/output, validate format, block sensitive content) placed around LLM to ensure the system is safe and compliant before returning results to end users.

String output = chatClient.prompt().user(userInput).call().content();

if (containsPersonalData(output) || !isValidJson(output)) {
    throw new UnsafeModelOutputException("Output does not meet guardrail standards");
}

Guardrails can be on input (block prompt injection, harmful content) or output (validate JSON format, mask credit card numbers) — don’t skip this step when deploying LLM to production, even if the model “seems” to answer correctly in demos.

21. RLHF (Reinforcement Learning from Human Feedback)

RLHF is a training technique based on human feedback: humans rank multiple model answers, from which a “reward model” is trained to further optimize the base model so it gives more helpful and safe answers.

Step 1: Model generates 3-4 answers for the same question
Step 2: Human ranks them: best -> worst
Step 3: Train reward model to learn how to "score" like humans
Step 4: Use reward model to fine-tune LLM via reinforcement learning

RLHF is why ChatGPT, Claude answers have a “polite” tone, refuse dangerous requests — very different from base models that only learn to “predict the next word” purely.

22. Conclusion

The 20 concepts above are the minimum vocabulary to work seriously with LLM: from low-level (token, embedding, attention) to application-level (RAG, function calling, agent, guardrails). You don’t need to deeply understand the math of Transformer to integrate LLM into Spring Boot, but understanding these concepts correctly helps you choose the right solution — for example knowing when to use RAG instead of fine-tuning, when you need guardrails instead of blindly trusting model output.