Prompts, RAG, tools, agents, MCP

Prompts: system, user, few-shot

A request to a chat model is made of messages with roles: system (rules, role, format; highest priority), user (the person's request), assistant (the model's earlier answers), tool (results of tool calls).

Basic techniques:

• Zero-shot: only an instruction, no examples;
• Few-shot: a few "input → expected output" examples in the prompt; the best way to pin down a format;
• Structured output: require JSON matching a schema; the answer is then validated (JSON Schema / zod) because the model can still break the format;
• Chain-of-thought: ask the model to reason step by step before answering; it improves accuracy on hard tasks but adds tokens.

For QA a prompt is code: changing it can break behaviour just like changing a function. So prompts live in git, are versioned and are covered by an eval set on every change.

RAG: embeddings, chunking, retrieval

RAG (retrieval-augmented generation) means the model answers based on documents the system found and put into the context. It is a way to give the model fresh or private data without fine-tuning.

The pipeline: documents → chunking (splitting into fragments) → embeddings (a vector encoding the meaning) → vector DB → for a query find the top-k nearest fragments → insert them into the prompt → the model answers, ideally with source references.

Where RAG breaks, which is exactly where to test:

• Retrieval did not find the needed fragment (bad chunking, weak embeddings, k too small): the model answers without a source or makes things up;
• An irrelevant or outdated fragment got into the context;
• The model ignores the retrieved context and answers from "memory";
• The answer is not grounded in the documents (groundedness) or cites the wrong source;
• Access rights: a user received a fragment of a document they had no access to.

That is why retrieval and generation are evaluated separately: recall@k / precision@k for search and faithfulness / answer relevance for the answer.

Tool calling, agent and MCP

Tool calling (function calling): we describe tools to the model (name, description, JSON Schema of parameters). The model does not run them itself: it returns "I want to call search_orders with these arguments", our code performs the call and puts the result back into the context, and the model continues.

An agent is a loop: the model gets a goal → decides which tool to call → sees the result → decides what next → … → a final answer or a stop at a limit. Unlike a plain chatbot, an agent has actions with side effects (create a ticket, send an email, change data) and chooses their order itself.

while (steps++ < MAX_STEPS) {
  const reply = await model.call(messages, tools);
  if (!reply.toolCalls.length) return reply.text;      // final answer
  for (const call of reply.toolCalls) {
    const result = await runTool(call.name, call.args); // our code, not the model
    messages.push(toolResult(call.id, result));
  }
}
throw new Error('Agent exceeded step limit');

MCP (Model Context Protocol) is an open protocol that lets an application with a model connect external tools, resources and prompts through a standard interface: one MCP server works with different clients. For QA it is one more surface: the tool contract, their schemas, permissions and server error handling.