What is a large language model?

A model breaks text into small pieces called tokens. A token can be a word, part of a word or a single character.

Training starts with pretraining. The model works through a very large body of text and learns to predict the next piece from what came before. The Transformer architecture rests on a method called attention, which weighs how each piece relates to the others.

A pretrained model is not naturally good at following instructions. It is then fine-tuned on example answers written by people and trained further on their rankings of its answers. That second step is called reinforcement learning from human feedback (RLHF).

When writing, the model works out the probability of each possible next piece and picks one. A setting called temperature controls how freely it picks. Even at the lowest setting, the same question may not get word-for-word the same answer.

An LLM is most useful where both the input and the output are text. Sorting incoming email by topic, summarising a long contract and pulling names, dates and amounts out of a free-text enquiry are typical cases. Writing the first draft of a quote, a reply or a report belongs in the same group.

A model can also pick up a new task without retraining, from instructions and a few examples written into the request.

Chatbots and AI agents are built on top of this model. A chatbot uses it to hold a written conversation, while an AI agent connects it to tools and works step by step. The language model is the part they share.

A model's general knowledge comes from its training data, which does not include your price list, customer records or internal procedures. That information is supplied with the request. The amount of text a model can read at once is limited, and that limit is called the context window.

The method for this is retrieval-augmented generation (RAG). When a question arrives, the relevant passage is looked up in a document store and sent to the model with the question. The answer is then written from that passage. The method suits work that needs current information, specialist knowledge or cited sources.

An answer is only as good as its source, so the setup names the document that counts as the source of truth and says when it is updated. Retraining the model on extra data (fine-tuning) is a separate route, used to adapt it to a domain, a task or a writing style.

The first limit is invented answers, known as hallucination. Where a model is unsure, it can still write a fluent, convincing sentence that is wrong. Researchers link this to how models are trained and scored: on common benchmarks, guessing earns a higher score than admitting uncertainty.

Anthropic's documentation lists ways to reduce the risk. They include letting the model say it does not know, backing answers with direct quotes from the supplied document and asking for a source for each claim. The same page notes that these techniques do not remove hallucinations entirely.

The second limit is out-of-date knowledge. A model's knowledge stops where its training data ends, and vendors publish that cutoff for each model. Where current information matters, the answer is drawn from a source.

The third limit is consistency. The same question can get different answers on two runs, so amounts, dates and calculations are checked separately. Steps that message a customer, make a payment or delete a record wait for human approval.

Text sent to a model is processed on the infrastructure of the vendor that runs it, so the first question is what leaves your systems. Personal data is removed when it is not needed, and only the question and the relevant passage are sent.

The second question is training. Anthropic states that, by default, it does not use inputs or outputs from its commercial products to train models. OpenAI states that data sent to its API is not used for training unless the customer explicitly opts in. Consumer apps may work differently, so a separate rule covers staff who upload company data through personal accounts.

The third question is the written record under GDPR. The privacy notice says which data goes to which vendor, where it is processed, how long it is kept and who can open it. Some vendors offer processing in a set region, Europe included, and the chosen region goes into the contract.

The choice starts with the job. Classifying short messages and reading long documents do not need the same model, and vendors offer fast, light models alongside slower, more capable ones in the same family.

The decision rests on your own examples. Candidate models answer the same set of real requests, the answers are read, and the set is run again whenever the model changes.

The connection runs through an API to your CRM, inbox or document store, and existing systems stay in place. Every run is written to a log. In AI integration work, ALTAI Digital writes down the reason for the model choice and hands the setup over with its documentation.