Large language models are software systems trained on vast amounts of text to predict the next word or word fragment in a sequence. Repeating that prediction produces full sentences, summaries and answers, which is why they power most modern chat-style assistants.
They sit underneath many tools described in the guide to the ai executive assistant, so knowing the basics helps in judging what those tools can and cannot do.
Large language models defined
A language model assigns probabilities to what comes next in a piece of text. The word “large” refers to the size of the model, meaning the number of adjustable internal values it learns during training, and to the size of the text collection it learns from.
NIST’s generative AI profile gives a plain description of the core mechanism: LLMs predict the next token or word in a sentence or phrase. A token is a small chunk of text, often a whole short word or part of a longer one. The model works on tokens rather than on letters or on meaning in the human sense.
This is the key idea for non-specialists. The model does not consult a database of facts. It produces text that is statistically likely given what came before, which usually yields accurate, readable writing, and sometimes does not.
Training in stages
Training happens in stages. Each stage changes the model’s internal values so that its predictions improve.
- Pre-training. The model reads a very large body of text and repeatedly tries to predict hidden or upcoming tokens. Errors are measured and the internal values are nudged to reduce them. This stage teaches grammar, common facts, writing styles and many patterns of reasoning as they appear in text.
- Fine-tuning. The pre-trained model is trained further on narrower examples, such as question-and-answer pairs, so it follows instructions and keeps a helpful format.
- Feedback and safety tuning. Human reviewers or rule sets rate outputs, and the model is adjusted toward responses judged better and away from harmful ones.
After training, the model is fixed until its developer releases an update. It does not learn from an individual conversation unless the product is built to store and reuse that information separately.
The transformer design behind modern models
Most current systems are built on the transformer architecture. It was introduced in a research paper on attention mechanisms, which proposed a network based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
Attention lets the model weigh how relevant each earlier token is when it works out the next one. In the sentence “The manager sent the report to the client because she needed it”, attention is how a model can link “she” and “it” to the right earlier words.
Two practical terms follow from this design. The context window is the amount of text the model can consider at once, covering both the question and the answer. A prompt is the input text that sets the task. Anything outside the context window is invisible to the model for that reply.
Strengths in everyday work
Because they are trained on text, language models handle language tasks well. Common examples include drafting and rewriting, summarizing long documents, classifying messages, extracting names or dates from free text, translating, and turning rough notes into structured lists. Speech tools often hand a transcript to a language model to produce notes, as covered in the page on the ai meeting assistant.
Everyday uses and their risks are laid out in the companion page on generative ai at work. The short version is that a person should be able to read the output and judge it quickly.
From model to product
A language model on its own only turns text into more text. The tools people use at work wrap it in extra layers that change what it can do and how much it can be trusted.
- Instructions. A hidden system prompt tells the model how to behave, what tone to use and what to refuse.
- Retrieval. The application searches documents, a knowledge base or the web, then places the relevant passages in the context window so the model can base its answer on them.
- Tools. Some assistants can call a calculator, a calendar or a spreadsheet, because the model itself is unreliable at exact arithmetic and has no direct access to live data.
- Guardrails. Filters check inputs and outputs for sensitive data or disallowed content.
Retrieval is a common way to make answers more grounded, because the model quotes or paraphrases supplied text instead of relying on memory alone. It reduces invented details but does not remove them, since the model can still misread or overstate a passage. For that reason, citing the source passage next to each claim makes review faster.
Where they fall short
The same prediction mechanism that makes the writing fluent also explains the main weaknesses.
- Confident errors. NIST describes confabulation as the production of confidently stated but erroneous or false content, and links it to the way models approximate their training data. The result can read as authoritative even when a detail is invented.
- Opaque training data. NIST notes that risk estimation is made harder by a lack of visibility into GAI training data, so users often cannot tell what a model has or has not seen.
- Bias. NIST lists non-representative training data among the causes of outputs that can discriminate or amplify biases.
- Limited, dated knowledge. The model reflects the text it was trained on up to a cutoff, and it may not know recent events unless it is connected to a search or document tool.
- Inconsistency. Repeating the same prompt can give different answers, because outputs are sampled from probabilities.
These limits are why careful teams keep a human reviewer in the loop for anything that is sent outside the organization or used for a decision.
Reading claims about language models
Marketing for AI products often uses terms that blur what the model actually does. A few questions help cut through them.
- Is the tool a language model itself, or an application that calls one and adds features such as recording, search or calendar access?
- Where does the information in an answer come from: the model’s training, a document supplied by the user, or a live lookup?
- What happens to the text sent in, including whether it is stored or used for further training?
- Who checks the output before it is acted on?
Answers to these questions say more about reliability and privacy than the size of the model or the name of the vendor.
Understanding the model as a very capable text predictor, not a knowledge base or a person, is the most useful mental model for working with it.
Frequently asked questions
What does a large language model actually do?
Show answer
It predicts the next token in a sequence of text, and by repeating that step it produces sentences, summaries, code and answers.
Is a large language model the same as generative AI?
Show answer
Not exactly. Generative AI is the wider category that includes image and audio generators. Large language models are the text-focused kind.
Why do language models get facts wrong?
Show answer
They generate likely text rather than looking up verified facts, so they can state false details fluently. NIST calls this confabulation.
Does the model learn from my conversations?
Show answer
Not by default within a chat. Whether inputs are stored or used for later training depends on the product and its data terms, which are worth reading before sharing anything sensitive.
