Loading
A large language model (LLM) is a type of foundation model trained on massive volumes of text using a transformer neural network architecture, giving it the ability to predict, generate, and manipulate human language for tasks ranging from conversation and summarization to reasoning and code generation. LLMs are the engine behind most of today's generative AI products, chatbots, coding assistants, search copilots, and are typically the underlying model rather than the branded product itself. Because a single LLM can be embedded in thousands of downstream applications, EU regulators treat it as a distinct regulatory object: a "general-purpose AI (GPAI) model" with obligations that attach to the model provider, separate from whatever product is built on top of it. For governance teams, LLMs carry a risk profile that traditional software risk frameworks were not built to catch, prompt injection, sensitive-data leakage, hallucinated output, and murky training-data provenance. Two frameworks now anchor most LLM governance programs: the OWASP Top 10 for LLM Applications for technical and security risk, and the EU AI Act's GPAI regime (Chapter V, Articles 51-56) for regulatory obligations on model providers.
Run the free AI Health CheckLarge Language Model (LLM), a foundation model specifically trained on a very large corpus of text to predict and generate human language, capable of conversation, reasoning, code generation, and a wide range of language-related tasks.
LLMs are the most economically significant AI development of the past five years. Frontier LLMs (GPT-5.5, Claude 4 Opus, Gemini 2.5, etc.) are now used in enterprise productivity, customer service, software engineering, and research. Governance attention has focused on data residency, prompt injection vulnerabilities (OWASP LLM Top 10), training data IP exposure, hallucination, and, for the largest models, systemic risk classification under the EU AI Act.
Source: EU AI Act, Art. 3(63) & Art. 53 (GPAI classification, applies to all LLMs); Art. 51 & Art. 55 (systemic-risk obligations only); OWASP Gen AI Security Project, LLM Top 10 (2025 edition)
LLMs are built on the transformer architecture, first described by Google researchers in 2017. During pretraining, the model is shown enormous quantities of text and learns, token by token, to predict what comes next given everything that came before. The model's "knowledge" is encoded as billions of numerical weights (parameters) rather than as retrievable facts, there is no lookup table or database being queried.
After pretraining, most providers apply further stages such as instruction tuning and reinforcement learning from human feedback (RLHF) to steer the model toward outputs humans rate as helpful, honest, and safe. This is what turns a raw text-prediction engine into something that behaves like a conversational assistant.
At inference time, an LLM generates one token at a time based on statistical patterns learned in training, not by verifying claims against a source of truth. That is precisely why LLMs can sound fluent and confident while stating something false, a failure mode the field calls hallucination, and why identical prompts can yield different answers depending on decoding settings and randomness.
LLM01: Prompt Injection
Attacker-crafted input causes the model to follow injected instructions instead of the developer's intended task, because LLMs process instructions and untrusted data in the same channel. Holds the top spot for a second consecutive edition.
LLM02: Sensitive Information Disclosure
The model memorizes and can reproduce fragments of training data or context, PII, proprietary business data, credentials, often extractable through targeted queries. Jumped from sixth to second place in the 2025 edition.
LLM03: Supply Chain
Risk introduced through third-party models, datasets, fine-tunes, plugins, or adapters (e.g., LoRA) with unclear provenance, licensing, or tampering.
LLM04: Data and Model Poisoning
Manipulation of pretraining, fine-tuning, or embedding data to introduce vulnerabilities, backdoors, or biases that surface later in production.
LLM05: Improper Output Handling
Downstream systems trust LLM output without validation or sanitization, enabling injection-style attacks (e.g., XSS, SSRF, code execution) in whatever consumes the output.
LLM06: Excessive Agency
An LLM-based system is granted more functionality, permissions, or autonomy than its task requires, so a manipulated or erroneous output can trigger real-world actions.
LLM07: System Prompt Leakage
Confidential instructions, credentials, or business logic embedded in a system prompt are extracted by a user, exposing information the developer assumed was hidden.
LLM08: Vector and Embedding Weaknesses
Vulnerabilities in how vectors and embeddings are generated, stored, or retrieved in retrieval-augmented generation (RAG) pipelines, enabling injection or data leakage through the retrieval layer.
LLM09: Misinformation
Confident but false or misleading output, including hallucination, that users act on because the model's fluent delivery masks the lack of a verified factual basis. New in the 2025 edition.
LLM10: Unbounded Consumption
Excessive or uncontrolled resource usage against the model, through volume, complexity, or replication attacks, leading to denial of service, runaway cost, or model extraction. New in the 2025 edition.
The AI Act does not define "large language model" directly; it regulates them as a subset of "general-purpose AI (GPAI) models," defined in Article 3(63) by capability rather than technique or compute: a model that "displays significant generality" and can "competently perform a wide range of distinct tasks." In practice, essentially every LLM with broad instruction-following ability meets this bar and falls under Chapter V (Articles 51-56), regardless of its size.
Article 53 sets baseline obligations for every GPAI model provider: maintain technical documentation, give downstream deployers the information they need to understand and use the model responsibly, adopt a policy to comply with EU copyright law, and publish a sufficiently detailed public summary of the content used to train the model.
A smaller subset carries heavier obligations. Article 51 classifies a GPAI model as having "systemic risk" if it has high-impact capabilities, with a rebuttable presumption of high impact once cumulative training compute exceeds 10^25 floating-point operations (FLOPs), or if the European Commission designates it as such. Providers of these systemic-risk models face Article 55 duties on top of Article 53: adversarial testing and model evaluation, systemic-risk assessment and mitigation at EU level, mandatory serious-incident reporting to the AI Office, and adequate cybersecurity for the model and its infrastructure.
Timing matters for compliance planning: the AI Act entered into force on 1 August 2024, but the GPAI provisions (Articles 53 and 55) only became applicable on 2 August 2025. Providers of models already on the market before that date have until 2 August 2027 to come into compliance, and the AI Office's full enforcement powers (information requests, recalls, mitigation orders, fines) phase in by 2 August 2026.
Training-data provenance and copyright exposure
Unclear rights over scraped or licensed training text create IP and copyright litigation risk for model providers and, in some cases, downstream deployers.
Data residency and cross-border transfer
Prompts and outputs sent to a hosted LLM API may leave the organization's jurisdiction, raising data-residency and privacy-law questions distinct from model security.
Shadow AI usage
Employees pasting sensitive data into consumer-facing LLM chat interfaces outside any sanctioned procurement or DPA is one of the most common real-world LLM incidents reported by security teams.
Vendor and model risk management
Treating an LLM API as a critical third-party dependency, with change monitoring, fallback plans, and contractual assurances on data use and retraining, rather than a plug-and-play utility.
Audit logging and explainability gaps
Because LLM output is probabilistic rather than rule-based, reconstructing why a specific output was produced is harder than with traditional software, complicating audit trails and incident post-mortems.
What is a large language model in plain terms?
It is a machine learning model trained on huge amounts of text that learns to predict the next word in a sequence, which turns out to be enough to let it write, summarize, translate, code, and hold conversations. It generates language from learned statistical patterns, not from looking facts up in a database.
Is every generative AI tool an LLM?
No. Generative AI is the broader category, it includes image, audio, and video generators alongside text models. An LLM is specifically a text-focused foundation model; the chat product a user interacts with is usually an application layer built around one or more LLMs, often combined with retrieval, tools, or agents.
Does the EU AI Act classify all LLMs as "high-risk"?
No, and this is a common confusion. The AI Act's "high-risk" category (Annex III) is about specific use cases like employment screening or credit scoring, not about LLMs as a technology. LLMs are instead regulated as "general-purpose AI models" under Chapter V, with a smaller subset also classified as carrying "systemic risk" under Article 51.
What's the difference between a GPAI model and a GPAI model with systemic risk under the AI Act?
Every LLM with broad task generality is a GPAI model and falls under Article 53's baseline obligations (documentation, copyright policy, training-data summary). A GPAI model additionally carries "systemic risk" under Article 51 if it has high-impact capabilities, presumed once cumulative training compute exceeds 10^25 FLOPs, or if the Commission designates it, which triggers the heavier Article 55 obligations: adversarial testing, EU-level risk mitigation, and incident reporting.
What is the OWASP Top 10 for LLM Applications, and who should use it?
It is a community-developed list of the ten most critical security risks specific to LLM-powered applications, published by the OWASP Gen AI Security Project and most recently updated for 2025 (covering risks like prompt injection, sensitive information disclosure, and unbounded consumption). It's aimed at developers, security teams, and auditors building or reviewing LLM applications, similar in spirit to the long-standing OWASP Top 10 for web applications.
If my company just calls a third-party LLM's API, do the EU AI Act's GPAI obligations apply to us?
The Article 53/55 GPAI obligations fall on the model provider (the organization that develops and places the GPAI model on the market), not on every company that calls its API. Deployers building products on top of a third-party LLM instead need to review the provider's documentation and may have separate obligations if their specific application falls into the AI Act's high-risk categories.
Last reviewed July 2026
This page is general information about What Is a Large Language Model (LLM)?, not legal, regulatory, or professional advice, and does not capture every nuance or exception. Requirements change and can be fact-specific. Always verify against primary sources and your own qualified legal counsel before relying on it.