AI Compliance for a company using generative models in production is not a question of how often an LLM refuses to execute unlawful instructions. It depends on many other factors, primarily on how the system around the model is built. Current European Artificial Intelligence regulations assign legal responsibility to whoever integrates a model into a specific system – the deployer – and not to whoever trains and distributes the model. A distinction that changes how corporate governance and compliance must be designed.
Compliance Benchmarks on AI Models
In recent months, a study conducted by Aithos, a Dutch non-profit research foundation, was widely circulated after testing 12 major AI models on simulated scenarios of real-world work environments. The framework is called LARA — Legal Assessment for Real-world Agents — and measures how many times a model executes instructions that would require violating European AI rules or the GDPR. The results showed compliance rates that vary significantly among the tested models, with the best offering no resistance in 46% of scenarios and the worst in 93%.
The study touches on a concrete issue: AI systems in production can produce output or execute actions that are problematic from a regulatory perspective. An issue that every company bringing AI into its operational workflows must address.
However, what the framework measures must be clarified: what LARA evaluates is the model’s propensity to refuse a task in ad-hoc constructed scenarios. The score is calculated by the same AI system being evaluated, without a peer-review process, and does not correspond to the compliance posture of a company using that model in production. The reason is structural and depends on how European legislation distributes responsibilities among the different entities in the supply chain.
Providers and Deployers: How AI Legislation Distributes Obligations
To understand where compliance responsibility lies, one must start from a distinction that European legislation introduces with precision: that between GPAI model providers and deployers.
GPAI (General Purpose AI) providers are the companies that train and distribute foundation language models: OpenAI, Anthropic, Google, Meta, Mistral, etc. Generalist systems, trained on massive data corpora, capable of performing a wide range of tasks across different domains. It is precisely this generality that makes them practically impossible to govern a priori for every application context in which they will end up.
For these entities, the legislation sets obligations that revolve around transparency and documentation: produce and keep updated technical documentation on the model (architecture, training data, computational resources); provide deployers with the necessary information to understand capabilities and limitations; adopt a policy respecting European copyright law; publish a summary of the content used for training. For models with systemic risk, those trained with very high computational thresholds, obligations for risk assessment and management are added.
None of these obligations require the model to refuse illegal tasks, and this is a coherent choice: a generalist model lacks the context to distinguish an unlawful task from a legitimate one in the possible use cases where it will be employed. This assessment belongs to whoever knows the specific application context.
Deployer Compliance Responsibility: Obligations for High-Risk Systems
The deployer is the company that takes a model and integrates it into a specific system for a determined use (for example, a customer service agent or a document analysis tool in financial or legal contexts). With this choice, the deployer assumes legal responsibility for that system’s behavior within its operational context.
The obligations that regulations place on deployers of high-risk systems are explicit: guarantee human oversight entrusted to personnel with adequate skills and authority; monitor system operations and suspend it in case of risks; retain generated logs for a minimum of six months; inform workers when the system is used within their activities.
The difference with the GPAI provider is structural: the provider knows the model, the deployer knows the use case. The AI Act distributed responsibilities accordingly. A model that executes a potentially unlawful instruction in a simulated scenario is not necessarily a non-compliant model: it is a model that lacks the context to evaluate that specific instruction in the operational environment where it is used. Building that context and the appropriate control mechanisms falls on the deployer.
If an AI agent for customer service produces problematic outputs, legal responsibility lies with the company that configured and put that system into production, regardless of the underlying model.
Transparency, Oversight, and Traceability
Provisions on transparency, human oversight, and traceability of automated decisions are now fully operational, as are the enforcement powers of the European Commission.
For providers, this means technical documentation must already be ready and available upon request by authorities. Fines reach up to 3% of global annual turnover or 15 million euros in case of non-compliance.
For deployers, obligations depend on the system’s classification. High-risk systems, such as those in HR, credit scoring, critical infrastructure, and judicial processes, carry the most stringent requirements: documentable human oversight, auditable logging, and conformity assessment before deployment. But even outside this category, transparency rules mandate declaring that one is interacting with an AI system when it is not obvious to the user.
Mapping one’s AI systems remains the prerequisite of any compliance strategy: knowing which systems are in use, in what operational context, with what degree of autonomy, and who has effective oversight over them. Without this visibility, any assessment of regulatory risk remains incomplete.
Stack-Level AI Compliance: Guardrails, Logging, and Access Controls
The practical consequence of this distinction is that building a solid compliance posture is a stack engineering problem, not just a model selection issue. Relying on model behavior as the primary line of defense introduces systemic legal risk: models change with updates, their behavior varies with context and prompts, and the company’s liability remains unchanged regardless of what the model does.
The guardrails that matter for compliance are built at the application and infrastructure level: inbound filters that analyze requests before they reach the model; outbound filters on generated responses; structured and auditable logs of every interaction; access controls defining who can use the system and with what autonomy; escalation mechanisms to human oversight when the system encounters high-uncertainty scenarios.
For agentic architectures, systems that operate autonomously across complex workflows, call external tools, and self-correct, this becomes even more critical. An agent executing actions on external systems such as CRM, ERP, or databases cannot be governed solely through system prompts: an infrastructure layer is needed that can intercept, log, and block anomalous behaviors before they produce irreversible effects.
The human oversight required by regulations is not a human reading every output: it is the demonstrable ability to intervene in the AI process, stop it, and audit its decisions. This capability must be technically demonstrable through logs, alerts, and override mechanisms, it cannot be a statement of principle in an internal policy.
AI Gateway: A Centralized Governance Layer for Compliance
Enterprise companies bringing AI into multiple workflows simultaneously face governance fragmentation that makes auditing difficult and incident response slow.
The infrastructural response to this problem is a centralized layer between corporate applications and LLM providers. This layer applies inbound and outbound guardrails cross-functionally across all projects without requiring code changes to individual applications. Structured logging for every call, with metadata on model, latency, user, and cost, produces the auditable trail necessary to respond to an inspection or reconstruct system behavior at any given time.
Intelligent multi-model routing allows assigning each request to the most appropriate model based on compliance criteria: isolating certain use cases on on-premise models or geographically appropriate deployments is a decision managed at the gateway level without touching application code.
Decoupling from vendor lock-in means that if provider terms change in policy, pricing, or availability, the transition requires no architectural rewrites.
Radicalbit AI Gateway is designed for precisely this scenario: centralizing LLM call governance, making AI system spending and behavior visible and controllable, and providing the infrastructure layer upon which to build an auditable compliance posture.
Conclusion
Compliance with European AI regulations is not solved by selecting the safest model or relying on refusal behaviors tested in simulated benchmarks. These benchmarks are useful for understanding how a model behaves, but they measure a different dimension than conformity: compliance does not reside in the model; it lives in the system the deployer builds around it.
This shifts the challenge from procurement to engineering. No single model, once chosen, shields a company: what exists is a well-designed stack, with guardrails at the right level, logs that withstand an audit, and human oversight that can actually stop the system when necessary. It is less visible work than choosing the highest-performing model, but it is what regulations assign to deployer responsibility.
Once this framework is accepted, AI Compliance stops being a moving target tied to provider updates and becomes a property of your system: something you can design, measure, and demonstrate.
To learn how Radicalbit AI Gateway can support your governance strategy, visit our dedicated page or book a demo with our team.
