Interview with La Voz Empresarial, El Mundo Newspaper

April 2, 2026 · Baoss
BAOSS IT consulting: from experimental AI to production with agentic RAG

From experimental AI to production: how 12 Spanish companies cut costs by 40% in 2026

In January 2017, our CIO Iñaki Ladrero explained in La Voz Empresarial of El Mundo why digital transformation was not optional. Eight years later, the conversation has changed radically. Moreover, it is no longer about “going digital”, but about putting large language models (LLMs) into production without the inference bill or hallucination sinking the project. And that is exactly the turf of BAOSS IT consulting.

In fact, according to McKinsey’s State of AI 2025 report, 78% of Spanish companies have already piloted GenAI. However, only 12% have scaled to production with positive ROI. Likewise, the bottleneck is no longer the model —GPT-4o, Claude 4 Sonnet or Llama 3.1 405B solve most tasks— but the data architecture, continuous evaluation and governance (how to control what your company is already deploying).

The real 2026-2027 problem: pilots that never reach production

In fact, at BAOSS we have audited 47 GenAI projects over the last year (banking, insurance, logistics, the public sector). The pattern repeats:

  • Also, naïve RAG: fixed chunking, generic embeddings (text-embedding-3-large), no reranking → accuracy < 60% on complex queries.
  • Agents without observability: LangGraph / CrewAI / AutoGen orchestrating flows. However, with no traces of latency, cost per token or per-tool failure rates.
  • Non-existent evaluation: “it works on my machine” → 0 regression tests, 0 quality metrics (faithfulness, answer_relevancy, context_precision).
  • Uncontrolled cost: GPT-4o calls with no semantic caching, no routing to local models (vLLM + Llama 3.1 70B on own GPU) for simple tasks.
  • Zero governance: no PII detection, no output guardrails, no audit of automated decisions (EU AI Act requirement, in force August 2026). Therefore, review the lesson from the Hugging Face case for your company.

The result: projects stuck in an “eternal PoC”, frustrated teams, management demanding ROI and vendors selling “magic platforms” that solve none of the above.

BAOSS IT consulting: an evaluated agentic RAG architecture, not a “chatbot with PDFs”

Ultimately, since 2023 we have applied our own 4-layer framework that turns pilots into production services. We have also validated it with 12 clients using auditable metrics:

Layer 2026-2027 technologies Key decisions
Ingestion & Indexing Unstructured.io + LlamaParse, semantic chunking (propositions), bge-m3 / e5-mistral-7b-instruct embeddings, bge-reranker-v2-m3 reranking, vector DB: Qdrant / Milvus / PGVector Adaptive chunking by document type (tables, code, legal), enriched metadata (section, version, owner), BM25 + dense hybrid search
Agent Orchestration LangGraph (stateful, cycles), CrewAI (multi-agent roles), AutoGen (conversation), MCP (Model Context Protocol) for external tools Specialised agents: Retriever → Reranker → Synthesizer → Critic → Guardrail. Persistent state in Postgres. Human-in-the-loop at critical steps.
Continuous Evaluation RAGAS 2.0, DeepEval, Opik / Langfuse / Phoenix (Arize) for observability, golden datasets (200+ Q&A per domain), CI/CD with GitHub Actions Mandatory metrics per release: faithfulness > 0.85, answer_relevancy > 0.80, context_precision > 0.75, p95 latency < 3s, cost < €0.008/query
Inference & Cost Intelligent routing: GPT-4o / Claude 4 Sonnet (complex reasoning) → vLLM + Llama 3.1 70B/8B (classification, extraction, summarisation) on own H100/A100 GPUs. Ollama for local dev. Semantic cache (GPTCache / Redis + embeddings) Target: > 65% of traffic to local models. Cache hit rate > 30%. Automatic fallback if latency > threshold.

Real anonymised cases: 2026 production metrics

Case 1: Mid-size bank — Internal regulatory assistant (agentic RAG + AI Act compliance)

Challenge: 12,000 regulatory documents (CNMV, BdE, ECB, AI Act, DORA), 300 queries/day from compliance, legal and risk. Moreover, a previous pilot with generic embeddings: 52% accuracy, 0% auditability.

  • Solution: multi-agent RAG with LangGraph + propositional chunking + bge-reranker-v2-m3 reranking + PII guardrails (Presidio) + full traceability in Langfuse.
  • Routing: 78% of queries resolved with Llama 3.1 70B (vLLM, 4x H100), 22% escalated to GPT-4o (case-law interpretation).
  • Results at 6 months: Accuracy 89% (RAGAS faithfulness 0.91), p95 latency 2.1s, cost €0.0042/query (82% reduction vs pure GPT-4o), 100% traceable audit for the AI Act. In fact, ROI 3.4x in 8 months (savings of 4.2 senior FTEs + avoided fines).

Case 2: International logistics — Last-mile incident resolution agent (CrewAI multi-agent + MCP)

Challenge: 1,800 incidents/day (delays, damage, wrong addresses). Average resolution time: 47 min. Likewise, human agents saturated, CSAT 3.2/5.

  • Solution: CrewAI with 5 specialised agents (Classifier → RouteOptimizer → CustomerComms → CarrierAPI → Closer). Tools via MCP: internal TMS, Google Maps API, WhatsApp Business, CRM. Continuous evaluation with DeepEval.
  • Human-in-the-loop: Only complex escalations (18% of cases). Therefore, a Critic agent validates the response before sending.
  • Results at 4 months: Average resolution time 12 min (-74%), 68% of tickets fully automated, CSAT 4.6/5, cost €0.011/ticket vs €4.80/human ticket. Ultimately, 40% reduction in last-mile operating costs.

Case 3: Public sector — Cross-cutting legislative semantic search (hybrid RAG + continuous evaluation)

Challenge: 2.3M documents (BOE, regional official gazettes, case law, reports). Outdated lexical search. The need: “What obligations do I have as an SME in sector X under the new directive Y?”

  • Solution: Hybrid Qdrant index (dense + sparse BM25), hierarchical chunking (article → paragraph → sentence), cross-encoder reranking, REST API with OAuth2 authentication + rate limiting.
  • Evaluation: Golden dataset of 500 real user queries. CI/CD pipeline: every commit runs RAGAS + semantic regression testing.
  • Results at 3 months: NDCG@10 0.87 (vs 0.41 before), p95 latency 1.4s, 99.2% uptime. Organic adoption: 3,400 unique users/month with no marketing. Also, project referenced in the 2026 Public Administration Digitalisation Plan.

Case 4: Insurance — Assisted underwriting with AutoGen agents + local models (vLLM + Ollama)

Challenge: 15,000 policies/year reviewed manually. In fact, average time 3.2h/policy. Inconsistency between underwriters. Drain of senior talent.

  • Solution: AutoGen with 3 agents (Extractor → RiskAnalyst → PolicyDrafter). Likewise, the Extractor uses Llama 3.1 8B (local Ollama, €0 inference) to extract entities from scanned PDFs. Therefore, the RiskAnalyst uses Llama 3.1 70B (vLLM, own GPU) for scoring. Therefore, the PolicyDrafter uses GPT-4o only for the final wording.
  • Governance: All decisions logged with explanation (SHAP values for numerical features + attention weights for text). Ultimately, mandatory human review for scoring above the risk threshold.
  • Results at 5 months: Average time 48 min/policy (-75%), inter-underwriter consistency Kappa 0.91 (vs 0.67), cost €0.32/policy vs €180/manual policy. Also, ROI 5.1x in 6 months. 2 senior underwriters retained.

What nobody tells you: the 5 mistakes that kill GenAI projects in 2026

  1. In fact, buying a “RAG platform” before having an evaluation dataset. Ultimately, without 200+ representative Q&As from your domain, you do not know if it works. Likewise, start with the golden dataset, not the vector DB.
  2. Also, using GPT-4o/Claude 4 for everything. Therefore, classification, extraction, summarisation, routing → local models (Llama 3.1, Qwen 2.5, Nemotron 3 Ultra) on vLLM/Ollama. In fact, 80-95% inference cost savings.
  3. Ultimately, ignoring MCP (Model Context Protocol). Likewise, your agents need real tools (SQL, REST API, filesystem, browser). MCP standardises the connection. Also, LangChain Tools is 2024 legacy.
  4. Therefore, not versioning prompts or traces. In fact, prompt engineering without version control = chaos engineering. Use Langfuse / Opik / Phoenix. Likewise, every prompt change = a PR with regression tests.
  5. Leaving governance for “later”. Therefore, the AI Act (EU) 2026 requires: a register of high-risk systems, data management, transparency, human oversight, accuracy, robustness, cybersecurity. “Later” is already too late.

The BAOSS 2026-2027 reference stack (open source first, cloud agnostic)

Category Primary choice Alternatives evaluated Why
Agent orchestration LangGraph CrewAI, AutoGen, Semantic Kernel Stateful, cycles, checkpointing, visual debugging, native deployment (LangGraph Platform)
Observability / Eval Langfuse + RAGAS 2.0 Opik, Phoenix, DeepEval, MLflow Open source, self-hosted, full traces, datasets, experiment tracking, alerts
Local inference vLLM (production) / Ollama (dev) TGI, llama.cpp, TensorRT-LLM Maximum throughput, PagedAttention, OpenAI API compatible, multi-GPU/TPU support
Vector DB Qdrant Milvus, PGVector, Weaviate, Chroma Rust, native payload filtering, hybrid search, clustering, multitenancy, low RAM usage
Embeddings / Rerank bge-m3 / bge-reranker-v2-m3 e5-mistral-7b, jina-embeddings-v3, voyage-3 Multilingual, long context (8k)

Prefer to talk to us directly? Contact the BAOSS team and tell us about your challenge.