Steps for Building a Big Data Team

June 3, 2025 · Baoss
BAOSS IT consulting: steps for building a Big Data team ready for the era of AI agents

BAOSS IT consulting: how to build a data team in 2025, from GenAI euphoria to real production

In June 2015 we published on this blog the “steps for building a Big Data team”. Ten years later, the term “Big Data” has become small. What then required on-premise Hadoop clusters, MapReduce engineers and PhD-holding data scientists is now solved with LLMs in production, autonomous agents and RAG architectures. Thus, deployment takes hours, not months. In fact, at BAOSS IT consulting we see it every week with our clients.

But the paradox persists: 78% of generative AI projects never reach production (Gartner, 2024). In addition, most Spanish companies keep hiring 2018 profiles for 2025 problems. At BAOSS IT consulting we have guided 40+ organizations through this transition. And if you are still unsure who to build with, first check out how to choose an AI technology partner in 2026. This article is not theory. It is, in fact, the playbook we apply with real clients (anonymized data, verifiable metrics).

The real 2025-2026 problem: not technology, but decision architecture

The post-ChatGPT euphoria (November 2022) produced a phenomenon we see in 90% of board meetings: technological FOMO without a quantified business case. For example, committees buy Copilot Enterprise licenses, hire “prompt engineers” and launch RAG pilots over PDFs with no governance. Therefore, the result is clear: average cost €180K/year, negative ROI at 12 months.

  • Skills gap: 65% of the “data scientists” hired in 2023-24 cannot deploy a model with vLLM or Ollama on shared GPU, nor orchestrate agents with LangGraph/CrewAI.
  • Invisible technical debt: In addition, there are pilots without observability (traces, evals, guardrails) that become ungovernable “shadow IT”.
  • Missing data culture: Even the CEO approves GPU budget but does not demand data contracts between domains (real Data Mesh, not PowerPoint).

BAOSS case (retail, €2,400M revenue): They arrived with 3 parallel GenAI pilots (customer service, product categorization, pricing), but 0 in production. The 2-week diagnosis was clear: none had automated evals, CI/CD for prompts, or defined data ownership. Then, the solution: a standardized internal platform (see section 3). As a result, 2/3 in production in 8 weeks, 40% reduction in deployment time, 3.2x ROI at 6 months.

Step 1: AI is a business priority, not an IT priority — and the CEO must own the “North Star Metric”

In 2015 we said “the CEO must lead by example”. In 2025 it is more concrete: the CEO must define and own the north-star metric that AI is going to move. For example, not “adopt AI”, but “reduce churn 15% with an autonomous retention agent in Q3”.

  • Anti-pattern: the CTO buys GPUs, the CFO sees cost, the CMO wants a chatbot and the CISO blocks everything. Meanwhile, nobody answers: “which business metric are we moving, and over what horizon?”.
  • BAOSS pattern: first, AI Opportunity Mapping workshops (2 days, C-suite + domain owners). Then, the output is a backlog prioritized by value/risk/feasibility with quarterly OKRs tied to variable compensation.

In fact, the internal BAOSS metric confirms it: clients that complete this workshop go from 0 to 2.3 cases in production at 6 months (2024 average, n=27). By contrast, those that don’t: 0.4 cases.

Step 2: Platform architecture, not “loose tools” — MCP, RAG evals, and GPU-as-a-Service

2015 cloud computing was “renting servers”. By contrast, today the right abstraction is a unified inference platform exposing:

  • Versioned Model Registry: GPT-4o, Claude 4 Sonnet, Llama-3.1-70B-Instruct (vLLM/Ollama) and embeddings (BGE-M3, E5-mistral). In addition, with automatic fallback and cost tracking per call.
  • RAG Engine with continuous evals: Semantic chunking, reranking (Cohere Rerank v3.5), retrieval evaluated nightly (RAGAS + business metrics). Because without evals, there is no production.
  • Agent Runtime (LangGraph/CrewAI/AutoGen): Stateful orchestration, native human-in-the-loop and observability (LangSmith / Arize / Phoenix).
  • MCP (Model Context Protocol) servers: Standardized connectors to ERP, CRM, data lake and internal APIs. This eliminates the spaghetti of ad-hoc integrations.
  • Guardrails & PII: Presidio / Microsoft Presidio / LlamaGuard 3 in a sidecar, immutable logging for audit (AI Act ready). Because mastering the risk of AI agents is a precondition for deploying them in production.

2025 build vs buy decision: Therefore, don’t build your own Kubernetes for LLMs. Use GPU-as-a-Service (RunPod, Lambda Labs, Fluidstack, or public cloud with spot A100/H100 instances). Then, add a managed platform (Databricks Mosaic AI, Azure AI Studio, Vertex AI) or go self-hosted with vLLM + Kubernetes (KServe/Knative) if you have sustained scale >500 req/s.

BAOSS case (mid-size bank, €800M assets): first, migration of 12 scattered pilots (direct OpenAI API, no evals) to a unified vLLM + LangGraph + MCP platform. The investment was €220K (infra + 3 months consulting). Then the savings came: 62% inference cost savings vs closed API, 0 security incidents, 3 agents in production in 10 weeks (complaint classification, KYC assistant, risk report generator).

Step 3: The 2025 team — hybrid profiles, not “unicorn data scientists”

The 2015-2024 mistake: looking for a “data scientist who knows everything”. That is why, in 2025, specialization is mandatory. The minimum viable team (3-5 people) to operate the platform above:

  • AI Platform Engineer (1-2): Kubernetes, vLLM/Ollama, CI/CD for prompts (DVC/MLflow + PromptLayer), GPU scheduling and cost optimization. Profile: senior DevOps/MLOps, not a PhD.
  • Agent & RAG Engineer (1-2): LangGraph/CrewAI/AutoGen, chunking strategies, reranking, evals (RAGAS, custom business metrics), guardrails. Profile: software engineer + applied NLP.
  • AI Product Owner / Domain Translator (1): Owns the backlog, defines business evals, manages stakeholders, prioritizes by ROI. Profile: ex-consultant / technical PM / domain expert.
  • Data Steward / Governance (0.5-1, shared): Data contracts, quality, lineage and AI Act compliance. That is, not a DBA: they are the “product owner of the data”.

What you do NOT need to hire yet: pure prompt engineers (a skill that is being democratized), ML researchers (unless pure R&D) or classic data scientists without production engineering.

In addition, the BAOSS metric: teams built with this model reach their first agent in production in 6-8 weeks (vs 6-12 months with the traditional model). Then, the average salary cost: €380K/year (Madrid/BCN, 2025) vs €650K/year for a 2023 “unicorn” team.

Step 4: Lightweight governance — AI Act, evals as tests, and “data contracts” between domains

The EU AI Regulation (full application August 2026) requires several things: a register of high-risk systems, risk management, data quality, human oversight and technical documentation. It is not optional. Nevertheless, there is good news: the Step 2 architecture already generates 80% of the evidence (logs, evals, guardrails, versioning). For example, to see how this translates into practice, read our lesson on AI agent governance.

  • Evals = Automated tests: every prompt/agent PR goes through the pipeline. First, unit tests (structure). Then, integration tests (RAGAS: faithfulness, answer_relevancy, context_precision). Finally, business tests (domain metric: % resolutions without escalation, NPS, conversion). No merge if it doesn’t pass.
  • Data Contracts (real Data Mesh): each domain publishes a schema (Avro/Protobuf), freshness SLA, owner and semantics. The agents consume contracts, not raw tables. This breaks the coupling.
  • Unified observability: traces (LangSmith/Phoenix), business metrics (Grafana), costs (FinOps) and alerts (drift, latency, cost/call > threshold).

BAOSS case (insurance, €1,200M premiums): nightly evals implementation + data contracts between claims, underwriting and actuarial. Thus, the result: early detection of RAG degradation (faithfulness -12% in 3 days) → automatic rollback → 0 customer impact. AI Act audit: technical documentation 90% auto-generated from the pipeline.

Step 5: A culture of “eval-driven development” — from PoC to product

The biggest cultural change: nobody deploys to production without evals that measure business value. That is, not “accuracy”, but “30% reduction in complaint handling time”. Therefore, this requires:

  • First, define a business proxy metric before writing a single line of code.
  • Then, build a representative evaluation dataset (minimum 200-500 real cases, updated weekly).
  • Next, automate A/B comparison (champion/challenger) in staging with shadow traffic.
  • Finally, celebrate “kill your darlings”: if the agent doesn’t beat the human baseline + cost, it gets switched off. So, no drama.

At BAOSS we run a biweekly AI Review Board (30 min). In each session, the PO presents metrics, the technical team shows the eval diff and the decision is one of three: scale / iterate / kill. Healthy kill rate: 35-40% of initiatives, which prevents zombie projects.

Executive summary: 2025 checklist for your steering committee

Area Key question Red flag BAOSS action
Strategy Which 3 business metrics will AI move in 12 months? “We want to use AI” with no OKR AI Opportunity Mapping (2 days)
Platform Do we have unified inference infra with evals, guardrails, MCP? 12 direct OpenAI APIs, 0 evals vLLM/LangGraph/MCP platform (8-12 wks)
Team Do we have an AI Platform Engineer + Agent Engineer + AI PO? Only “data scientists” without prod skills Upskilling + 60-day hiring plan
Governance Do agents pass business eval pipelines before merge? Manual deploy, no tests CI/CD + RAGAS + business evals
Culture Are projects that don’t beat the baseline in staging killed? Zombie pilots >6 months Biweekly AI Review Board

Aggregated BAOSS 2024 data (n=31 production clients):

  • Time-to-first-value: 8.2 weeks (median) from kickoff to the first agent in production already generating a business metric.
  • Total platform + team cost first year: €420K (P25) – €680K (P7

    Passionate about data and want to grow on projects that matter? At BAOSS we are looking for data, AI, cloud, development and cybersecurity profiles to work with leading banking, retail, healthcare and public-sector clients, alongside partners like IBM, Red Hat and Google. Work with us and send us your CV.

    Prefer to talk to us directly? Contact the BAOSS team and tell us about your challenge.