Senior AI Engineer

The Job in short The Data Enablement team is here to enable every team in the organisation with their data needs. Our job starts the moment that data enters our platform and ends when it reaches whoever needs it. We are a small, senior team of data and AI engineers, working across customer data, product knowledge, and the data the organisation runs on. We bring data in, reconcile it into one version people can rely on, and make it available to the right audience. Increasingly that audience is agents as well as people, so everything we build has to work for both. Two things must hold at every step: that only the right people can see it, and that we can prove it is correct. The second is where this role sits. As a Senior AI Engineer you own the promise that what we serve is right. You are the leading voice on assurance and evaluation across everything the team delivers, from the source through to the answer someone reads. Product calls this assurance. Engineers call it evaluation. It is the same job. In practice it is two kinds of checks and one framework. Deterministic checks on structure, completeness and freshness. Probabilistic checks on whether a generated answer is grounded and correct, including the judges and the golden datasets behind them. Today those checks exist per project and per tool, built by whoever needed them. You turn them into one configurable framework, reusable across sources, reporting into a single view of platform health. Product knowledge is where it starts, because it is the one journey we run end to end ourselves, from ingesting the source to the answer a customer reads. That framework is the first phase, not the ceiling. Once it is running and trusted, the same judgement applies to everything else we build: retrieval quality, agent behaviour, and the systems that serve them. We are hiring for the authority you bring on assurance and evaluation, because it is the capability that decides how far the rest can go. Meet the job The list below describes the AI Engineer discipline at Backbase. In Data Enablement your first focus is the evaluation and assurance responsibilities that follow. The wider discipline opens up once that foundation is in place. Agentic Orchestration: design and implement complex agentic workflows and assistant platforms using the LangChain ecosystem. Advanced Retrieval: develop and optimise RAG and GraphRAG pipelines to give agents deep, contextual domain knowledge. System Design: architect scalable, distributed AI services that integrate into our Kubernetes environments. Agentic Ops (AIOps): implement robust monitoring, tracing and evaluation frameworks (LangSmith, Langfuse, Promptfoo). Skill Integration: build and manage Skills and toolsets for agents, including the Model Context Protocol (MCP). Human-in-the-Loop: design HIL patterns so high-stakes financial decisions remain governed and accurate. Mentorship: provide technical leadership to junior engineers and contribute to internal AI strategy and best practices. Evaluation datasets: Build them from acceptance criteria and expert grounding, split dev and test on different samples so tuning and grading never share data, and freeze golden sets. LLM judges: Run failure-mode analysis on each acceptance criterion, map it to measurable metrics, and write and iterate the judge prompts. Calibration: Rate judges against human ratings and measure agreement as true positive and true negative rate. A judge does not gate a release until its agreement with human raters clears an agreed threshold. Retrieval evaluation: Judge chunk relevance, corpus coverage and freshness separately from answer quality, and run the quality gate that blocks a bad release. How about you A Bachelor's or Master's degree in Computer Science, Data Science or a related field. 5+ years of professional engineering experience, including at least one LLM-based system you took to production. Excellent written and verbal communication skills in English. Agent frameworks:…