Sr. AI Engineer

ABOUT THE ROLE: Rightway helps members get the right care at the right time and price. Hundreds of thousands of the requests we handle every year involve questions like: What is the status of my prior authorization? What will this drug cost? How does my benefit apply? Answering these questions well requires getting the right data from the right systems, understanding the context, and communicating the answer clearly. Today, much of that work is done manually. We are building Agent OS, our platform for creating and operating AI-powered workflows across Rightway. It provides the foundation for connecting AI to our systems, evaluating its performance, and running it safely and reliably in production. This is an early, foundational platform build. You will be one of a small number of engineers shaping the architecture and engineering patterns behind how AI operates across Rightway. The hard problems here are not just about model or prompt quality. They are about building systems whose correctness can be measured, whose access to sensitive data can be controlled, and whose architecture can support many workflows without becoming fragile or overly specialized. WHAT YOU’LL DO: Build the core platform for running AI-powered workflows reliably in production. Create safe, well-defined ways for AI systems to interact with Rightway's existing services and data. Design the authorization and data-access patterns required to operate AI in a healthcare environment. Build evaluation infrastructure that helps us measure quality, compare changes, catch regressions, and make informed decisions about what is ready for production. Develop the observability, tracing, and auditability needed to understand how AI systems behave in production. Design operational controls that allow us to deploy AI workflows safely and evolve them over time. Establish reusable platform patterns that other engineers can use to build AI-powered capabilities. Make thoughtful build-versus-buy decisions as the ecosystem around agents, models, tools, and evaluation continues to evolve. Partner closely with product, security, compliance, data, and engineering teams to turn new AI capabilities into production systems. Help shape the technical direction of Rightway's broader AI platform as both the technology and our use cases evolve. WHO YOU ARE: 5+ years of experience building and operating production backend, infrastructure, or platform systems. Experience shipping LLM-based or AI-powered systems to production, beyond prototypes and demos. Strong backend and distributed systems fundamentals, including APIs, typed service contracts, service boundaries, failure handling, schema evolution, scalability, and operating against systems you do not fully control. A strong understanding of authorization, data boundaries, and the engineering controls required when AI systems operate on sensitive information. Exercises good judgment about where models should be used, where deterministic software is the better choice, and where additional safeguards or human intervention are required. Experience designing abstractions and platform capabilities that can support multiple use cases without overfitting to the first one. Comfort working in ambiguity and turning emerging problems into clear technical decisions, documented patterns, and working systems. Strong written and verbal communication across engineering, product, security, compliance, and data teams. A bias toward pragmatism - shipping useful systems, learning from real-world usage, and iterating. We also expect depth in several of the following: Agent-based systems, including agent orchestration, tool/function calling, MCP, structured outputs, context management, or similar patterns for connecting models to external systems and data. AI evaluation systems, including offline evals, regression testing, version comparison, and measuring whether changes improve or degrade system behavior. AI observability and tracing, including understandi…