Software Engineer — AI Infrastructure

About Snorkel At Snorkel, we believe meaningful AI doesn’t start with the model, it starts with the data. We’re on a mission to help enterprises transform expert knowledge into specialized AI at scale. The AI landscape has gone through incredible changes since 2015, when Snorkel started as a research project in the Stanford AI Lab, to the generative AI breakthroughs of today. But one thing has remained constant: the data you use to build AI is the key to achieving differentiation, high performance, and production-ready systems. We work with some of the world’s largest organizations to empower scientists, engineers, financial experts, product creators, journalists, and more to build custom AI with their data faster than ever before. Excited to help us redefine how AI is built? Apply to be the newest Snorkeler! The AI Infrastructure team within Snorkel owns the platform layer that powers everything at Snorkel — and that layer is evolving. Beyond the data platform (pipelines, access layers, event systems, governance, compute), we are now building Snorkel's AI infrastructure: the foundational agentic stack that will let every team at Snorkel build, run, and govern AI agents, and the LLM efficiency layer that keeps AI costs under control as usage scales. We are a small team with a large surface area, in the middle of two foundational shifts: moving from a single-database data path to a multi-source, event-driven platform (Postgres/RDS, Snowflake, S3, metrics platform), and moving from bespoke, one-off agent implementations to a shared, governed, agentic-first platform. The decisions being made now will define how data and agents operate at Snorkel for years. You will be making them. You'll also shape our AI-native development workflow, contribute to modernizing CI/CD (Buildkite, GitHub Actions), and integrate AI SRE tooling. Your work will directly accelerate developer velocity, reliability, and product quality across the company. What You'll Do Build the Agentic Factory Foundation. Design and build the opinionated agentic stack that FDEs, delivery, and product engineering teams will use to scaffold agent workflows: a common orchestration layer for defining and running agents, a memory layer (short-term working memory and long-term persistence), a context graph / knowledge layer grounding agents in project, spec, and platform state, an MCP gateway providing secured, governed, auditable tool access, an evaluation layer testing agents against trace-level and outcome-level criteria, and an observability layer for traces, feedback, metrics, and cost. Start pragmatic — leverage existing building blocks to ship real use cases (self-healing agents, spec-to-eval pipelines, debugging agents) before going deep on every layer. Build the LLM cost and efficiency platform. LLM token spend is growing with the business, and controlling it is a first-class engineering problem. Build the queuing and throttling layer that governs synchronous LLM requests, async and batched call paths for workloads that don't need real-time responses, token optimization (prompt compression, caching, model routing), token usage metering and attribution so teams can see what they spend and why, and world-model approaches that let agents reuse knowledge instead of re-querying models. Build the foundational data access layer and SDKs. Design the shared access library that Platform, Packaging, and Dataset API teams use to read from and write to multiple data sources (Snowflake, S3, RDS) — abstracting entity data from the specific infrastructure underneath so we can scale and improve infrastructure without every product team absorbing the change. Interfaces provide built-in auth, RBAC enforcement, pagination, and query governance. Design and implement event-driven data flows using event brokers, CDC connectors, schema registry, event routing, and dead letter queues. Make sure events flow reliably and failures are visible and recoverable. Build governance, lineage, and audi…