Senior Forward Deployed Engineer I (AI Inference)

Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here. We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world. Position Overview We are looking for a Senior Forward Deployed Engineer I (FDE) who is passionate about operationalizing and collaborating closely with strategic AI enterprises and high-growth startups. As AI-native startups scale, serving large generative AI models at low latency without blowing up infrastructure budgets is their biggest hurdle. That’s where you come in. As a Sr. Forward Deployed AI Inference Engineer (FDE) I , you won't just optimize benchmarks in a lab—you will embed directly with high-growth AI founders to solve high-throughput, cluster-scale LLM serving challenges on DigitalOcean’s GPU cloud. You operate at the intersection of distributed systems, specialized AI hardware, and real-world customer impact. You will own the full lifecycle of high-performance LLM deployment, turning raw GPU compute into hyper-efficient, resilient serving infrastructure for our high-value customers. You will spend your time architecting distributed inference systems, profiling network bottlenecks, debugging KV-cache locality issues, and shipping code that directly reduces time-to-first-token (TTFT) and time-per-output-token (TPOT) for top AI teams. What You’ll Do Technical Leadership: Act as an AI Inference lead on the FDE team, driving the end-to-end design, development, and delivery of critical AI workloads leveraging large generative AI models. Design & Scale Distributed AI Inference Systems: Architect and deploy production-grade, multi-tenant LLM inference engines using Kubernetes-native frameworks like llm-d, NVIDIA Dynamo, Ray Serve, vLLM, and SGLang. Lead Forward Deployed Engagement: Embed alongside external tech leads to debug latency spikes, profile GPU memory utilization, and refactor inference code for high-concurrency production workloads. You are the bridge between our customers and our internal AI infrastructure teams. Optimize at Cluster Scale: Solve the distributed-systems problems unique to LLM serving. You’ll implement strategies for prefill/decode disaggregation, KV-cache-aware routing, tiered prefix caching, and wide expert parallelism for MoE models. Drive Hardware Efficiency: Guide customers on compute efficiency utilizing techniques like tensor/data parallelism, continuous batching, and quantization (FP8/FP4) to inflate tokens-per-second per dollar. Build Internal Tooling & Upstream Value: Translate customer edge cases into reusable internal blueprints and contribute performance fixes directly back to open-source inference ecosystems (vLLM, llm-d) on behalf of DigitalOcean. Travel & Collaboration Requirements: Ability to travel up to 30% for customer engagements, strategic workshops, conferences, and internal collaboration. Ability to consistently overlap with North American business hours, including availability until at least noon Eastern Time, to collaborate effectively with customers, Product, Engineering, and go-to-market teams. What You’ll Add to DigitalOcean Deep Distributed Inference Fluency: 6+ years in AI/ML systems, with a deep understanding of why cluster-scale serving is hard (e.g., partitioning KV-cache across workers, fast cross-pod KV transfer, and inference-aware load balancing). Framework Mastery: Hands-on experience with vLLM, llm-d, SGLang, TensorRT-LLM, or Modular MAX, including a solid grasp of internals like continuous batching and paged attention. Code & Architecture Proficiency: Expert-level proficiency in Python or GoLang , familiarity with gRPC, and experience running critical services on Kubernet…