Senior Machine Learning Engineer, Inference

Senior Machine Learning Engineer, Inference — Inworld

Remote (US, Canada, UK, Germany, Switzerland, Serbia) · Full-time · $250K–$300K + equity · Mountain View, CA office available

Who We're Looking For

A year ago, reliably working agentic systems and sub-second multimodal inference at scale barely existed. Nobody has a decade of experience here. So we're not screening for a resume template — we're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.

Experience We Find Useful

You don't need all of this. But you need enough to make a case.

• Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
• Model Acceleration. Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
• High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs.
• Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.
• Public work. Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups.
• Full-cycle ownership. You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.
• Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.

Who Thrives Here

• You don't need a roadmap to start walking; you're comfortable picking a direction and building the map as you go.
• You believe engineering isn't finished until it's shipped and stable. You have a bias for impact over purely theoretical optimizations.
• You don't just ship code; you obsess over the why. You're the first to question an architecture if you think there's a better way to solve the core latency or throughput problem.
• You aren't satisfied with "the PM said so." You thrive on deep context and want to understand the fundamental logic behind every decision we make.

Compensation & Benefits

• Base salary $250K–$300K USD, adjusted by location (closer to the top of the range for SF Bay Area/NYC), plus competitive equity.
• Visa transfers supported (OPT, H-1B transfers).
• Relocation assistance available for candidates willing to move to the SF Bay Area.

Interview Process

1. Recruiter Interview (30 min)
2. Technical Interview – ML Exercise (90 min) — a real-world production problem, not algorithmic LeetCode. On-site in Mountain View strongly preferred for local candidates; remote OK otherwise.
3. Hiring Manager Interview (30 min) — introductory conversation on fit, mutual interest, and alignment with a fast-paced startup environment.
4. ML System Design Interview (45 min) — architecting end-to-end ML systems: scalability, trade-offs, real-world constraints.
5. Executive Interview – CTO (45 min) — mostly behavioral, possibly technical; culture fit, leadership, alignment with mission and pace.
6. Executive Interview – CEO (30 min) — cultural alignment, ambition, and overall fit with the company's vision and values.

Senior (L5/E5-equivalent) ML inference / systems engineer, 5+ years, with hands-on vLLM or TRT-LLM inference optimization, GPU-level performance work (C++/CUDA/Rust/optimized Python), model acceleration (quantization, speculative decoding, paged attention, continuous batching), and distributed serving at scale (Kubernetes, Ray, multi-GPU/multi-node). Must be located in the US, Canada, UK, Germany, Switzerland, or Serbia.

Dealbreakers

• Senior-level experience (Google L5 / Meta E5 equivalent; Inference Engineer, Senior Software Engineer). Staff-level candidates go to Inworld's separate Staff role, not this one.
• Experience in ML inference optimization (vLLM or TRT-LLM)
• Located in the United States, Canada, United Kingdom, Germany, Switzerland, or Serbia

Required

• 5+ years as an ML inference or systems engineer
• Experience building backend or ML systems and serving ML models in production
• Proficiency in systems programming and GPU optimization (C++, CUDA, Rust, or optimized Python)
• Model acceleration techniques (quantization, distillation, speculative decoding, caching strategies, continuous batching, or paged attention)
• Distributed systems and scaling (Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, or high concurrency)

Nice to have

• Internal career progression / multiple promotions within the same company
• Public technical work (systems projects, open-source contributions to inference engines, technical write-ups)
• PhD in CS, Physics, Math, or equivalent practical systems experience; top CS program a plus
• Able to commute to the Mountain View, CA office