Washington jobs

Member of Technical Staff — Model Optimization and Inference (Experienced)

Nuance LabsSeattle, WAOn-siteSponsors US visas

$250k–$350k/yr listed by the employer

Looks real. 1 thing to check before you apply.

On Nuance Labs’ own careers site (Greenhouse), checked open today.

  • To check: Posted by the employer 5 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: Checked open todayGreenhouse board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundGreenhouse board
  • Trust score 89 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Optimize end-to-end inference across LLM, audio, and diffusion models.
  • Develop KV cache strategies for long-context conversations, including eviction, compression, and memory-efficient attention.
  • Evaluate, deploy, and extend inference-serving frameworks for the team's workloads.
  • Profile and benchmark latency and throughput, identify bottlenecks, and build tooling such as profiling viewers and inference test harnesses.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Significant hands-on experience with LLM inference optimization, including shipped work on KV caching, memory layout, attention kernels, or batching strategies in production or a high-traffic research
  • Experience optimizing diffusion model inference
  • Experience doing inference optimization at scale at a senior to senior-staff level at big tech, a leading AI lab, or a high-traffic inference team
  • Hands-on experience with post-training quantization such as GPTQ or AWQ (preferred)
  • Experience deploying real-time AI systems with hard latency SLAs (preferred)
  • Prior work at an AI lab, inference startup, or high-traffic model serving platform (preferred)
  • Contributions to open-source inference frameworks (preferred)

Skills

  • Proficiency with inference serving frameworks such as vLLM, SGLang, TensorRT-LLM, or similar, including adapting them to non-standard workloads
  • Strong Python and PyTorch skills
  • A systematic approach to profiling and optimization
  • Familiarity with speculative decoding or other inference-time acceleration techniques
  • Comfort reading and writing CUDA or Triton kernels (preferred)
  • Familiarity with multimodal or streaming inference architectures (preferred)

Other requirements

  • In-person work in Seattle five days a week

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Health insurance
Variety of health plans, including an HDHP with approximately $2,000 in annual company HSA contributions
Retirement plan
401(k) match: 100% of the first 3% contributed and 50% of the next 2%
Paid time off
15 days of PTO, 10 public holidays, and a full week of office closure at year-end
Equity
Meaningful equity
Wellness & mental health
$2,000 per year for gym memberships, fitness equipment, training or coaching, classes, and more
Commuter benefits
Pre-tax commuter benefits of up to $340 per month for parking and transportation
Meals
Lunch, drinks, and snacks provided every workday
Other
One-time $500 allowance for technology or accessories, Unlimited tokens for AI-native tooling

Full job description

From Nuance Labs’ careers site

About Nuance Labs

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person.

We're a research company, with PhDs from MIT, UW, Oxford, CMU, and Johns Hopkins, and industry experience from Apple, Meta, Amazon AGI, and more. Backed by Accel, Lightspeed, South Park Commons, and NVIDIA, we combine frontier research with ruthless engineering needed for consumer-grade, real-time systems. The team is small, the work is real, and the problems are unsolved.

How Nuance Differentiates

Show full descriptionShow less

Most conversational AI avatars today are hacks — a face slapped on a speech-to-speech pipeline, stuck in the uncanny valley: emotionless, mechanical, one-turn-at-a-time. Current systems take 2–5 seconds to respond; natural conversation requires sub-500ms. That's a 10x improvement, and it demands rethinking the entire stack.

That rethinking starts with full-duplex: an AI that listens and speaks simultaneously, perceives emotion in real time, and responds with a face that actually reflects it. It's an extremely hard problem, and we're developing foundation models designed for it from the ground up.

About the Role

We can train a great model. The next problem is making it fast enough to actually use in a real-time conversation — and that gap is enormous. A model that responds in 3 seconds is a demo. A model that responds in under 500ms is a product.

We’re looking for someone who specializes in taking trained models and squeezing every last millisecond out of them. You understand the full stack from model weights to serving infrastructure — quantization, KV cache optimization, kernel-level acceleration, batching strategies — and you know which lever to pull for which problem. You’ve worked with vLLM, SGLang, or similar frameworks at scale and have strong opinions about where they fall short.

This posting is aimed at experienced engineers and researchers who’ve operated at a senior to senior-staff level at big tech, a leading AI lab, or a high-traffic inference team. Everyone at Nuance is MTS — we don’t run title ladders — but we’re hiring people who have already done this work at scale.

Our stack is more complex than a standard LLM deployment: we’re serving a full-duplex multimodal system that must satisfy strict real-time latency constraints. There’s a lot of unsolved optimization work here, and we need someone who finds that genuinely exciting.

What You’ll Do

  • Own end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
  • Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
  • Evaluate, deploy, and extend inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) for our specific workloads
  • Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
  • Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
  • Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
  • Apply and develop quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
  • Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

What We’re Looking For

  • Significant hands-on experience with LLM inference optimization — you’ve shipped work on KV caching, memory layout, attention kernels, or batching strategies in a production or high-traffic research context
  • Proven proficiency with inference serving frameworks — vLLM, SGLang, TensorRT-LLM, or similar — including going well beyond default configurations and adapting them to non-standard workloads
  • Experience optimizing diffusion model inference (latency reduction, step distillation, caching, or kernel-level work)
  • Strong Python and PyTorch skills; comfort reading and writing CUDA or Triton kernels is a significant plus
  • A systematic approach to profiling and optimization — you measure first, then optimize
  • Familiarity with speculative decoding or other inference-time acceleration techniques

Bonus Points

  • Hands-on experience with post-training quantization (GPTQ, AWQ, or similar) and a clear sense of quality/performance tradeoffs
  • Familiarity with multimodal or streaming inference architectures
  • Experience deploying real-time AI systems with hard latency SLAs
  • Prior work at an AI lab, inference startup, or on a high-traffic model serving platform
  • Contributions to open-source inference frameworks

Compensation

$250,000 – $350,000 base salary, plus meaningful equity. We think long-term ownership matters and structure equity accordingly.

Logistics

  • Location: In-person in Seattle, five days a week — we believe in the compounding value of working shoulder-to-shoulder.
  • Visa sponsorship: We sponsor visas (O-1, H-1B, green card, etc.) from day one.
  • AI-native tooling: Do your best work with the best tools, including unlimited tokens.

Benefits & perks

  • Health: We offer a variety of plans that meet your needs, including an HDHP with ~$2,000 in annual HSA contributions by the company (roughly 2x what most big tech companies put in).
  • Wellness: $2,000/yr to spend on gym memberships, fitness equipment, training/coaching, classes, and more.
  • Work setup: One-time $500 to spend on any tech or accessories that you might need to optimize your workflow.
  • Time off: 15 days of PTO, 10 public holidays, and we close the office for a full week at year-end.
  • Food: Lunch, drinks, and snacks on us every workday. We observe boba tea Tuesdays and Thursdays.
  • Commuter benefits: Utilize pre-tax money (up to $340/month) for parking and transportation.
  • 401(k): 4% match (100% of 1st 3% + 50% of next 2% contributions).

Nuance Labs is an equal opportunity employer. We believe diverse teams build better AI.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

Performance Marketer

Nuance Labs · Seattle, WA

$200k–$225k/yrOn-site

Looks goodChecked open todayPosted 1 month ago

See more like this