IntelliPro Group Inc. (agency)San Francisco, CAFull-timeHybridFirst seen 16 days ago
$170k–$320k/yr listed by the employer
About the typical $235k a year for senior Machine Learning Engineer jobs in CaliforniaHow we compare
How we compare. Typical is the median yearly pay of 157 senior Machine Learning Engineer jobs in California that list pay, from the employers' own postings. About means within 5%.
Build and deploy high-throughput, low-latency inference systems for large language and foundational speech models.
Optimize inference performance for latency, throughput, and time to first token or audio in real-time streaming environments.
Develop real-time conversational AI serving systems and improve performance across GPU clusters.
Summarised from the posting by AI and can be wrong. The full description is the source.
Requirements
Experience
Hands-on experience building and deploying high-throughput, ultra-low-latency inference engines for large language models or foundational speech models
Practical experience with continuous batching, KV cache management, and stateful connections for real-time conversational AI
Under-the-hood familiarity with vLLM, TensorRT-LLM, SGLang, or NVIDIA Triton Inference Server (preferred)
Open-source contributions to LLM serving framework repositories (preferred)
Experience handling continuous audio streams over WebSockets or WebRTC, deploying neural audio codecs, and managing chunked audio generation (preferred)
Experience implementing speculative decoding, lookahead decoding, or chunked prefill (preferred)
Hands-on experience with post-training quantization and deploying models in FP8, INT8, AWQ, or GPTQ (preferred)
Experience deploying multi-GPU and multi-node inference pipelines and managing autoscaling infrastructure with Kubernetes (preferred)
Skills
Understanding of latency, throughput, and Time-To-First-Token or Time-To-First-Audio tradeoffs in real-time streaming environments
Deep understanding of NVIDIA Ampere/Hopper GPU architectures and memory hierarchy
Other requirements
Hybrid work with a minimum of 3 days in-office per week
Your match
Sign in and add your resume to see which of this job's requirements it shows.
Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Benefits
Health insurance
Healthcare coverage for employees and dependents, with a generous employer subsidy
Dental & vision
Dental and vision coverage
Retirement plan
401(k) plan with company matching for full-time employees
Paid time off
Unlimited PTO plus 13 paid holidays
Parental leave
12 weeks of paid leave for new parents, regardless of gender
Bonus
Performance bonus
Other
Hybrid office; minimum of 3 days in-office per week, Choice of top-of-the-line laptops or workstations, annual offsites, and a fully stocked office
Also: equity
Full job description
From IntelliPro Group Inc.’s careers site
Machine Learning Engineer, Inference & Serving (Speech LLM) — San Francisco, CA (Hybrid)
Overview
We are looking for a Machine Learning Engineer, Inference & Serving (Speech LLM) to build and deploy high-throughput, ultra-low-latency inference systems for large language models and foundational speech models.
You may be a good fit if you:
Have hands-on experience building and deploying high-throughput, ultra-low-latency inference engines for large language models or foundational speech models.
Understand the intricate tradeoffs between latency, throughput, and Time-To-First-Token (or Time-To-First-Audio) in real-time streaming environments.
Show full descriptionShow less
Have practical experience with continuous batching, KV cache management (e.g., PagedAttention), and stateful connections necessary for real-time conversational AI.
Possess a deep understanding of GPU architectures (NVIDIA Ampere/Hopper) and the memory hierarchy, allowing you to identify and eliminate hardware bottlenecks.
Communicate clearly and collaborate effectively, as you will sit at the critical intersection between the core ML training team and the backend infrastructure team.
Thrive in fast-moving environments and genuinely enjoy the systems-engineering challenge of squeezing every last drop of performance out of a cluster of GPUs.
Are obsessed with building AI systems that natively understand and generate speech.
Strong candidates may also have experience with:
Frontier Serving Frameworks: Deep, under-the-hood familiarity with modern LLM serving frameworks like vLLM, TensorRT-LLM, SGLang, or NVIDIA Triton Inference Server (bonus points for active open-source contributions to these repositories).
Real-Time Audio Streaming: Experience handling continuous audio streams over WebSockets or WebRTC, deploying neural audio codecs, and managing chunked audio generation to minimize conversational latency.
Advanced Inference Techniques: Implementing cutting-edge generation algorithms such as speculative decoding, lookahead decoding, or chunked prefill.
Model Compression & Quantization: Hands-on experience with post-training quantization (PTQ), deploying models in FP8, INT8, AWQ, or GPTQ, without degrading audio naturalness or ASR accuracy.
Large-Scale Distributed Systems: Deploying multi-GPU (Tensor Parallelism) and multi-node inference pipelines, and managing autoscaling infrastructure using Kubernetes.
Comprehensive Benefits: Top-tier healthcare for employees and dependents, including dental and vision, and a generous employer subsidy.
Retirement Planning: 401(k) plan for full-time employees with company matching.
Paid Time Off: Unlimited PTO, plus 13 paid holidays.
New Parent Leave: 12 weeks of paid time off to spend time with your new family, regardless of gender.
Hybrid Office: Minimum of 3x in-office per week to foster highly collaborative, fast-paced research.
Gear & Perks: Choice of top-of-the-line laptops/workstations, annual offsites, and a fully stocked office.
APN Job ID: 173624
About Us
IntelliPro Group Inc. is a global staffing and talent solutions company. We connect exceptional professionals with leading organizations across technology, engineering, and professional services. By partnering closely with both clients and candidates, we build long-term relationships that help businesses grow and careers thrive.
We are an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristics.