(Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra
Together AI · San Francisco, CA
$250k–$320k/yrFull-time

Together AISan Francisco, CAFull-timePosted 26 days ago
$160k–$280k/yr listed by the employer
How we compare. Typical is the median yearly pay of 208 senior DevOps, SRE & Cloud Engineer jobs in California that list pay, from the employers' own postings. About means within 5%.
On Together AI’s own careers site (Greenhouse).
Summarised from the posting by AI and can be wrong. The full description is the source.
Sign in and add your resume to see which of this job's requirements it shows.
Sign in to check your matchSaved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Also: health insurance
From Together AI’s careers site
About the Role
We're looking for a software engineer, reliability-minded but still a builder, to own what happens in production to the Kubernetes-native control plane that provisions and runs our GPU inference fleet. The platform team builds the manifest-driven API and controllers that let the inference team declare what they need, a cluster, a model deployment, a capacity change, and get it reconciled underneath. Your job is to keep that promise true in the real world: triaging InfiniBand issues when prefill and decode nodes stall exchanging KV cache over RDMA, diagnosing hero-model and LoRA-serving faults that don't show up in generic health checks, and extending the automated self-healing system so each new failure mode becomes a permanent fix instead of a recurring page.
The core value we're after is keeping the decoupling real even when things break, and keeping the fleet efficient without that efficiency becoming a new source of instability. You'll own SLOs and error budgets for inference serving, lead incident response, and turn what postmortems teach you into actual changes in the controllers and automation, not just updated runbooks. You'll also work the reliability side of capacity efficiency, since a mistuned defragmentation or bin-packing pass is itself a production risk. Strong candidates have hands-on production-reliability experience on large-scale distributed or GPU-serving systems, are comfortable troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and think about GPU infrastructure, and the tools that keep it reliable, as software to be engineered.
A product mindset applies: the on-call experience, the clarity of an alert, and the judgment built into an automated remediation are things you ship and iterate on.
Responsibilities
Requirements
About Together AI
Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
Compensation
We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $160,000 - $280,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Application Policy
Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
Please see our privacy policy at https://www.together.ai/privacy.
Something wrong with this job? Report it

Together AI · San Francisco, CA
$250k–$320k/yrFull-time
StrongChecked open todayPosted 1 month ago

Five9 · San Ramon, CA +1 place
$91k–$252k/yr · see postingHybrid
Wide pay rangeChecked open todayPosted 26 days ago

Taskrabbit · San Francisco, CA
$135k–$185k/yrHybrid
StrongChecked open todayPosted 16 days ago

Planet · San Francisco, CA
$145k–$181k/yrFull-timeHybrid
StrongChecked open todayPosted 19 days ago

Robinhood · Bellevue, WA +2 places
$153k–$180k/yrHybrid
StrongChecked open todayPosted 20 days ago

Five9 · San Ramon, CA +1 place
$91k–$252k/yr · see postingHybrid
Wide pay rangeChecked open todayPosted 26 days ago