Software Engineering jobs

Software Engineer, Inference

Thinking Machines LabSan Francisco, CA +1 placeFull-timeOn-sitePosted 1 month ago

$300k–$400k/yr listed by the employer

Key facts

Where
San Francisco, CA · New York, NY · on-site
US visa
Sponsors US visas

Looks real and checked open today.

On Thinking Machines Lab’s own careers site (Ashby).

  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 5 checksShow fewer
How we check

What you’ll do

  • Operate and scale production inference systems serving live traffic, including Tinker's multi-tenant platform.
  • Manage safe, incremental rollouts of models, model versions, and inference optimizations.
  • Improve observability, alerting, and capacity planning to detect, diagnose, and resolve production issues.
  • Partner with inference and research teams to productionize serving techniques, and lead incident response and root cause analysis.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Experience operating large-scale, latency-sensitive production systems
  • Experience with observability, monitoring, and incident response for production services
  • Experience running production inference for large language models or other large-scale ML systems (preferred)
  • Experience with deployment and rollout systems, such as canarying, blue/green deploys, or feature flags (preferred)
  • Experience with capacity planning and cost optimization for GPU or TPU infrastructure (preferred)
  • Comfortable being on-call and leading incident response for critical production systems (preferred)
  • Comfortable working with high autonomy in a fast-changing, early-stage environment (preferred)

Skills

  • Proficiency in Python and Go or another systems language
  • Strong understanding of distributed systems and how they fail at scale
  • Familiarity with inference-specific techniques, such as batching, caching, or quantization, and their operational implications (preferred)

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Health insurance
Generous health benefits
Dental & vision
Generous dental and vision benefits
Paid time off
Unlimited PTO
Parental leave
Paid parental leave
Relocation help
Relocation support as needed

Full job description

From Thinking Machines Lab’s careers site

ABOUT THINKING MACHINES

The mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.

ABOUT THE ROLE

Show full descriptionShow less

We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models to real users. Our research and inference teams push the limits of model performance and serving efficiency; this role makes sure those gains reach production safely and stay up — powering Tinker's live, multi-tenant serving and the products built on top of our models.

This is a production-facing systems role at the center of the company. You'll be the bridge between cutting-edge inference techniques and the day-to-day reality of serving real traffic: rollouts, capacity, incidents, and everything that keeps a fast-growing platform online.

WHAT YOU'LL DO

  • Operate and scale the production inference systems that serve live traffic, including Tinker's multi-tenant serving platform
  • Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment to production
  • Build and improve observability, alerting, and capacity planning so the team can detect, diagnose, and resolve production issues quickly
  • Partner with inference and research teams to productionize new serving techniques without compromising reliability
  • Lead incident response for production inference issues, driving root cause analysis and durable fixes
  • Design for graceful degradation, failover, and redundancy so that serving stays resilient as usage grows
  • Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale

SKILLS & QUALIFICATIONS

MINIMUM QUALIFICATIONS

  • Experience operating large-scale, latency-sensitive production systems
  • Proficiency in Python and Go or another systems language
  • Experience with observability, monitoring, and incident response for production services
  • Strong understanding of distributed systems and how they fail at scale

PREFERRED QUALIFICATIONS

  • Experience running production inference for large language models or other large-scale ML systems
  • Experience with deployment and rollout systems, such as canarying, blue/green deploys, or feature flags
  • Experience with capacity planning and cost optimization for GPU or TPU infrastructure
  • Familiarity with inference-specific techniques, such as batching, caching, or quantization, and their operational implications
  • Comfortable being on-call and leading incident response for critical production systems
  • Comfortable working with high autonomy in a fast-changing, early-stage environment

LOGISTICS

  • Location: This role is based in San Francisco, California or New York, New York.
  • Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $300,000 - $400,000 USD.
  • Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
  • Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

Software Engineer

talentpluto · San Francisco, CA

$165k–$250k/yrFull-timeOn-site

StrongChecked open todayPosted yesterday

Software Engineer

ServiceNow · Santa Clara, CA

$126k–$195k/yrFull-time

StrongChecked open todayPosted 4 days ago

See more like this