California jobs

ML Model Serving Engineer

SesameSan Francisco, CA +2 placesFull-timeOn-siteSan Francisco, CA · New York, NY · Bellevue, WA · on-sitePosted 18 days ago

$175k–$280k/yr listed by the employer

This job is closed.

The employer took it down 11 days ago, after 9 days open. Here are similar open jobs.

  • Checked: On the company's own careers siteAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Note: Last seen open 16 days agoLive check
  • Trust score 100 / 100 · Strong
See all 5 checksShow fewer
How we check

Similar open jobs

AI/ML Engineer

Conviva · Foster City, CA +1 place

$180k–$250k/yrFull-time

StrongChecked open todayPosted 1 month ago

See more like this

What you’ll do

  • Improve serving for LLM, speech, and vision models.
  • Build a fast, cost-effective, accurate, and reliable model serving layer with ML infrastructure and training engineers.
  • Extend LLM serving frameworks such as vLLM and SGLang to use high-performance serving techniques.
  • Work with training engineers to identify ways to make models faster without reducing quality.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Expertise optimizing machine learning models for reliable serving at high throughput and low latency
  • Significant systems programming experience
  • Significant performance engineering experience
  • Familiarity with high-performance LLM serving, including vLLM or SGLang deployment and internals (preferred)
  • Experience with a public cloud platform such as GCP, AWS, or Azure (preferred)
  • Experience deploying and scaling inference workloads in the cloud using Kubernetes or Ray (preferred)
  • Track record of leading complex multi-month projects independently (preferred)

Skills

  • Expertise in a differentiable array computing framework
  • PyTorch (preferred)

Other requirements

  • Stay up to date on the latest model serving optimization techniques
  • Interest in learning new things and working across multiple roles (preferred)

Benefits

Health insurance
100% employer-paid health benefits for employees and their dependents, Medical flexible spending account with employer matching up to $1,650 per year
Dental & vision
100% employer-paid dental and vision benefits for employees and their dependents
Retirement plan
401(k) with employer match up to 3.5% of compensation
Paid time off
Unlimited PTO and sick time
Equity
Competitive stock options
Wellness & mental health
Guardian Employee Assistance Program (EAP)

Full job description

From Sesame’s careers site

About Sesame

Sesame believes in a future where computers are lifelike - with the ability to see, hear, and collaborate with us in ways that feel natural and human. With this vision, we're designing a new kind of computer, focused on making voice agents part of our daily lives. Our team brings together founders from Oculus and Ubiquity6, alongside proven leaders from Meta, Google, and Apple, with deep expertise spanning hardware and software. Join us in shaping a future where computers truly come alive.

RESPONSIBILITIES:

  • Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
Show full descriptionShow less
  • Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.
  • Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.
  • Work with the training team to identify opportunities to produce faster models without sacrificing quality.
  • Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
  • Find ways to reduce model initialization times without sacrificing quality.

REQUIRED QUALIFICATIONS:

  • Expert in some differentiable array computing framework, preferably PyTorch.
  • Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.
  • Significant systems programming experience; ex. Experience working on high-performance server systems—you’d be just as comfortable with the internals of VLLM as you would with a complex PyTorch codebase.
  • Significant performance engineering experience; ex. Bottleneck analysis in high-scale server systems or profiling low-level systems code.
  • Always up to date on the latest techniques for model serving optimization.

PREFERRED QUALIFICATIONS:

  • Familiarity with high-performance LLM serving; ex. experience with VLLM, SGlang deployment, and internals.
  • Experience with a public cloud platform such as GCP, AWS, or Azure.
  • Experience deploying and scaling inference workloads in the cloud using Kubernetes, Ray, etc.
  • You like to ship and have a track record of leading complex multi-month projects without assistance.
  • You’re excited to learn new things and work in a multitude of roles.

Full-time Employee Benefits:

  • 401 (k) max employer match: 3.5% of compensation
  • 100% employer-paid health, vision, and dental benefits for you and your dependents
  • Unlimited PTO and sick time
  • Flexible spending account with employer matching up to $1,650/year (medical FSA)
  • Guardian Employee Assistance Program (EAP)
  • Opportunity to share in the company's success with competitive stock options

Benefits do not apply to contingent/contract workers.

Sesame is committed to a workplace where everyone feels valued, respected, and empowered. We welcome all qualified applicants, embracing diversity in race, gender, identity, orientation, ability, and more. We provide reasonable accommodations for applicants with disabilities. Contact careers@sesame.com for assistance.

Something wrong with this job? Report it