Software Engineering jobs

Software Engineer, Inference

LumaRedwood City, CA +3 placesFull-timeHybridPosted 1 month ago

$195k–$345k/yr listed by the employer

Key facts

Schedule
Days
Where
Redwood City, CA · United States · New York, NY · Seattle, WA · hybrid

This job is closed.

The employer took it down 13 days ago, after 9 days open. Here are similar open jobs.

  • Checked: On the company's own careers siteAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 4 checksShow fewer
How we check

Similar open jobs

Software Engineer

talentpluto · San Francisco, CA

$165k–$250k/yrFull-timeOn-site

StrongChecked open todayPosted 2 days ago

Software Engineer

ServiceNow · Santa Clara, CA

$126k–$195k/yrFull-time

StrongChecked open todayPosted 4 days ago

See more like this

What you’ll do

  • Integrate new model architectures into the inference engine and optimize model deployments.
  • Build tooling to measure, profile, and track inference jobs and workflows.
  • Automate, test, and maintain inference services, deployment pipelines, and scheduling systems.
  • Manage inference workloads across clusters and hardware providers, scaling deployments and optimizing GPU use while meeting SLOs.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar
  • Experience with queues, scheduling, traffic control, and fleet management at scale
  • Experience with Linux, Docker, and Kubernetes
  • Experience with orchestration, deployment, and scheduling
  • Experience with high-performance large-scale ML systems (100+ GPUs) (preferred)

Skills

  • Strong Python skills
  • Strong system-architecture skills
  • Familiarity with Redis and S3-compatible storage
  • Familiarity with modern networking stacks including RDMA (RoCE, InfiniBand, NVLink) (preferred)
  • CUDA (preferred)
  • FFmpeg or multimedia processing (preferred)

Full job description

From Luma’s careers site

You'll own how Luma's models get served — integrating new architectures into the inference engine, scaling deployments across thousands of machines, and keeping expensive GPU fleets busy while meeting internal SLOs.

This is large-scale inference systems work: scheduling, fleet management, deployment pipelines, and reliability across clusters and hardware providers. It fits a strong systems engineer comfortable with model serving and Kubernetes at scale. If you want pure modeling rather than the systems that run models, this is firmly the systems side.

What You'll Own

  • Ship new model architectures by integrating them into the inference engine.
Show full descriptionShow less
  • Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
  • Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
  • Automate, test, and maintain inference services for maximum uptime and reliability.
  • Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
  • Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

First 90 Days

One way the first 90 could unfold.

  • Days 1–30 — Immerse & Diagnose: Learn the inference stack, the fleets, and where reliability or utilization break.
  • Days 30–60 — Ship & Validate: Integrate a model or ship tooling/scheduling that improves uptime or GPU utilization.
  • Days 60–90 — Scale & Systemize: Harden deployment pipelines and scheduling across clusters and providers.

What You Bring

  • Strong Python and system-architecture skills.
  • Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.
  • Experience with queues, scheduling, traffic control, and fleet management at scale.
  • Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.
  • Familiarity with Redis and S3-compatible storage.

Nice to Have

  • Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink).
  • High-performance large-scale ML systems (100+ GPUs).
  • CUDA, and FFmpeg or multimedia processing.

About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.

Something wrong with this job? Report it · This flag looks wrong