Software Engineer jobs

Distributed Training and Inference Engineer

SciforiumSan Francisco, CAFull-timeOn-site

$190k–$250k/yr listed by the employer

About the typical $210k a year for senior Software Engineer jobs in CaliforniaHow we compare

How we compare. Typical is the median yearly pay of 1,288 senior Software Engineer jobs in California that list pay, from the employers' own postings. About means within 5%.

Key facts

Experience
5+ years
Degree
Bachelor's degree

Looks real. 1 thing to check before you apply.

On Sciforium’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 2 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 95 / 100 · Strong
See all 6 checksShow fewer
How we check

What you’ll do

  • Maintain, update, and optimize ML libraries and frameworks across environments and hardware configurations.
  • Build and improve the ML software stack, from CUDA and ROCm drivers through JAX and PyTorch tooling.
  • Configure and optimize model sharding and partitioning for distributed training and serving.
  • Integrate and validate modules across multi-node GPU and accelerator clusters for correctness, memory efficiency, and scalability.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 5+ years of industry experience in ML systems, distributed training, or related fields
  • Experience with multi-node distributed training systems and orchestration frameworks such as DTensor or GSPMD
  • Hands-on experience maintaining or building ML training stacks involving CUDA, ROCm, NCCL, XLA, or similar technologies
  • Extensive experience with the XLA/JAX stack, including compilation internals and custom lowering paths (preferred)
  • Background in GPU kernel optimization or accelerator-aware model partitioning (preferred)

Education

  • Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field

Skills

  • Strong programming experience in Python and C++
  • Familiarity with ML tooling and distributed systems
  • Deep understanding of profiling tools such as Nsight, ROCm Profiler, XLA profiler, or TPU tools
  • Deep expertise with partitioning configuration in modern ML frameworks such as PyTorch and JAX
  • Familiarity with distributed serving or large-scale inference frameworks such as vLLM, TensorRT, or FasterTransformer (preferred)
  • Strong understanding of low-level C++ building blocks used in ML frameworks, such as XLA, CUDA kernels, or custom ops (preferred)

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Health insurance
Medical insurance
Dental & vision
Dental and vision insurance
Retirement plan
401k plan
Paid time off
Flexible time off
Meals
Daily lunch, snacks, and beverages
Other
Competitive salary

Also: equity

Full job description

From Sciforium’s careers site

Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency serving platform. Backed by multi-million-dollar funding and direct sponsorship from AMD with hands-on support from AMD engineers the team is scaling rapidly to build the full stack powering frontier AI models and real-time applications.

ABOUT THE ROLE

Show full descriptionShow less

Sciforium is seeking a highly skilled Distributed Training and Inference Engineer to build, optimize, and maintain the critical software stack that powers our large-scale AI training and serving workloads. In this role, you will work across the entire machine learning infrastructure from low-level CUDA/ROCm runtimes to high-level frameworks like JAX and PyTorch to ensure our distributed training systems are fast, scalable, stable, and efficient.

This position is ideal for someone who loves deep systems engineering, debugging complex hardware–software interactions, and optimizing performance at every layer of the ML stack. You will play a pivotal role in enabling the training and deployment of next-generation LLMs and generative AI models.

WHAT YOU'LL DO

  • Software Stack Maintenance: Maintain, update, and optimize critical ML libraries and frameworks including JAX, PyTorch, CUDA, and ROCm across multiple environments and hardware configurations.
  • End-to-End Stack Ownership: Build, maintain, and continuously improve the entire ML software stack from ROCm/CUDA drivers to high-level JAX/PyTorch tooling.
  • Distributed System Optimization: Ensure all model implementations are efficiently sharded, partitioned, and configured for large-scale distributed training and serving.
  • System Integration: Continuously integrate and validate modules for runtime correctness, memory efficiency, and scalability across multi-node GPU/accelerator clusters.
  • Profiling & Performance Analysis: Conduct detailed profiling of compilation graphs, training workloads, and runtime execution to optimize performance and eliminate bottlenecks.
  • Debugging & Reliability: Troubleshoot complex hardware–software interaction issues, including vLLM compilation failures on ROCm, CUDA memory leaks, distributed runtime failures, and kernel-level inconsistencies.
  • Collaborate with research, infrastructure, and kernel engineering teams to improve system throughput, stability, and developer experience.

IDEAL CANDIDATE PROFILE

  • 5+ years of industry experience in ML systems, distributed training, or related fields.
  • Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or related technical fields.
  • Strong programming experience in Python, C++, and familiarity with ML tooling and distributed systems.
  • Deep understanding of profiling tools (e.g., Nsight, ROCm Profiler, XLA profiler, TPU tools).
  • Deep expertise with partitioning configuration on the modern ML frameworks such as PyTorch and JAX.
  • Experience with multi-node distributed training systems and orchestration frameworks (DTensor, GSPMD, etc.).
  • Hands-on experience maintaining or building ML training stacks involving CUDA, ROCm, NCCL, XLA, or similar technologies.

NICE-TO-HAVE

  • Extensive experience with the XLA/JAX stack, including compilation internals and custom lowering paths.
  • Familiarity with distributed serving or large-scale inference frameworks (e.g., vLLM, TensorRT, FasterTransformer).
  • Background in GPU kernel optimization or accelerator-aware model partitioning.
  • Strong understanding of low-level C++ building blocks used in ML frameworks (e.g., XLA, CUDA kernels, custom ops).

BENEFITS INCLUDE

  • Medical, dental, and vision insurance
  • 401k plan
  • Daily lunch, snacks, and beverages
  • Flexible time off
  • Competitive salary and equity

EQUAL OPPORTUNITY

Sciforium is an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

Software Engineer

Simple AI · San Francisco, CA

$130k–$280k/yr · see postingFull-timeOn-site

Wide pay rangeChecked open todayPosted 9 days ago

Software Engineer, Product

Ambrook · New York, NY +2 places

$115k–$260k/yr · see postingFull-timeHybrid

Wide pay rangeChecked open todayPosted 4 days ago

Software Engineer, Robotics

Integrated Biosciences, Inc. · Redwood City, CA

$140k–$230k/yrOn-site

StrongChecked open todayPosted 7 days ago

See more like this