DevOps, SRE & Cloud Engineer jobs

Senior AI Infrastructure Engineer, Physical Infrastructure

AndurilCosta Mesa, CAOn-site

$166k–$220k/yr listed by the employer

About the typical $203k a year for senior DevOps, SRE & Cloud Engineer jobs in CaliforniaHow we compare

How we compare. Typical is the median yearly pay of 208 senior DevOps, SRE & Cloud Engineer jobs in California that list pay, from the employers' own postings. About means within 5%.

Key facts

Experience
10+ years
Clearance
TOP SECRET clearance required

This job is closed.

The employer took it down 4 days ago, after 17 days open. Here are similar open jobs.

  • To check: Posted by the employer 2 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundGreenhouse board
  • Note: Last seen open 9 days agoLive check
  • Trust score 95 / 100 · Strong
See all 6 checksShow fewer
How we check

Similar open jobs

Infrastructure Engineer

Simple AI · San Francisco, CA

$140k–$250k/yrFull-timeOn-site

StrongChecked open todayPosted 9 days ago

Senior Cloud Engineer

Earnest · Remote · US only

$184k–$230k/yr

StrongChecked open todayPosted 8 days ago

Senior Platform Engineer

Doma Technology LLC · Remote · US only

$160k–$190k/yrFull-time

StrongChecked open todayPosted 9 days ago

See more like this

What you’ll do

  • Rack, cable, bring up, and validate large-scale GPU systems, including their physical infrastructure and firmware.
  • Build and tune networking and parallel storage for distributed training and inference clusters.
  • Automate cluster deployment and configuration, and operate GPU scheduling and workload isolation.
  • Monitor fleet health, diagnose hardware and network faults, and build automated resilience mechanisms.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 10+ years in a hands-on infrastructure, HPC, or datacenter engineering role supporting GPU compute at scale
  • Hands-on experience bringing up, cabling, and managing firmware/drivers for H200/B200/B300 or comparable GPU systems
  • Experience with NVLink, InfiniBand, RoCE, or Spectrum-X in clusters of hundreds of GPUs
  • Experience with high-performance parallel storage such as VAST, DDN, Weka, Lustre, or similar
  • Kubernetes experience
  • Run:AI or similar GPU scheduling/orchestration experience (preferred)
  • Experience with NVIDIA NVL72 rack-scale systems (preferred)
  • Experience supporting LLM token serving/inference infrastructure alongside training clusters (preferred)
  • Network fabric tuning experience, including congestion control, adaptive routing, or QoS for RoCE/InfiniBand at scale (preferred)
  • Familiarity with GPU/network observability tooling such as DCGM and fabric telemetry, and automated fault detection (preferred)
  • Experience supporting infrastructure as a shared platform for multiple internal customer teams with differing requirements (preferred)

Skills

  • Strong automation background building repeatable, automated deployment pipelines

On the job

  • Able to lift and move 50+ lbs and perform physical datacenter work

Other requirements

  • Eligible to obtain and maintain an active U.S. Top Secret clearance

Benefits

Equity
Highly competitive equity grants are included in the majority of full-time offers.
Other
A comprehensive benefits package is available at little to no cost to employees; specific benefits are not listed.

Full job description

From Anduril’s careers site

Anduril Industries is a defense technology company with a mission to transform U.S. and allied military capabilities with advanced technology. By bringing the expertise, technology, and business model of the 21st century’s most innovative companies to the defense industry, Anduril is changing how military systems are designed, built and sold. Anduril’s family of systems is powered by Lattice OS, an AI-powered operating system that turns thousands of data streams into a realtime, 3D command and control center. As the world enters an era of strategic competition, Anduril is committed to bringing cutting-edge autonomy, AI, computer vision, sensor fusion, and networking technology to the military in months, not years.

Show full descriptionShow less

ABOUT THE TEAM

CorpTech Infrastructure Engineering builds and operates the foundational infrastructure that powers Anduril at large. We give engineers, researchers, and product teams across the company a place to deploy fast, scalable infrastructure without having to become infrastructure experts themselves. As Anduril's AI and autonomy ambitions grow, our team is responsible for delivering the next generation of compute, networking, and storage capabilities that make cutting edge model training and inference possible company wide.

ABOUT THE JOB

We’re looking for a Senior AI Infrastructure Engineer to lead the vision, execution, and long-term stability of how Anduril trains with GPUs at scale. In this role, you will take absolute ownership of cluster robustness, ensuring our high-performance GPU systems are highly available, fault-tolerant, and resilient for ML platform and research teams company-wide. This is a highly hands-on role where your primary focus is logical stability and automated resilience—building self-healing mechanisms to proactively detect and isolate hardware faults, tuning NCCL and high-speed networking, and optimizing Kubernetes, Run:AI, and Ray scheduling. By replacing manual triage with automated deployment tooling and deep observability, you will ensure our massive-scale training infrastructure runs seamlessly and scales without linear headcount growth.

WHAT YOU'LL DO

  • Rack, stack, cable, and bring up GPU compute (H200/B200/B300, NVL72) including physical topology, power, cooling, firmware/BIOS, and burn in validation.
  • Build and tune the interconnect fabric (NVLink, InfiniBand, RoCE, Spectrum-X) connecting hundreds of GPUs into low latency training and inference clusters.
  • Integrate high performance parallel storage (VAST, DDN, Weka) to sustain the throughput demanded by distributed training and terabyte scale multi modal datasets across Anduril's programs.
  • Automate cluster deployment and configuration end to end, including infrastructure as code for bring up, firmware/driver management, and fabric config, so new capacity comes online with minimal manual work.
  • Operate and extend our Kubernetes/Run:AI environment for GPU scheduling, quota management, and multi tenant workload isolation across research and engineering teams company wide.
  • Own fleet health: monitoring, alerting, and rapid triage of hardware and network faults (bad transceivers, GPU Xid errors, NCCL/collective failures, RoCE congestion).
  • Onboard engineers and researchers onto the platform and act as their escalation point, working directly alongside them to debug, train, and optimize their workloads whenever infrastructure, not the model, is the bottleneck.
  • Partner with product facing teams across Anduril to understand emerging compute needs and translate them into platform capability.

REQUIRED QUALIFICATIONS

  • 10+ years in a hands on infrastructure, HPC, or datacenter engineering role supporting GPU compute at scale.
  • Hands on experience with H200/B200/B300 (or comparable) GPU systems: bring up, cabling, firmware/driver management.
  • Experience with high performance interconnects (NVLink, InfiniBand, RoCE, Spectrum-X) in clusters of hundreds of GPUs.
  • Experience with high performance parallel storage (VAST, DDN, Weka, Lustre, or similar).
  • Kubernetes required; Run:ai or similar GPU scheduling/orchestration experience strongly preferred.
  • Strong automation background. You build repeatable, automated deployment pipelines rather than manual processes.
  • Able to lift/move 50+ lbs and perform physical datacenter work (rack/stack/cable/troubleshoot).
  • Eligible to obtain and maintain an active U.S. Top Secret clearance.

PREFERRED QUALIFICATIONS

  • Experience with NVIDIA NVL72 rack scale systems.
  • Experience supporting LLM token serving/inference infrastructure alongside training clusters.
  • Network fabric tuning experience (congestion control, adaptive routing, QoS) for RoCE/InfiniBand at scale.
  • Familiarity with GPU/network observability tooling (DCGM, fabric telemetry) and automated fault detection.
  • Experience supporting infrastructure as a shared platform serving multiple internal customer teams with differing requirements.

US Salary Range

$166,000—$220,000 USD

The salary range for this role is an estimate based on a wide range of compensation factors, inclusive of base salary only. Actual salary offer may vary based on (but not limited to) work experience, education and/or training, critical skills, and/or business considerations. Highly competitive equity grants are included in the majority of full time offers; and are considered part of Anduril's total compensation package. Additionally, Anduril offers top-tier benefits for full-time employees, including:

Benefits

At Anduril, we invest in our people. Our comprehensive, competitive benefits package (available at little to no cost to employees) ensures you’re supported in health, recovery, and whatever comes next. For more information, Explore Our Benefits.

Protecting Yourself from Recruitment Scams

Anduril is committed to maintaining the integrity of our Talent acquisition process and the security of our candidates. We've observed a rise in sophisticated phishing and fraudulent schemes where individuals impersonate Anduril representatives, luring job seekers with false interviews or job offers. These scammers often attempt to extract payment or sensitive personal information.

To ensure your safety and help you navigate your job search with confidence, please keep the following critical points in mind:

-

No Financial Requests: Anduril will never solicit payment or demand personal financial details (such as banking information, credit card numbers, or social security numbers) at any stage of our hiring process. Our legitimate recruitment is entirely free for candidates.

  • Please always verify communications:
  • Direct from Anduril: If you receive an email from one of our recruiters, it will only come from an @anduril.com address.
  • Via Agency Partner: If contacted by a recruiting agency for an Anduril role, their email will clearly identify their agency. If you suspect any suspicious activity, please verify the agency's authenticity by reaching out to contact@anduril.com.

-

Exercise Caution with Unsolicited Outreach: If you receive any communication that appears suspicious, contains grammatical errors, or makes unusual requests, do not engage. Always confirm the sender's email domain is @anduril.com before providing any personal information or clicking on links.

-

What to Do If You Suspect Fraud: Should you encounter any questionable or fraudulent outreach claiming to be from Anduril, please report it immediately to contact@anduril.com. Your proactive caution is invaluable in protecting your personal information and upholding the security and trustworthiness of our recruitment efforts.

Data Privacy

By submitting your application, you consent to Anduril Industries using a third-party service provider to conduct pre-employment risk, integrity, and due diligence screening and assessing potential risks as part of your application process. This third-party service provider provides risk-intelligence services that may include analysis of sanctions and watchlists, adverse media, public-record information, and other lawful open-source or commercial data sources. This third-party service provider does not act as a consumer reporting agency. Use of this provider helps to ensure compliance with applicable laws and protect technology, intellectual property, and organizational security.

To view Anduril's candidate data privacy policy, please visit https://anduril.com/applicant-privacy-notice/.

Something wrong with this job? Report it · This flag looks wrong