Network Engineer jobs

GPU Cluster Engineer, Networking

SciforiumSan Francisco, CAFull-timeOn-site

$150k–$180k/yr listed by the employer

About the typical $169k a year for Network Engineer jobs in California (all levels)How we compare

How we compare. Typical is the median yearly pay of 107 Network Engineer jobs in California (all levels) that list pay, from the employers' own postings. About means within 5%.

Looks real. 1 thing to check before you apply.

On Sciforium’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 3 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 90 / 100 · Strong
See all 6 checksShow fewer
How we check

What you’ll do

  • Design and bring up complete GPU-cluster networks, including compute, storage, frontend, management, and perimeter networks.
  • Configure and validate RDMA fabrics, benchmark performance, and troubleshoot network bottlenecks and failures.
  • Operate routing, switching, firewalls, VPNs, network automation, telemetry, and network security in production.
  • Plan and operate inter-site fiber/DWDM links and cloud connectivity, and engineer WAN traffic for data movement.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Licenses and certifications

  • CCIE/JNCIE or NVIDIA networking certification (InfiniBand/Spectrum-X/UFM) (preferred)

Experience

  • 7+ years designing and operating production data center networks, including designing or operating at least one large-scale HPC/AI cluster fabric using InfiniBand or RoCE v2
  • Hands-on experience with 100–800G optics, high-radix switching, and Clos or rail-optimized topologies
  • Production network security experience with enterprise firewalls, site-to-site and remote-access VPNs, and segmentation design
  • Cloud networking experience with VPC design and dedicated interconnects (Direct Connect/ExpressRoute class)
  • Dark fiber or DWDM procurement and operations experience (preferred)
  • BlueField DPU or SmartNIC deployments (preferred)
  • Experience supporting distributed training at 1,000+ GPU scale or multi-cluster/cross-site training (preferred)

Skills

  • Expert-level BGP (including EVPN-VXLAN), OSPF, ECMP, and VRF segmentation across at least two major vendor platforms
  • Deep RDMA expertise in lossless RoCE v2 tuning (PFC/ECN/DCQCN) and/or InfiniBand fabric management (subnet managers, adaptive routing, SHARP)
  • Network automation proficiency with Python and Ansible (or Nornir/NAPALM), plus Git-based configuration workflows
  • Working knowledge of the host-side RDMA stack (MOFED/DOCA, NIC tuning, GPUDirect) and how NCCL/RCCL collectives map onto physical fabric
  • Kubernetes networking for GPU serving (CNI, SR-IOV, Multus) (preferred)

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Health insurance
Medical insurance
Dental & vision
Dental and vision insurance
Retirement plan
401(k) plan
Paid time off
Flexible time off
Meals
Daily lunch, snacks, and beverages
Other
Competitive salary

Also: equity

Full job description

From Sciforium’s careers site

Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency serving platform. Backed by multi-million-dollar funding and direct sponsorship from AMD with hands-on support from AMD engineers the team is scaling rapidly to build the full stack powering frontier AI models and real-time applications.

Role Overview

Show full descriptionShow less

We are looking for a Senior Network Engineer to own the complete networking stack of our GPU clusters — from the RDMA compute fabric to the data center perimeter to cross-site and cloud connectivity. You will design the network architecture for new large-scale clusters, lead its bring-up, and operate it in production. This role owns everything from the NIC port outward: backend InfiniBand/RoCE fabrics, frontend and storage networks, out-of-band management, firewalls and edge security, and the links that connect our clusters to each other and to the cloud. You will work alongside our Hardware Operations team (physical install) and Systems & Platform team (host software stack) to deliver a fabric that never becomes the bottleneck.

Key Responsibilities

  • Fabric Architecture & Cluster Network Design
  • Greenfield Design: Architect the full network for new GPU clusters — compute/backend fabric, storage network, in-band frontend, out-of-band management (BMC/IPMI), and perimeter — including topology (fat-tree/Clos, rail-optimized), oversubscription analysis, and growth planning.
  • RDMA Fabric Selection: Design and justify InfiniBand (NDR/XDR, Quantum-class) vs. lossless Ethernet/RoCE v2 (Spectrum-X, Tomahawk-class) fabrics for training and inference workloads, including scale-up domains (NVLink/NVSwitch, NVL72-class rack systems).
  • Hardware & Physical Plant: Specify switches, NICs (ConnectX/BlueField), optics (400/800G OSFP/QSFP-DD), and cabling plans; produce the port maps and cable schedules Hardware Operations executes.
  • Logical Design: Own IP addressing plans, VLAN/VRF segmentation, and BGP/EVPN-VXLAN underlay-overlay architecture.
  • RDMA & Performance Engineering
  • Lossless Ethernet: Configure and validate RoCE v2 at scale — PFC, ECN/DCQCN congestion control, QoS/DSCP classification, buffer tuning — and prove no-drop behavior under full training load.
  • InfiniBand Operations: Run subnet managers (UFM/OpenSM), adaptive routing, SHARP in-network aggregation, partition keys, and virtual-lane QoS.
  • Fabric Validation: Benchmark and sign off new fabrics with perftest/ib_write_bw and nccl-tests/rccl-tests; verify GPUDirect RDMA paths and topology-aware placement with the Systems & Platform team.
  • Performance Debugging: Root-cause fabric-level issues — congestion trees, ECMP polarization, link flaps, degraded optics, straggler nodes caused by the network.
  • Production Network Operations & Security
  • Routing & Switching: Configure and maintain BGP, OSPF, ECMP, and EVPN-VXLAN across multi-vendor NOSes (Arista EOS, NVIDIA Cumulus, Cisco NX-OS, Junos, SONiC).
  • Perimeter & Security: Own firewalls, VPNs (WireGuard/IPsec), NAT, ACLs, and network segmentation between production, research, and management domains.
  • Network as Code: Generate configs from a source of truth (NetBox), deploy via Ansible/Nornir/NAPALM with Git-reviewed, CI-validated changes and zero-touch provisioning for new switches.
  • Telemetry: Build fabric observability — streaming telemetry/gNMI, sFlow, optics DOM, buffer/drop counters — into Prometheus/Grafana with actionable alerting.
  • Lifecycle & On-Call: Run switch/NIC firmware upgrade campaigns with minimal workload disruption, and serve as escalation point for network incidents.
  • Cross-Cluster & Cloud Connectivity
  • Inter-Site Links: Design and operate dark fiber and DWDM capacity between sites, including latency budgeting and carrier/IX peering.
  • Hybrid Cloud: Architect AWS Direct Connect / Azure ExpressRoute / GCP Interconnect links, VPC and Transit Gateway designs, and VPN failover paths.
  • Data Movement: Engineer WAN QoS and traffic paths for cross-cluster replication, dataset transfer, and checkpoint traffic.

Qualifications

  • Must-Haves:
  • 7+ years designing and operating production data center networks, including at least one large-scale HPC/AI cluster fabric (InfiniBand or RoCE v2) you designed or ran.
  • Expert-level routing and switching: BGP (including EVPN-VXLAN), OSPF, ECMP, VRF segmentation, across at least two major vendor platforms.
  • Deep RDMA expertise: lossless RoCE v2 tuning (PFC/ECN/DCQCN) and/or InfiniBand fabric management (subnet managers, adaptive routing, SHARP).
  • Hands-on experience with 100–800G optics, high-radix switching, and Clos/rail-optimized topologies.
  • Production network security experience: enterprise firewalls, site-to-site and remote-access VPNs, segmentation design.
  • Network automation proficiency: Python plus Ansible (or Nornir/NAPALM), with Git-based configuration workflows.
  • Working knowledge of the host-side RDMA stack (MOFED/DOCA, NIC tuning, GPUDirect) and how NCCL/RCCL collectives map onto physical fabric.
  • Cloud networking experience: VPC design and dedicated interconnects (Direct Connect/ExpressRoute class).
  • Nice-to-Haves:
  • Dark fiber / DWDM procurement and operations experience.
  • BlueField DPU or SmartNIC deployments.
  • Experience supporting distributed training at 1,000+ GPU scale, or multi-cluster/cross-site training.
  • Kubernetes networking for GPU serving (CNI, SR-IOV, Multus).
  • Expert certifications (CCIE/JNCIE) or NVIDIA networking certification (InfiniBand/Spectrum-X/UFM).

BENEFITS INCLUDE

  • Medical, dental, and vision insurance
  • 401k plan
  • Daily lunch, snacks, and beverages
  • Flexible time off
  • Competitive salary and equity

EQUAL OPPORTUNITY

Sciforium is an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

Network Engineer

ATOMS · San Francisco, CA

$122k–$154k/yrFull-timeOn-site

StrongChecked open todayPosted 12 days ago

Network Engineer

AMAX · Fremont, CA

$120k–$150k/yrFull-timeOn-site

StrongLast seen open yesterdayPosted 1 month ago

Network Engineer

Neuralink · South San Francisco, CA

$83k–$234k/yr · see postingFull-timeOn-site

Wide pay rangeChecked open todayPosted 1 month ago

Network Engineer

Security Level 5 · San Francisco, CA

$200k–$350k/yrFull-timeOn-site

StrongOpen 3 monthsChecked open todayPosted 3 months ago

Network Engineer

OpenAI · San Francisco, CA

$347k–$510k/yrFull-timeHybrid

Looks goodOpen 3 monthsChecked open todayPosted 3 months ago

See more like this