Software Engineering jobs

Member of Technical Staff - Inference

Hyperbolic LabsSan Francisco, CAFull-time

Pay not listed

Looks real. 2 things to check before you apply.

On Hyperbolic Labs’ own careers site (Ashby), checked open today.

  • To check: No pay range listed, though one is likely required in CAAsk for the pay range before you interview. The law here likely requires one.
  • To check: Posted by the employer 2 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: No reposts foundAshby board
  • Trust score 85 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Deploy and serve models on Forge and Kubernetes across distributed clusters and heterogeneous hardware.
  • Evaluate inference frameworks and build monitoring, gateways, and endpoints for production deployments.
  • Develop inference optimization, autoscaling, and KV-cache orchestration capabilities.
  • Debug customer inference deployments and performance issues.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Strong general inference background and broad understanding of the stack from request to token
  • Deep Kubernetes experience, including hands-on operation of production clusters
  • Experience setting up monitoring, gateways, and endpoints for production inference services
  • Proven ability to build a product end to end and take it from nothing to serving real traffic
  • Experience spanning inference deployment and inference optimization (preferred)
  • Hands-on model optimization, such as quantization, batching strategies, or kernel-level tuning (preferred)
  • Experience deploying inference across heterogeneous accelerators (preferred)
  • Experience supporting customers with inference debugging and performance issues (preferred)
  • Experience at a GPU cloud, inference provider, or AI infrastructure company (preferred)

Skills

  • Solid understanding of TTFT, disaggregated inference, speculative decoding, and KV cache internals
  • Familiarity with modern inference frameworks and serving engines, and ability to evaluate and select them for a workload
  • Working knowledge of NVIDIA Dynamo and its role in distributed serving architecture
  • Understanding of RDMA and high-performance networking for distributed serving (preferred)

Other requirements

  • Strong self-initiative and comfort owning an area with minimal direction
  • Willingness to take on adjacent work as product needs require

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Full job description

From Hyperbolic Labs’ careers site

WHO WE ARE

Join us in putting the world’s compute behind the people building what comes next.

At Hyperbolic, we’re making global AI compute liquid. We bring fragmented, heterogeneous GPU capacity together into one unified, productive platform, making it easier to access, sell, resell, manage, and orchestrate the resources that power AI. For neocloud, we turn GPU capacity into productive infrastructure that delivers lasting value and accelerate their time to market.

Show full descriptionShow less

We believe bold ideas deserve the computing power to become reality. We’re building toward a future where compute flows as freely as electricity, available when and where it’s needed, so builders everywhere can turn their most ambitious ideas into breakthroughs.

We’re looking for curious, driven people who take ownership, challenge assumptions, and care deeply about what they build. If you’re energized by hard problems and want to help shape how the world accesses AI, we’d love to build with you.

ABOUT THE ROLE

We're looking for an Inference Engineer to build inference capabilities on top of Forge, our unified control plane, so customers can consume model tokens without managing GPUs and our NeoCloud partners get a full-stack path to their own token-factory offering. You'll own how models get deployed and served across clusters distributed around the world, on heterogeneous hardware.

Deployment comes first: serving models on Forge and our Kubernetes offering, evaluating inference frameworks, and standing up the monitoring, gateways, and endpoints that make a deployment production-ready. From there the work expands into optimization, autoscaling, KV-cache orchestration, and customer inference debugging. This is the primary seat for inference at Hyperbolic \u2014 you'll build it end to end, with real influence over where the scope lands.

WHO YOU ARE

  • Strong general inference background with a broad, high-level command of the stack rather than a narrow specialty — you can reason about the whole path from request to token
  • Deep Kubernetes experience, including hands-on ability to operate clusters in production, not just deploy to them
  • Solid grasp of the concepts that govern inference performance: TTFT, disaggregated inference, speculative decoding, and KV cache and its inner workings
  • Familiarity with modern inference frameworks and serving engines, and the judgment to evaluate and select among them for a given workload
  • Working knowledge of NVIDIA Dynamo and how it fits into a distributed serving architecture
  • Experience setting up monitoring, gateways, and endpoints for production inference services
  • Proven ability to build a product end to end — you've taken something from nothing to serving real traffic
  • Strong self-initiative and comfort operating as the primary owner of an area with minimal direction
  • Generalist instincts: you're willing to pick up adjacent work when it's what the product needs

PREFERRED QUALIFICATIONS

  • Experience spanning both inference deployment and inference optimization
  • Hands-on model optimization work — quantization, batching strategies, kernel-level tuning, or similar
  • Understanding of RDMA and high-performance networking as they apply to distributed serving
  • Experience deploying inference across heterogeneous accelerators
  • Background supporting customers directly on inference debugging and performance issues
  • Experience at a GPU cloud, inference provider, or AI infrastructure company

Hyperbolic is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

Staff Production Engineer

Zscaler · Santa Clara, CA +1 place

$152k–$190k/yrFull-timeHybrid

StrongChecked open todayPosted today

See more like this