Machine Learning Engineer jobs

Member of Technical Staff - Research, Inference

ModalNew York, NY +1 placeFull-timeOn-siteNew York, NY · San Francisco, CA · on-site

$150k–$350k/yr listed by the employer

Looks real. 2 things to check before you apply.

On Modal’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 4 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • To check: Wide pay range: top of the range is 2.33× the bottomAsk what the job pays at your level before you interview.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 7 checksShow fewer
How we check

What you’ll do

  • Own inference research projects from idea through results, targeting cost per token and tail latency.
  • Train custom speculators on production traffic and use findings to improve target models.
  • Work with customers to deploy and tune models, then feed lessons into research.
  • Collaborate with outside research labs on inference techniques.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Research-leaning or systems background in LLM inference, with work you can point to
  • A record of shipping research or systems that other people build on, in a lab or in industry
  • Ability to independently take a research bet from idea to result

Skills

  • Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling

Other requirements

  • Able to work in person in the NYC or San Francisco office

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Full job description

From Modal’s careers site

ABOUT US: AI NEEDS A NEW INFRASTRUCTURE LAYER. WE'RE BUILDING IT AT MODAL.

Every era of computing brought new workloads that previous infrastructure couldn't support: mainframes, databases, and the cloud. Each time, the company that rebuilt the layer underneath defined the decade. AI is no different, except it touches everything instead of one slice, and the window to build the layer underneath it is open right now.

Show full descriptionShow less

Our customers include category-defining companies like Ramp https://modal.com/blog/how-ramp-built-a-full-context-background-coding-agent-on-modal, Cognition, DoorDash, and Suno. They rely on Modal for instant GPU access, sub-second container starts, and native storage, so it's simple to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale.

We've passed $1B in annualized run rate, up 15x from a year ago. In October 2026 we raised a $750M Series D led by Accel at a $15.75B valuation.

Our team includes creators of popular open-source projects (e.g., Seaborn https://github.com/mwaskom/seaborn, Luigi https://github.com/spotify/luigi), academic researchers, international olympiad medalists, and experienced engineering and product leaders with decades of experience.

THE ROLE:

Most of the value of owning a model shows up at serving time. We’re building a platform that covers the whole life of an LLM (train it, deploy it, observe it), and inference is where teams feel the difference every day. We already run elastic inference, sandboxes, distributed volumes, and multi-node training, and we control the infrastructure underneath, so the serving stack is ours to shape rather than something we resell.

You will do hands-on inference research at Modal, working with the research lead to pick high-impact bets and owning them end to end. The bets that matter most are the ones that move cost per token and tail latency on the workloads our customers actually run.

WHAT YOU’LL WORK ON:

  • Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky serverless traffic, and whatever else the research agenda calls for
  • Train custom speculators against real production traffic and feed what you learn back into target models; acceptance length is the metric that decides the win
  • Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research
  • Carry and expand collaborations with outside research labs, for example:
  • our work with ZLab on DFlash https://modal.com/blog/spec-is-all-u-need, a speculator design built on KV injection and blockwise parallel drafting
  • our work with SGLang on specdec https://modal.com/blog/host-overhead-inference-efficiency and multimodal inference https://modal.com/blog/boosting-multimodal-inference-performance-by-greater-than-10-with-a-single-python-dictionary performance
  • our work on Flash Attention 4 kernels https://modal.com/blog/flash-attention-4-faster
  • Work with engineering to turn frontier serving techniques into products: primitives for disaggregation, fast weight refresh for models that keep training after deployment, observability for quality and latency in production, or even a next-generation inference engine
  • Help shape the research agenda

None of the above is prescriptive; your work will help guide our future.

YOU MAY BE A FIT IF:

  • You have a research-leaning or systems background in LLM inference, with work you can point to
  • You’re fluent in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling
  • You have a record of shipping research or systems that other people build on, whether in a lab or in industry
  • You have the drive to independently take a research bet from idea to result, working in the open with the rest of the team
  • You’re able to work in person in our NYC or San Francisco office

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

See more like this