Machine Learning Engineer jobs

Member of Technical Staff, LLM & Production

HarkSan Jose +1 placeFull-timePosted 1 month ago

$180k–$450k/yr listed by the employer

Above the typical $281k a year for staff-level Machine Learning Engineer jobs in CaliforniaHow we compare

How we compare. Typical is the median yearly pay of 158 staff-level Machine Learning Engineer jobs in California that list pay, from the employers' own postings. About means within 5%.

Key facts

Where
San Jose · San Jose, CA
Experience
3+ years

Looks real. 1 thing to check before you apply.

On Hark’s own careers site (Greenhouse), checked open today.

  • To check: Wide pay range: top of the range is 2.5× the bottomAsk what the job pays at your level before you interview.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: Checked open todayGreenhouse board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundGreenhouse board
  • Trust score 100 / 100 · Strong
See all 6 checksShow fewer
How we check

What you’ll do

  • Build and maintain end-to-end LLM evaluation suites, including task specifications, grading logic, and dashboards.
  • Operate the evaluation harness as a production service with sandboxed execution, tool calling, deterministic replay, and caching.
  • Own data pipelines for collection, filtering, deduplication, labeling, synthetic data generation, and quality checks.
  • Build and stabilize RL training environments, including reward functions, simulators, verifiers, and task generators.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 3+ years in software engineering, including 1–2 years hands-on with LLM training, evaluation, or data infrastructure
  • Real distributed-systems experience
  • Built evaluation infrastructure, including designing tasks, graders, and the surrounding plumbing
  • Comfortable working in ambiguity next to researchers and making half-formed ideas measurable
  • Agentic evaluation experience, including tool use, code execution, sandbox environments, or multi-turn tasks (preferred)
  • Hands-on experience with training frameworks and inference stacks such as PyTorch, JAX, vLLM, SGLang, or in-house equivalents (preferred)
  • Experience running data-labeling operations or collecting human preferences at scale (preferred)
  • Contributions to open-source evaluation frameworks or public benchmarks (preferred)
  • Previous work on verifiers, code sandboxes, or simulation environments (preferred)

Skills

  • Deep Python fluency
  • Working knowledge of SFT and RL for language models
  • Write tests and insist on reproducibility

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Full job description

From Hark’s careers site

About Hark

Hark is an artificial intelligence company building advanced, personalized intelligence. One that is proactive, multimodal, and capable of interacting with the world through speech, text, vision, and persistent memory.

We're pairing that intelligence with next-generation hardware to create a universal interface between humans and machines. While today's AI largely operates through chat boxes and decade-old devices, Hark is focused on what comes next: agentic systems that interact naturally with people and the real world.

Show full descriptionShow less

To get there, we're developing multimodal models and next-generation AI hardware together - designed from the ground up as a single, unified interface for a new era of intelligent systems.

About the Role

The Omni team at Hark is building the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, audio, and vision. Our goal is to create seamless, real-time multimodal intelligence that powers intuitive and immersive user experiences.

We're looking for an engineer who sits at the intersection of model research and production systems. Together with our researchers, you'll decide what "better" means for a model and how to prove it — then build the infrastructure that makes those proofs reliable and repeatable: evaluation harnesses, data pipelines, and model training that hold up at scale and that the whole team can build on.

This is a hands-on, individual-contributor position. The systems that judge whether a model improved, and the environments that drive that improvement, will be yours.

Responsibilities

  • Build and own end-to-end evaluation suites for LLM capabilities: task specs, grading logic, and the dashboards researchers check every day.
  • Run the eval harness as a production service — sandboxed execution, tool calling, multi-turn and agentic tasks, deterministic replay, and caching — so a full evaluation pass is cheap and gives the same answer twice.
  • Own the data pipeline from source to training: collection, filtering, dedup, labeling workflows, synthetic data generation, and the quality gates that catch contamination and distribution drift before anything reaches a run.
  • Build RL training environments — reward functions, simulators, verifiers, task generators — and keep them stable as they fan out across many workers.
  • Work side by side with researchers: turn a hypothesis into a runnable eval or environment within days, then help make sense of what comes back.
  • Pull capabilities out of the product layer and into the model. Find where product teams lean on long prompt instructions or hand-written skills, build the evals and environments that teach the model to do those things natively, and retire the workarounds once it can.
  • Measure everything: eval regressions, environment health, throughput, cost — and make it obvious when a metric moved for the wrong reason.
  • Raise the bar for research code across the team: tests, versioned datasets and evals, and clean boundaries between training, inference, and evaluation.

Requirements

  • 3+ years in software engineering, including 1-2 years hands-on with LLM training, evaluation, or data infrastructure.
  • Deep Python fluency and real distributed-systems experience: you've chased failures across a fleet of machines, dealt with unreliable workers, and know where the compute budget actually goes.
  • You've built evaluation infrastructure, not just run benchmarks — designing tasks, graders, and the plumbing around them.
  • Working knowledge of SFT and RL for language models.
  • Production instincts: you write tests, insist on reproducibility, and treat research tooling as something others depend on — because they do.
  • Comfortable in ambiguity next to researchers: handed a half-formed idea, you can make it measurable.

Bonus Qualifications

  • Agentic evaluation experience: tool use, code execution, sandbox environments, multi-turn tasks.
  • Hands-on with training frameworks and inference stacks (e.g., PyTorch, JAX, vLLM, SGLang, or in-house equivalents).
  • Experience running data-labeling operations or collecting human preferences at scale.
  • Contributions to open-source evaluation frameworks or public benchmarks.
  • Previous work on verifiers, code sandboxes, or simulation environments.

Compensation

The US base salary range for this full-time position is between $180,000 - $450,000 annually.

The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.

Something wrong with this job? Report it

Similar jobs nearby

Applied Researcher

River AI Inc. · Palo Alto, CA

$200k–$420k/yr · see posting

Wide pay rangeChecked open todayPosted 15 days ago

See more like this