Machine Learning Engineer jobs

Research Engineer

AndoSan Francisco, CAFull-timeOn-site

Pay not listed

Looks real. 2 things to check before you apply.

On Ando’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 4 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • To check: No pay range listed, though one is likely required in CAAsk for the pay range before you interview. The law here likely requires one.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: No reposts foundAshby board
  • Trust score 80 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Mine production workspace data, with consent and redaction safeguards, to create benchmarks and labeled datasets.
  • Design label schemas, coordinate rigorous labeling, and build evaluation harnesses.
  • Run experiments on offline data and carry successful work into production features.
  • Evaluate vendors and research teams for evaluation, observability, and continual-learning tools.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Strong applied research background with depth in model evaluation, benchmarking, and/or failure analysis
  • Work samples or code demonstrating skills such as evaluation frameworks, benchmark suites, failure-analysis reports or tooling, or labeling infrastructure
  • Comfort working with incomplete, ambiguous real workspace data and edge cases

Skills

  • Familiarity with simulation (preferred)
  • Familiarity with human-in-the-loop evaluation (preferred)
  • Familiarity with Cartridges and related context/memory-compression work (preferred)
  • Familiarity with memory for multi-party, long-running settings (preferred)
  • Familiarity with agent observability standards, such as setting up Langsmith or similar (preferred)

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Dental & vision
Dental and vision insurance
Equity
Generous equity grant vested over 4 years
Wellness & mental health
Free Equinox membership and other health perks

Also: health insurance

Full job description

From Ando’s careers site

Ando is a messaging platform where AI agents take on work alongside their human teammates. We’re rebuilding Slack from the ground up around two core ideas: durable memory and agents as first-class participants.

We have real, longitudinal, multi-party workspace data, and human / agent users whose behavior tells you whether the systems you're designing are actually meaningfully improving. Working with live business communication means permissions, redaction, and security are things we have to think about as well. If you want to have your research come into contact with reality, Ando is the place for it.

Some concrete research problems on our plate right now:

Show full descriptionShow less
  • Proactivity: An agent embedded in a team's channels has to decide, message by message, whether to ignore, quietly track, or intervene. Evaluating that judgment means building benchmarks where the ground truth includes silence. Existing agent benchmarks are almost entirely reactive.
  • Memory: What should a workspace agent remember across weeks and months of participation, in what representation, and how do you measure whether memory is helping versus hurting agent usefulness?
  • Continual Learning: How much does basic memory, retrieval, and context impact agent performance, vs where do we genuinely need RL and continual learning?

What you’ll be doing at Ando

You'll be one of the first members of a small research team, working close to both the data and the product.

  • Build evaluation from real data - Mine production workspace data (carefully, with consent and redaction pipelines you'll help design) into benchmarks and labeled datasets. Design label schemas, run labeling with real inter-rater rigor, and build the harnesses that make expert judgment cheap to capture and hard to corrupt.
  • Run experiments that ship - Initial work happens on offline workspace data; the destination is production systems used by every Ando customer. The distance between "the benchmark improved" and "the feature shipped" should be weeks and you'll own both ends.
  • Decide what we build versus who we partner with - We won't do everything in-house. Part of the job is evaluating frontier vendors and research teams (eval infrastructure, observability, continual-learning tooling) and choosing who we build with.
  • Publish when we have something real - We expect the team to publish as we make meaningful progress. But ideas are not Ando's moat; execution, product quality, and customer experience are. Research here is in service of customers first, and the publications will be better for it: they'll describe things that actually worked on real data.

WHAT WE'RE LOOKING FOR

  • Strong applied research background, with depth in model evaluation, benchmarking, and/or failure analysis. You've built evals you trusted enough to make decisions with.
  • Evidence over credentials. Work samples or code that demonstrate the skills: eval frameworks, benchmark suites, failure-analysis reports or tooling, labeling infrastructure. Show us something you built to find out whether a system actually worked.
  • Strong technical communication. You can explain complex ideas simply and hold high-bandwidth, generative technical conversations with researchers and with our product team.
  • Comfort with mess. Real workspace data is incomplete, ambiguous, and full of edge cases that break clean abstractions. You treat that as signal, not noise.
  • Bonus: familiarity with simulation (Park et al. https://arxiv.org/abs/2304.03442), human-in-the-loop evaluation (Scale HIL leaderboard https://labs.scale.com/leaderboard/hil), Cartridges https://arxiv.org/abs/2506.06266 and related context/memory-compression work, memory for multi-party long-running settings, or agent observability standards (setting up Langsmith or similar).

Hiring process

1. 30 min intro call

2. Technical conversation - walk us through an app you've shipped

3. Paid take-home or IRL work trial

Benefits

  • Free Equinox membership & other health perks
  • Generous equity grant vested over 4 years
  • Health, dental, vision insurance

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

AI Research Engineer III

Agility Robotics · Fremont, CA +2 places

$166k–$260k/yrFull-timeHybrid

StrongChecked open todayPosted 11 days ago

Machine Learning Engineer

Gotion, Inc. · Fremont, CA

$110k–$135k/yrFull-timeHybrid

StrongChecked open todayPosted 19 days ago

Machine Learning Engineer

Sift · San Francisco, CA +1 place

$170k–$240k/yrFull-timeHybrid

StrongChecked open todayPosted 25 days ago

See more like this