Remote · US only jobs

Member of Technical Staff, Agents

Clera (agency)Remote · US onlyFull-timePosted 13 days ago

$200k–$300k/yr listed by the employer

Key facts

Experience
3+ years
US visa
No US visa sponsorship

This job is closed.

The employer took it down 7 days ago, after 5 days open. Here are similar open jobs.

  • Checked: On the company's own careers siteAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 4 checksShow fewer
How we check

Similar open jobs

Member of Technical Staff

Roboflow · San Francisco, CA

$150k–$300k/yr · see postingFull-timeOn-site

2 to checkChecked open todayPosted 2 months ago

See more like this

What you’ll do

  • Build agents that complete delegated work for users end to end.
  • Define success measures and maintain representative evaluations for agent tasks.
  • Develop and validate agent capabilities before release.
  • Investigate production failures, fix underlying causes, and build systems to detect failure patterns.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • At least 3 years of data science or machine learning experience building evaluation systems, metrics, or quality measurement for products
  • Experience designing evaluation frameworks, datasets, measurable criteria, and appropriate metrics
  • A track record delivering production AI, machine learning, agent, or agent-evaluation systems for real users
  • Experience improving agent reliability, owning systems end to end, and measuring results in production
  • Demonstrated progression and sustained experience in a role, with the ability to work through ambiguity and ship quickly
  • Experience with fine-tuning, post-training, or reinforcement learning (preferred)

Skills

  • Production-quality Python and SQL skills, including experience analyzing large datasets and building automated pipelines
  • Familiarity with statistics, experiment design, and realistic end-to-end simulations
  • Experience with Python, Django, React, and TypeScript
  • Familiarity with evaluation tools such as Braintrust or Raindrop (preferred)

Other requirements

  • Able to work on-site in Palo Alto four days a week
  • Must not require visa sponsorship

Full job description

From Clera’s careers site

ABOUT THE ROLE

Join a small, senior team building autonomous AI assistants for executive workflows. You will develop agent capabilities and evaluation systems that help ensure real-world tasks are completed reliably, with measurable quality.

WHAT YOU'LL DO

  • Build agents that complete delegated work end to end for real users.
  • Define reliable measures of success for agent tasks and maintain representative evaluations.
  • Develop new agent capabilities and validate them before they reach users.
  • Investigate production failures and deliver fixes that address underlying causes.
  • Build systems to detect known failure patterns and emerging issues at scale.
  • Improve agent recovery when tools or websites fail.
Show full descriptionShow less
  • Create repeatable methods for engineers to measure the impact of agent changes.

WHAT WE'RE LOOKING FOR

  • At least 3 years of data science or machine learning experience building evaluation systems, metrics, or quality measurement for products.
  • Production-quality Python and SQL skills, including experience analyzing large datasets and building automated pipelines.
  • Experience designing evaluation frameworks, datasets, measurable criteria, and appropriate metrics.
  • A track record delivering production AI, machine learning, agent, or agent-evaluation systems for real users.
  • Experience improving agent reliability, owning systems end to end, and measuring results in production.
  • Familiarity with statistics, experiment design, and realistic end-to-end simulations; experience with fine-tuning, post-training, or reinforcement learning is valuable.
  • Experience with Python, Django, React, and TypeScript; familiarity with evaluation tools such as Braintrust or Raindrop is useful.
  • Demonstrated progression and sustained experience in a role, with the ability to work through ambiguity and ship quickly.

COMPENSATION & BENEFITS

Base salary is $200,000 to $300,000 USD annually. Visa sponsorship is not available.

LOCATION

On-site in Palo Alto, California, with four days a week in the office.

Something wrong with this job? Report it · This flag looks wrong