Remote · US only jobs

Member of Technical Staff, Agents

Clera (agency)Remote · US onlyFull-timePosted 7 days ago

$200k–$300k/yr listed by the employer

Key facts

Experience
3+ years
US visa
No US visa sponsorship

This job is closed.

The employer took it down 2 days ago, after 5 days open. Here are similar open jobs.

  • Checked: On the company's own careers siteAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 4 checksShow fewer
How we check

Similar open jobs

Member of Technical Staff

Roboflow · San Francisco, CA

$150k–$300k/yr · see postingFull-timeOn-site

2 to checkChecked open todayPosted 2 months ago

Member of Technical Staff, Product

Raydar (agency) · San Francisco, CA

$200k–$400k/yr · see postingFull-timeOn-site

Wide pay rangeLast seen open yesterdayPosted 25 days ago

See more like this

What you’ll do

  • Build agents that complete delegated work end to end.
  • Define success criteria and evaluate agent performance across tasks.
  • Develop and evaluate agent capabilities before release.
  • Investigate production failures and implement durable fixes.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • At least 3 years of data science or machine learning experience building evaluation systems, metrics, or product quality measurement
  • Experience designing evaluation frameworks, representative datasets, meaningful metrics, and realistic end-to-end simulations
  • A track record delivering production AI, ML, agent, or agent-evaluation systems for real users and measuring their impact
  • Experience improving agent reliability, owning systems end to end, and using statistics and experiment design to evaluate changes
  • Experience with Django, React, and TypeScript
  • Demonstrated progression and sustained experience in a role
  • Familiarity with post-training, fine-tuning, or reinforcement learning (preferred)

Skills

  • Production-quality Python and SQL skills for analysis, large datasets, and automated pipelines

Other requirements

  • On-site in Palo Alto, California
  • Visa sponsorship is not available

Full job description

From Clera’s careers site

ABOUT THE ROLE

Join a small, senior team building autonomous AI agents for executive workflows. You will develop agent capabilities and evaluation systems that help ensure real-world tasks are completed reliably, with measurable improvements reaching users quickly.

WHAT YOU'LL DO

  • Build agents that complete delegated work end to end, without requiring users to check each step.
  • Define measurable success criteria and trusted ways to assess agent performance across tasks.
  • Develop and evaluate new agent capabilities before releasing them to users.
  • Investigate production failures and deliver durable fixes based on their underlying causes.
Show full descriptionShow less
  • Build systems to detect known failure patterns and less predictable issues at scale, using tools such as Braintrust and Raindrop.
  • Improve agent recovery when tools or websites fail.
  • Create repeatable evaluation workflows that let engineers measure whether changes improve agent performance.

WHAT WE'RE LOOKING FOR

  • At least 3 years of data science or machine learning experience building evaluation systems, metrics, or product quality measurement.
  • Production-quality Python and SQL skills for analysis, large datasets, and automated pipelines.
  • Experience designing evaluation frameworks, representative datasets, meaningful metrics, and realistic end-to-end simulations.
  • A track record delivering production AI, ML, agent, or agent-evaluation systems for real users and measuring their impact.
  • Experience improving agent reliability, owning systems end to end, and using statistics and experiment design to evaluate changes.
  • Familiarity with post-training, fine-tuning, or reinforcement learning is useful.
  • Experience with Django, React, and TypeScript; the role also uses Python, SQL, Braintrust, and Raindrop.
  • Demonstrated progression and sustained experience in a role, with comfort working in a fast-moving, ambiguous environment.

COMPENSATION & BENEFITS

Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.

LOCATION

On-site in Palo Alto, California.

Something wrong with this job? Report it · This flag looks wrong