Lead data and evaluation programs from scoping through production, quality assurance, delivery, and retrospective.
Turn evolving requests into specifications with milestones, owners, dependencies, and acceptance criteria.
Assess and improve data quality using quantitative signals and qualitative inspection, including task coverage, difficulty, and reward distributions.
Manage external vendors and improve delivery pipelines, production workflows, documentation, automation, and handoffs.
Summarised from the posting by AI and can be wrong. The full description is the source.
Requirements
Experience
At least 3 years of experience owning complex technical programs in AI/ML, data operations, support engineering, applied research, or a similarly cross-functional environment
Experience delivering projects as a technical program manager or similar role at a startup
Hands-on familiarity with RLHF, reinforcement-learning environments, agent evaluation, human-in-the-loop data pipelines, annotation workflows, or expert data collection
Experience managing external vendors and scaling delivery pipelines
Experience supporting research teams, AI labs, or technical enterprise customers with urgent, changing requirements
Skills
Strong quantitative and qualitative judgment, including the ability to assess data quality and representativeness through metrics and individual examples
Other requirements
Remote candidates should be able to maintain 70% to 80% time zone overlap with San Francisco or Singapore
Your match
Sign in and add your resume to see which of this job's requirements it shows.
Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Full job description
From Clera’s careers site
ABOUT THE ROLE
Own complex, time-sensitive data and evaluation programs for AI research teams and internal partners. This role combines technical program management, data operations, and vendor coordination to turn evolving requirements into reliable programs that improve model and agent evaluation.
WHAT YOU'LL DO
Lead data and evaluation programs from scoping and requirements gathering through production, quality assurance, delivery, and retrospective.
Turn ambiguous requests into clear specifications with milestones, owners, dependencies, and acceptance criteria.
Show full descriptionShow less
Improve data quality procedures using quantitative signals and qualitative inspection, including analysis of task coverage, difficulty, and reward distributions.
Manage external vendors throughout delivery and scale dependable delivery pipelines.
Improve production workflows, documentation, automation, and cross-functional handoffs.
Partner with research, platform, and marketplace teams to turn findings into product improvements.
WHAT WE'RE LOOKING FOR
At least 3 years of experience owning complex technical programs in AI/ML, data operations, support engineering, applied research, or a similarly cross-functional environment.
Experience delivering projects as a technical program manager or similar role at a startup, with the ability to bring structure to ambiguity and work independently.
Hands-on familiarity with RLHF, reinforcement-learning environments, agent evaluation, human-in-the-loop data pipelines, annotation workflows, or expert data collection.
Experience managing external vendors and scaling delivery pipelines.
Strong quantitative and qualitative judgment, including the ability to assess data quality and representativeness through metrics and individual examples.
Experience supporting research teams, AI labs, or technical enterprise customers with urgent, changing requirements, paired with clear written and verbal communication.
COMPENSATION & BENEFITS
Base salary range: $130,000 to $225,000 annually. Visa sponsorship is available.
LOCATION
On-site or remote. Primary location: United States. Remote candidates should be able to maintain 70% to 80% time zone overlap with San Francisco or Singapore.