Research Engineer, Benchmarks
Clera (agency)SingaporeFull-timeOn-site2+ years experiencePosted 10 days ago
$100k–$170k/yr listed by the employer
This job is closed.
The employer took it down 4 days ago, after 6 days open.
- Checked: On the company's own careers siteAshby board
- Checked: Pay listedJob posting
- Checked: No reposts foundAshby board
- Trust score 89 / 100 · Looks good
This job is outside the US, so these US-only checks are not applied: pay-transparency law, pay likely required by state law, US visa sponsorship wording.
See all 4 checksShow fewer
Summary, requirements and match are for US jobs for now.
Full job description
From Clera’s careers site
ABOUT THE ROLE
Join a small, technical AI evaluation team building rigorous benchmarks for frontier AI agents. You will own the design and implementation of domain-specific evaluations that help research teams and customers understand how agents perform on realistic workflows.
WHAT YOU'LL DO
- Design, implement, and maintain internal benchmarks for evaluating frontier agents on domain-specific tasks.
- Partner with subject-matter experts to translate real-world workflows into benchmark tasks and evaluation criteria.
- Build reliable infrastructure to run models and agents against benchmark tasks at scale.
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
Show full descriptionShow less
- Validate whether benchmark results align with real-world evaluations and user needs.
- Write clear technical documentation and reports explaining benchmark methods and results.
WHAT WE'RE LOOKING FOR
- Two to four years of experience in software engineering, machine learning engineering, or research engineering, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments.
- Technical education or strong fundamentals in a technical field.
- Proficiency with Python, Docker, and Linux for research or production infrastructure.
- Hands-on experience designing and operating evaluations or environments for AI agents or large language models.
- Experience building reliable model or agent execution infrastructure, developing evaluation metrics or statistical analyses, and validating benchmark performance.
- Ability to work with subject-matter experts and understand workflows across technical or business domains.
- Strong technical writing, attention to detail, independent problem-solving, and comfort in an early-stage environment.
- Published research or technical writing on AI benchmarks, evaluation methods, or model failure modes is valued. Reinforcement learning pipelines, data generation, and agent evaluation experience are also relevant.
COMPENSATION & BENEFITS
Base salary: $100,000 to $170,000 USD annually. Visa sponsorship is available.
LOCATION
On-site in Singapore, Singapore.
Something wrong with this job? Report it · This flag looks wrong
