Data & AI jobs

AI Evaluators: Assessing A Shopping Assistant

TeracRemote · US onlyContractPosted 11 days ago

$50/hr listed by the employer

Key facts

Pay per year
≈ $104kEstimate for full-time hours.
Schedule
20+ hours a week

Looks real and checked open today.

On Terac’s own careers site (Ashby).

  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 5 checksShow fewer
How we check

What you’ll do

  • Review user interaction traces with an AI shopping assistant.
  • Identify logical failures, inaccuracies, and poor product recommendations.
  • Create structured rubrics and verifiers to evaluate response quality.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Experience in data evaluation, quality assurance, or AI training
  • Familiarity with e-commerce search and digital shopping experiences
  • Prior experience in prompt engineering (preferred)
  • Prior experience in complex data annotation (preferred)
  • Prior experience in software testing (preferred)

Skills

  • Strong analytical skills to spot subtle errors in text

Other requirements

  • Ability to commit to 20+ hours per week

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Full job description

From Terac’s careers site

WHAT WE'RE RESEARCHING

We're hiring AI evaluators to assess the accuracy and helpfulness of a new digital shopping assistant. This project focuses on understanding how well the system handles real-world e-commerce queries and where it falls short in its logic. Your analysis will directly feed into improving the underlying model and its response quality.

HOW IT WORKS

Show full descriptionShow less

You will review real interaction traces between users and the shopping assistant within our custom platform. As you analyze these conversations, you will pinpoint specific failures, logical errors, or unhelpful product recommendations. From there, you will create structured rubrics and verifiers to consistently judge future response quality. This is an ongoing remote engagement requiring 20+ hours per week.

WHO THIS IS FOR

This opportunity is ideal for quality assurance specialists, AI data evaluators, and e-commerce professionals with a strong eye for detail. We welcome applicants with prior experience in prompt engineering, complex data annotation, or software testing. You should be comfortable analyzing text interactions deeply and building structured evaluation frameworks from scratch.

WHAT YOU'LL DO

  • Review real user interaction traces with an AI shopping assistant
  • Identify logical failures, inaccuracies, or poor recommendations in the text
  • Create structured rubrics and verifiers to judge response quality
  • Commit to 20+ hours per week of evaluation work on our internal platform

WHO SHOULD APPLY

  • Experience in data evaluation, quality assurance, or AI training
  • Strong analytical skills with the ability to spot subtle errors in text
  • Familiarity with e-commerce search and digital shopping experiences
  • Ability to commit to a sustained workload of 20+ hours per week

COMPENSATION

$50 per hour

READY TO PARTICIPATE?

Start your paid interview now https://terac.com/interview/start/r/c5aab5ae-fa85-45ab-918d-a8b7cdf9f0ec?utm_source=ashby_listing_description

ABOUT TERAC

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

Learn more at terac.com https://terac.com or on YouTube at @jointerac https://www.youtube.com/@jointerac.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

TEMPLATE

Terac · Remote · US only +1 place

Pay not listedContract

Looks good2 to checkChecked open todayPosted 3 months ago

See more like this