QA & Test Engineer jobs

Test Engineer-AI/LLM

OPPO US Research CenterPalo Alto, CAFull-time

$100k–$200k/yr listed by the employer

Key facts

Experience
1+ years
Degree
Bachelor's degree or equivalent

Looks real. 2 things to check before you apply.

On OPPO US Research Center’s own careers site (Workable), checked open today.

  • To check: Posted by the employer 16 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • To check: Wide pay range: top of the range is 2× the bottomAsk what the job pays at your level before you interview.
  • Checked: On the company's own careers siteWorkable board
  • Checked: Checked open todayWorkable board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundWorkable board
  • Trust score 75 / 100 · Looks good
See all 7 checksShow fewer
How we check

What you’ll do

  • Design and run LLM performance tests and end-to-end tests of generative AI applications, including APIs, data pipelines, and user interfaces.
  • Evaluate model outputs for accuracy, factuality, coherence, bias, and safety; analyze failures, edge cases, and adversarial inputs.
  • Validate fine-tuned models and prompt variations, benchmark performance, and help optimize prompts.
  • Build or use automated testing and evaluation tools, including workflows for synthetic test data and prompt variations.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Full-time: 1+ years of experience in software testing, data science, or ML validation, with exposure to AI/ML systems
  • Full-time: Hands-on experience evaluating LLMs in production environments
  • Full-time: Experience with Git and agile development methodologies
  • Contractor: 6+ months of experience in software QA, data labeling, LLM evaluation, or ML testing projects
  • Experience with automated evaluation tools such as LangChain or TruLens, or LLM-specific test suites (preferred)
  • Background in statistics, quantitative analysis, or data visualization for test insights (preferred)
  • Contributions to AI safety/ethics initiatives or open-source LLM evaluation projects (preferred)
  • Experience testing mobile-integrated AI solutions on Android or iOS (preferred)

Education

  • Full-time: Bachelor’s degree in Computer Science, Data Science, Engineering, or a related technical field, or equivalent practical experience
  • Contractor: Bachelor’s degree or equivalent work experience in a technical field
  • Master’s degree in AI, Machine Learning, or a related field (preferred)

Skills

  • Full-time: Proficiency in Python and testing frameworks such as PyTest or Selenium
  • Full-time: Familiarity with GCP, Azure, or AWS and MLOps tooling such as MLflow or Weights & Biases
  • Contractor: Basic Python proficiency, especially for data processing and automation tasks
  • Contractor: Familiarity with LLMs and prompt-based outputs
  • Contractor: Familiarity with tools such as Jupyter, Postman, or testing dashboards
  • Expertise in prompt engineering, LLM fine-tuning (such as LoRA or RLHF), or optimization techniques (preferred)
  • Knowledge of data pipelines, SQL/NoSQL databases, and API testing such as Postman (preferred)

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Equity
Long-term incentives for the full-time position; details not specified
Bonus
Bonus for the full-time position; amount not specified

Full job description

From OPPO US Research Center’s careers site

OPPO US Research Center is seeking a full-time meticulous and innovative AI/LLM Test Engineer to join our cutting-edge AI team. In this critical role, you will evaluate the performance, reliability, and safety of Large Language Models (LLMs) in real-world product scenarios and test end-to-end generative AI solutions. Your work will directly shape how users experience AI-powered features by ensuring robustness, accuracy, and alignment with product goals. This is a unique opportunity to pioneer testing methodologies for next-generation AI systems at the forefront of technology.

Show full descriptionShow less

We are also seeking a Contractor based LLM Evaluation & QA Engineer to support the testing and validation of large language model (LLM)-powered applications. You will help implement test strategies, execute evaluation workflows, and assist in model performance validation across diverse generative AI use cases.

This contract role is ideal for someone with hands-on experience in AI/ML evaluation, QA engineering, or data analysis who wants to deepen their exposure to generative AI systems.

Requirements

Full-time position requirement:

Core Testing & Evaluation

  • Design and execute performance tests for LLMs across diverse product use cases (e.g., chatbots, content generation etc.).
  • Develop automated test frameworks to evaluate LLM outputs for accuracy, bias, safety, and coherence.
  • Conduct end-to-end testing of integrated generative AI solutions, including APIs, data pipelines, and user interfaces.

Optimization & Validation

  • Collaborate with ML engineers to validate fine-tuned models and optimize prompts for target scenarios.
  • Analyze model failures, edge cases, and adversarial inputs to identify risks and improvement areas.
  • Benchmark LLM performance against industry standards and product-specific KPIs.

Collaboration & Quality Assurance

  • Partner with product, engineering, and research teams to define test requirements and acceptance criteria.
  • Document defects, performance metrics, and test results to drive data-driven improvements.
  • Advocate for AI ethics and safety through rigorous testing of fairness, bias mitigation, and content moderation.

Innovation & Tooling

  • Build scalable tools for synthetic test data generation, prompt variation testing, and automated evaluation workflows.
  • Stay current with advancements in generative AI testing, including red-teaming techniques and evaluation frameworks (e.g., HELM, Dynabench).
  • Propose novel testing strategies for emerging challenges (e.g., hallucinations, context drift).

Basic Qualifications:

  • Bachelor’s degree in Computer Science, Data Science, Engineering, or a related technical field, or equivalent practical experience.
  • 1+ years of experience in software testing, data science, or ML validation, with exposure to AI/ML systems.
  • Proficiency in Python and testing frameworks (e.g., PyTest, Selenium).
  • Hands-on experience evaluating LLMs in production environments (e.g., GPT, Claude, Llama, Gemini).
  • Strong analytical skills for dissecting model behavior, statistical performance, and failure modes.
  • Familiarity with cloud platforms (GCP, Azure, or AWS) and MLOps tooling (e.g., MLflow, Weights & Biases).
  • Experience with version control (Git) and agile development methodologies.

Preferred Qualifications:

  • Master’s degree in AI, Machine Learning, or a related field.
  • Expertise in prompt engineering, LLM fine-tuning (e.g., LoRA, RLHF), or optimization techniques.
  • Experience with automated evaluation tools (e.g., LangChain, TruLens) or LLM-specific test suites.
  • Knowledge of data pipelines, SQL/NoSQL databases, and API testing (e.g., Postman).
  • Background in statistics, quantitative analysis, or data visualization for test insights.
  • Contributions to AI safety/ethics initiatives or open-source LLM evaluation projects.
  • Experience testing mobile-integrated AI solutions (Android/iOS).

Contractor position requirements:

Testing & Evaluation Support:

  • Execute pre-defined performance tests for LLMs across various tasks (e.g., summarization, Q&A, chatbot flows).
  • Run scripted evaluations to assess outputs for factuality, coherence, and safety.
  • Perform manual and automated test execution on APIs and LLM-integrated user interfaces.

Prompt & model validation:

  • Assist ML engineers in evaluating prompt variations and prompt-tuning outcomes.
  • Log and analyze failure cases, anomalies, and edge cases based on provided guidelines.

Collabration & Documentation

  • Work with QA leads, product managers, and ML engineers to understand test goals and criteria.
  • Report defects, compile evaluation summaries, and maintain testing logs.

Tooling & Antomation:

  • Use existing internal tools or frameworks to automate test runs and result collection.
  • Contribute to prompt generation, input templating, or result tagging processes.

Basic Qualifications:

  • Bachelor's degree or equivalent work experience in a technical field (e.g., Computer Science, Engineering, Data Science).
  • 6+ months experience in software QA, data labeling, LLM evaluation, or ML testing projects.
  • Basic Python proficiency, especially for data processing and automation tasks.
  • Familiarity with LLMs (e.g., GPT, Claude, Gemini) and prompt-based outputs.
  • Comfortable working with tools like Jupyter, Postman, or testing dashboards.
  • Detail-oriented with good documentation habits.

Contractor Details:

  • Duration: Long term
  • Rate: Commensurate with experience
  • Conversion Opportunity: High-performing contractors may be considered for full-time roles

Benefits

The US base salary range for this full-time position is $100,000-$200,000 + bonus + long term incentives benefits. Our salary ranges are determined by role, level, and location.

OPPO is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

Test Engineer

Foxconn Industrial Internet - FII · San Jose, CA

$85k–$110k/yrFull-timeOn-site

StrongChecked open todayPosted 20 days ago

Test Automation Engineer

Blue River Technology · Santa Clara, CA

$122k–$183k/yrFull-timeHybrid

StrongChecked open todayPosted 1 month ago

Staff Engineer, Firmware Test

Samsung Semiconductor · San Jose, CA

$163k–$253k/yrOn-site

StrongOpen 2 monthsChecked open todayPosted 2 months ago

See more like this