QA & Test Engineer jobs

Lead Quality Engineer - R01573517

BrillioBangalore, IndiaPosted 8 days ago

Pay not listed

Key facts

Experience
4+ years
Degree
Bachelor's degree

Looks real. 1 thing to check before you apply.

On Brillio’s own careers site (Lever), checked open today.

  • To check: No pay range listedAsk for the pay range before you interview.
  • Checked: On the company's own careers siteLever board
  • Checked: Checked open todayLever board
  • Checked: No reposts foundLever board
  • Trust score 89 / 100 · Looks good

This job is outside the US, so these US-only checks are not applied: pay-transparency law, pay likely required by state law, US visa sponsorship wording.

See all 5 checksShow fewer
How we check

Summary, requirements and match are for US jobs for now.

Full job description

From Brillio’s careers site

Lead Quality Engineer

Primary Skills

  • Test Data Management, Jenkins, Regression Testing, Postman, Selenium, Functional Testing, BDD - Behavior Driven Development, SoapUI, System Integration testing, Application Testing, Pixel Testing, Cross Browser Testing, Zephyr, TDD - Test Driven Development, Accessibility Testing (JAWS), Visual Testing, Cucumber

Specialization

  • Quality Engineering - Product Applications & Microservices: Lead Quality Engineer

Job requirements

Show full descriptionShow less
  • Experience Range: with at least 4 to 6 years of experience in quality engineering, including hands-on testing and evaluation of AI-powered product applications and microservices Key Responsibilities: 1. Develop and execute robust evaluation frameworks for AI agents, focusing on Responsible AI and safety metrics such as bias, toxicity, and alignment failures 2. Design synthetic testing environments and simulation strategies to stress-test agents under diverse real-world conditions 3. Implement adversarial testing, red-teaming, and prompt injection methodologies to identify vulnerabilities and ensure agent robustness 4. Integrate evaluation pipelines into CI/CD workflows for continuous quality assurance and rapid iteration 5. Collaborate with policy and legal teams to ensure agent evaluations meet regulatory compliance and ethical standards 6. Analyze evaluation results and provide actionable insights to development teams, prioritizing safety and risk mitigation 7. Configure and execute automated test suites using Python and relevant AI/ML testing libraries (e.g., PyTest, DeepEval, Ragas, Giskard) 8. Manage bug detection, documentation, and tracking using Jira, focusing on performance degradations and alignment failures Required Skills: 1. Python programming 2. PyTest 3. DeepEval 4. Ragas 5. Giskard 6. Langsmith 7. Google Agent Development Kit (ADK) 8. Vertex AI 9. Safety evaluations (red teaming, adversarial testing) 10. Prompt injection testing 11. API testing for AI/ML systems 12. Data analysis with Pandas and SQL Preferred Skills: 1. Gemini models 2. Hugging Face 3. Garbage-in/Garbage-out (GIGO) testing tools 4. Data poisoning techniques 5. AI ethics principles and policy background 6. Visualization tools for evaluation results Desired Qualifications: 1. Bachelor's degree in Computer Science, Artificial Intelligence, Information Technology, or Software Engineering 2. ISTQB Foundation Level certification or Certified Software Tester (CST) 3. Certification in AI/ML testing or Responsible AI (e.g., Google Cloud Certified - Professional Machine Learning Engineer)

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

Product Quality Engineer

Third Wave Systems · Eden Prairie, MN

$85k–$105k/yrFull-timeOn-site

StrongLast seen open yesterdayPosted 8 days ago

Product Validation Lead

PaceMate™ · Remote · US only

$130k–$160k/yrFull-time

StrongChecked open todayPosted 9 days ago

Senior QA Engineer

Rain Technologies Inc. · New York City, NY

$165k–$185k/yrFull-timeHybrid

StrongChecked open todayPosted 10 days ago

See more like this