Senior AI Test Engineer
TestPros · Remote · US only
$50–$80/hrPart-time

Bask HealthRemote · US onlyFull-time3+ years experiencePosted 9 days ago
Pay not listed
On Bask Health’s own careers site (Workable), checked open today.
Summarised from the posting by AI and can be wrong. The full description is the source.
Sign in and add your resume to see which of this job's requirements it shows.
Sign in to check your matchSaved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
From Bask Health’s careers site
About Bask Health
Bask Health is a software platform that enables entrepreneurs, doctors, and healthcare businesses to launch, manage, and grow direct-to-consumer telehealth businesses.
Our platform brings together patient experiences, clinical workflows, pharmacy operations, and commerce tools in one place.
We're an entrepreneurial, product-focused engineering team that values ownership, technical excellence, and building software that solves real problems. Engineers work closely with product teams and company leadership, take features from idea to production, and have a direct role in shaping the platform.
Engineering at Bask is AI-first and builder-led. We use tools like Cursor and Claude Code throughout development while maintaining high standards for reliability, testing, and software quality.
About the Role
We're looking for an AI Evaluation Engineer to help ensure that AI-powered features across Bask Health behave accurately, reliably, and consistently before and after they reach production.
Bask Health supports the technology behind direct-to-consumer telehealth businesses, connecting patient experiences, merchant operations, clinical workflows, pharmacy coordination, and commerce capabilities. As we introduce more AI-powered functionality, we need dependable ways to verify that these systems work as intended under realistic conditions.
You'll build and maintain evaluation workflows that test large language models, AI assistants, retrieval systems, and automated features against clearly defined product requirements.
This is a hands-on engineering role that combines AI evaluation, software testing, automation, and production quality assurance. You'll develop test scenarios, evaluation datasets, automated checks, and tools that help engineering teams identify problems before they affect users.
Your work may include testing AI-generated responses, validating structured outputs, evaluating tool-calling behavior, checking retrieval accuracy, and identifying regressions caused by changes to prompts, models, or application code.
You'll collaborate with applied AI engineers, machine learning engineers, backend engineers, product managers, and quality-focused engineering teams to define expected behavior and translate those expectations into repeatable evaluations.
You'll also help establish processes for reviewing AI failures, documenting recurring issues, and determining whether a feature meets its acceptance criteria. You'll work with engineers to reproduce problems and verify that fixes address the underlying issue.
Because Bask operates in healthcare-related environments, you'll pay particular attention to privacy, permissions, unsupported medical claims, and AI behavior that could interfere with approved clinical or pharmacy workflows. Evaluation systems should help verify that AI features remain within their intended operational boundaries.
This position emphasizes practical testing, thoughtful engineering, and continuous improvement. You'll contribute to evaluation standards while independently implementing and maintaining the tools and tests that support AI quality.
We're looking for an engineer who enjoys finding difficult edge cases, understands how AI behavior differs from traditional deterministic software, and wants to help build products that users can depend on.
What You'll Do
Requirements
Nice to Have
Something wrong with this job? Report it · This flag looks wrong

TestPros · Remote · US only
$50–$80/hrPart-time
StrongChecked open todayPosted 20 days ago

Skylight · Remote · US only
$150k–$170k/yrFull-time
StrongChecked open todayPosted 15 days ago
Sonatus · Remote · US only
$156k–$214k/yr
StrongChecked open todayPosted 16 days ago
Bright Vision Technologies · Remote · US only
$115k–$130k/yrFull-time
StrongChecked open todayPosted 19 days ago

Care Access · Remote · US only
$80k–$110k/yrFull-time
StrongChecked open todayPosted 19 days ago
GliaCell Technologies · Remote · US only
$180k–$260k/yrFull-time
StrongChecked open todayPosted 20 days ago