Review AI-generated medical content for gaps in clinical reasoning, unsafe recommendations, and departures from medical guidelines.
Develop clinical task specifications, reference answers, benchmarks, and evaluation datasets.
Assess AI responses across diagnosis, treatment planning, patient management, guidelines, and specialty-specific scenarios.
Provide structured feedback and collaborate with research and engineering teams on evaluation standards and frameworks.
Summarised from the posting by AI and can be wrong. The full description is the source.
Requirements
Licenses and certifications
Active, unrestricted medical license in at least one US state
Current board certification in the candidate's respective specialty
Experience
Minimum of 4 years of independent clinical practice following residency; residency and fellowship training do not count
Demonstrated expertise in a defined, recognized clinical specialty or subspecialty
Professional progression into senior clinical responsibilities, such as Attending Physician, Medical Director, Division Chief, Associate Professor, Professor, or Chief Medical Officer
Proven ownership of clinical decisions and experience applying independent medical judgment in patient care
Practical experience using large language models (LLMs) in a professional setting and distinguishing clinically sound reasoning from medically incorrect responses
Experience in utilization management, clinical informatics, medical affairs, or clinical quality assurance (preferred)
Experience developing clinical guidelines, medical protocols, or evidence-based evaluation frameworks (preferred)
Previous involvement in medical research, clinical education, or multidisciplinary healthcare initiatives (preferred)
Familiarity with AI model evaluation, clinical benchmark development, or medical dataset quality assessment (preferred)
Education
MD or DO degree from an accredited medical school
Completed residency training in a recognized medical specialty
Other requirements
Currently reside in the San Francisco Bay Area or be willing to relocate before the engagement begins
Cover relocation expenses; relocation assistance is not provided
Your match
Sign in and add your resume to see which of this job's requirements it shows.
Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Full job description
From Weekday AI’s careers site
This role is for one of our clients
Compensation: $70 - $110 per hour
Location: Bay Area, California, United States Work Arrangement: Hybrid (On-site multiple days per week, as required) Employment Type: Full-Time, W-2 Work Commitment: 40 hours per week Initial Engagement: 6 months
Job Overview
We are seeking an experienced, board-certified physician to work directly with the research and program management teams at a leading AI lab, helping advance how next-generation AI models understand, reason about, and respond to real-world clinical challenges.
Show full descriptionShow less
As a Clinical AI Expert, you will bring your medical expertise and independent clinical judgment to evaluate AI-generated medical content, develop high-quality clinical benchmarks, and establish reference solutions that define what accurate, safe, and evidence-based medical reasoning looks like.
This role is designed for practicing physicians with substantial post-residency experience, strong specialization in a clinical discipline, and the ability to assess medical reasoning beyond surface-level correctness. You will collaborate closely with AI researchers and engineering teams to improve the quality, reliability, and clinical relevance of advanced AI systems.
This is a full-time W-2 employment opportunity with a hybrid work arrangement in the San Francisco Bay Area. You will receive client-issued accounts and equipment and work directly within the client's internal tools and research environment.
Requirements
Key Responsibilities
Clinical Quality Assurance: Review and evaluate clinical knowledge tasks and AI-generated medical responses, identifying gaps in clinical reasoning, missing considerations, unsafe recommendations, and deviations from established medical guidelines.
Instruction Specifications and Reference Solutions: Develop detailed task specifications, create accurate reference answers, and design clinical problems that reflect real-world medical practice and decision-making.
Clinical Benchmark Development: Design challenging, specialty-specific clinical tasks and evaluation datasets to measure the accuracy, depth, safety, and reliability of AI-generated medical reasoning.
Medical Content Evaluation: Assess AI responses across diagnosis, treatment planning, patient management, clinical guidelines, and specialty-specific medical scenarios.
Technical Feedback: Provide clear, structured, and actionable written feedback to help research teams identify weaknesses and improve model performance.
Research Collaboration: Work with AI researchers, engineers, and specialists from adjacent domains to establish consistent evaluation standards and translate clinical expertise into measurable assessment criteria.
Clinical Standards: Help develop medicine-specific evaluation frameworks, tools, and workflows that ensure AI outputs meet professional clinical standards.
Required Qualifications
MD or DO degree from an accredited medical school.
Completed residency training in a recognized medical specialty.
Minimum of 4 years of independent clinical practice following residency. Residency and fellowship training do not count toward this requirement.
Active, unrestricted medical license in at least one US state.
Current board certification in the candidate's respective specialty.
Demonstrated expertise in a defined clinical specialty, including but not limited to:
Internal Medicine
Oncology
Radiology
Emergency Medicine
General or Specialty Surgery
Psychiatry
Cardiology
Neurology
Pediatrics
Other recognized medical specialties and subspecialties
Clear professional progression into senior clinical responsibilities, such as Attending Physician, Medical Director, Division Chief, Associate Professor, Professor, or Chief Medical Officer.
Proven ownership of clinical decisions and experience applying independent medical judgment in patient care.
Practical experience using large language models (LLMs) in a professional setting, with the ability to distinguish clinically sound reasoning from plausible but medically incorrect responses.
Excellent written communication skills and the ability to explain complex clinical concepts through precise, well-structured feedback.
Ability to commit reliably to 40 hours per week for the initial six-month engagement.
Preferred Qualifications
Experience in utilization management, clinical informatics, medical affairs, or clinical quality assurance.
Experience developing clinical guidelines, medical protocols, or evidence-based evaluation frameworks.
Previous involvement in medical research, clinical education, or multidisciplinary healthcare initiatives.
Familiarity with AI model evaluation, clinical benchmark development, or medical dataset quality assessment.
Employment Details
Work Location: San Francisco Bay Area, California, United States.
Work Arrangement: Hybrid, requiring on-site collaboration with the client's team multiple days per week, as needed.
Employment Type: Full-time, W-2 employment.
Work Schedule: 40 hours per week.
Initial Duration: 6 months.
Equipment and Access: Client-issued accounts and equipment will be provided.
Work Environment: Direct collaboration with the client's research and program management teams, using internal tools and established workflows.
Engagement Structure: This is a structured, role-based employment opportunity rather than a freelance or project-based engagement.
Location and Relocation Requirements
Candidates must currently reside in the San Francisco Bay Area or be willing to relocate before the engagement begins.
Relocation expenses must be covered by the candidate.
Relocation assistance is not provided.
Candidates must be available for regular on-site collaboration with the client's team multiple days each week, as required.