Remote · US only jobs

Senior Python Developer (AI Evaluation & Benchmarking)

Lifted, an Upwork Company™Remote · US onlyContract

$80–$100/hr listed by the employer

Key facts

Pay per year
≈ $166k–$208kEstimate for full-time hours.
Schedule
10–39 hours a week
Experience
4+ years

Looks real. 1 thing to check before you apply.

On Lifted, an Upwork Company™’s own careers site (SmartRecruiters), checked open today.

  • To check: Posted by the employer 4 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteSmartRecruiters board
  • Checked: Checked open todaySmartRecruiters board
  • Checked: Pay range listedJob posting
  • Checked: No reposts foundSmartRecruiters board
  • Trust score 90 / 100 · Strong
See all 6 checksShow fewer
How we check

What you’ll do

  • Design and develop coding benchmarks and structured programming scenarios to evaluate AI models.
  • Evaluate AI-generated code for correctness, reliability, efficiency, and edge cases.
  • Build and maintain data pipelines for AI evaluation workflows.
  • Work with large codebases and multi-language software environments.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 4+ years of professional software engineering experience
  • Experience working at a high-growth technology company or top-tier software organization
  • Experience with CI/CD pipelines and automated testing frameworks such as pytest, Mocha, or JUnit
  • Experience with AI/ML evaluation, model benchmarking, or Generative AI (preferred)
  • Background in security engineering (preferred)
  • Significant contributions to open-source software projects (preferred)
  • Experience working with large-scale distributed systems or enterprise software platforms (preferred)

Skills

  • Expert-level proficiency in Python
  • Proficiency in at least one additional programming language such as JavaScript, Go, or C++
  • Strong understanding of software engineering best practices, debugging, and code quality

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Full job description

From Lifted, an Upwork Company™’s careers site

Company Description

An enterprise client is seeking experienced Senior Python Developers to help build the next generation of Artificial Intelligence systems. This opportunity support the client who is a leading AI platform that enables organizations to build intelligent applications through high-quality human feedback, AI evaluation, and model alignment.

Show full descriptionShow less

The selected consultants will contribute to AI research by designing programming benchmarks, evaluating AI-generated code, and helping improve the performance, reasoning, and reliability of frontier AI models. This is an excellent opportunity for experienced software engineers who enjoy solving complex technical problems while contributing to the future of Generative AI.

Job Description

This opportunity is ideal for senior software engineers with strong Python expertise who enjoy writing high-quality code, reviewing technical solutions, and working on AI-related projects.

What You'll Do:

  • Design and develop coding benchmarks used to evaluate frontier AI models.
  • Analyze AI-generated code for correctness, reliability, efficiency, and edge cases.Build and maintain scalable data pipelines that support AI evaluation workflows.
  • Create structured programming scenarios to test reasoning, debugging, and code quality.
  • Work with large codebases and multi-language software environments.
  • Collaborate with teams focused on improving how AI models understand, generate, and evaluate software.
  • Write clean, maintainable, and well-tested Python code following software engineering best practices.

Qualifications

Requirements:

  • 4+ years of professional software engineering experience (required).
  • Expert-level proficiency in Python.
  • Experience working at a high-growth technology company or top-tier software organization.
  • Proficiency in at least one additional programming language such as JavaScript, Go, C++, or similar.
  • Experience with CI/CD pipelines and automated testing frameworks such as pytest, Mocha, or JUnit.
  • Strong understanding of software engineering best practices, debugging, and code quality.
  • Excellent analytical and problem-solving skills.

Nice to Haves:

  • Experience with AI/ML evaluation, model benchmarking, or Generative AI.
  • Background in security engineering.
  • Significant contributions to open-source software projects.
  • Experience working with large-scale distributed systems or enterprise software platforms.

Additional Information

  • Fully remote contract opportunity.
  • Compensation ranges from $80–$100 USD per hour.
  • Expected workload is 10–39 hours per week, depending on project needs.
  • Weekly payments for approved work completed during the previous week.
  • Work volume may fluctuate throughout the duration of the engagement.

Hiring Process:

  • Submit your proposal.
  • Complete a short qualification form provided by the client.
  • Qualified candidates will receive an Upwork contract offer and onboarding instructions.
  • After accepting the contract, candidates will gain access to the client's evaluation.
  • Candidates who successfully pass the evaluation will be invited to a technical interview with the client.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

Senior Web Designer

Lifted, an Upwork Company™ · Remote · US only

$60–$150/hr · see postingContract

Wide pay rangeChecked open todayPosted 2 days ago

Senior FinOps Analyst

Lifted, an Upwork Company™ · Remote · US only

Contract

StrongPay not listedChecked open todayPosted 1 month ago

Copy of Senior Web Designer

Lifted, an Upwork Company™ · Remote · US only +3 places

$60–$150/hr · see postingContract

Wide pay rangeChecked open todayPosted 2 days ago

See more like this