Machine Learning Engineer jobs

Senior Staff Machine Learning Engineer, Data & Eval

AirbnbRemote · US only

$244k–$305k/yr listed by the employer

About the typical $275k a year for staff-level remote Machine Learning Engineer jobsHow we compare

How we compare. Typical is the median yearly pay of 74 staff-level remote Machine Learning Engineer jobs that list pay, from the employers' own postings. About means within 5%.

Key facts

Experience
2+ years
Degree
PhD or equivalent

Looks real. 1 thing to check before you apply.

On Airbnb’s own careers site (Greenhouse), checked open today.

  • To check: Posted by the employer 20 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: Checked open todayGreenhouse board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundGreenhouse board
  • Trust score 75 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Set evaluation strategy and success metrics for GenAI systems, connecting offline evaluation to business and customer experience outcomes.
  • Build evaluation frameworks using golden sets, synthetic data, automated regressions, rubric-based grading, and LLM-as-judge where appropriate.
  • Design data flywheels for feedback collection, data quality, labeling, dataset versioning, and governance.
  • Develop and productionize dataset, monitoring, evaluation-at-scale, and continuous-testing pipelines.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 10+ years building, testing, and shipping ML/AI systems end-to-end
  • 2+ years of experience with GenAI/LLM systems in production
  • 5+ years leading large, ambiguous technical initiatives as a senior IC
  • Experience building data pipelines and quality systems, including labeling workflows, dataset curation, versioning, monitoring, and governance
  • Experience applying ML/AI to customer support workflows, such as agent assist, classification/routing, resolution recommendation, or QA (preferred)
  • Experience building robust evaluation platforms for agent behavior validation, safety/guardrails, and continuous improvement (preferred)
  • Ability to take evaluation and data flywheel work from incubation to production while iterating quickly and maintaining scientific rigor (preferred)

Education

  • PhD in Computer Science, Mathematics, Statistics, or a related technical field, or equivalent practical experience

Skills

  • Deep expertise in evaluation methodology, including offline/online alignment, metric design, human-in-the-loop evaluation, A/B testing, power analysis, and regression testing
  • Hands-on experience with GenAI systems, including orchestration, retrieval, tool calling, and memory

Other requirements

  • Must live in a state where Airbnb, Inc. has a registered entity

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Equity
May be eligible for equity
Bonus
May be eligible for a bonus
Other
May be eligible for benefits, Employee Travel Credits

Full job description

From Airbnb’s careers site

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way.

The Community You Will Join:

AI and ML are at the heart of the Airbnb product. From Trust to Payments, and from Customer Service to Marketing, we rely on ML to ensure that guests and hosts have the best possible experience with Airbnb.

Show full descriptionShow less

The Core ML team is responsible for driving Airbnb Assistance Engineering initiatives by adopting Generative AI technologies to enable an intelligent, scalable, and exceptional service experience. The team develops and enhances AI models, ML services, and tools including LLM fine-tuning and optimization, RAG/Search, LLM evaluation and testing automation, feedback-based learning, and guardrails for a wide range of applications at Airbnb.

The richness of Airbnb's data, the complexity of its marketplace, and the variety innate in our product mean that we need to operate at the state of the art of AI practice. We are committed to long-term innovation to solve complex problems, and to do that we need experienced ML

​​The Difference You Will Make:

In this Senior Staff Machine Learning Engineering role, you will set technical direction and lead execution for ML evaluation and the end-to-end data flywheel powering Airbnb Assistance Engineering (e.g., assistive agents, issue resolution, and tooling). Your work will define how we measure quality, how we turn feedback into learning signals, and how we continuously improve models and products safely and efficiently. You will partner closely with product, engineering, design, operations to build evaluation systems that are trusted, scalable, and actionable - connecting offline metrics to online outcomes.

A Typical Day:

  • Define evaluation strategy and success metrics for GenAI systems, aligning offline evaluation with online business and customer experience outcomes.
  • Build and scale evaluation frameworks (golden sets, synthetic data, automated regressions, rubric-based grading, LLM-as-judge where appropriate) with strong controls for bias, drift, and reliability.
  • Design the data flywheel: instrumentation, feedback collection, data quality checks, labeling strategy, dataset versioning, and governance to support continuous improvement.
  • Lead cross-functional quality initiatives across product, ops, and engineering, driving clarity on what “good” looks like and how teams act on evaluation results.
  • Develop and productionize pipelines for dataset creation, model monitoring, evaluation-at-scale, and continuous testing (pre-deploy and post-deploy).
  • Drive technical decisions and architecture for evaluation and data infrastructure, balancing speed, rigor, cost, and safety.

Minimum Qualifications:

  • Educational Background: PhD in Computer Science, Mathematics, Statistics, or related technical field (or equivalent practical experience).
  • Industry Experience: 10+ years building, testing, and shipping ML/AI systems end-to-end; including 2+ years of experience with GenAI/LLM systems in production.
  • Leadership Experience: 5+ years leading large, ambiguous technical initiatives as a senior IC, influencing roadmap and engineering/science direction across teams.
  • Technical Proficiency:
  • Deep expertise in evaluation methodology (offline/online alignment, metric design, human-in-the-loop evaluation, A/B testing, power analysis, regression testing).
  • Hands-on experience with GenAI systems, including orchestration, retrieval, tool calling, memory, etc.
  • Experience building data pipelines and quality systems (labeling workflows, dataset curation, versioning, monitoring, and governance).
  • Solid ML fundamentals and best practices (model selection, training/serving, monitoring, reliability, and model lifecycle management).

Preferred Qualifications:

  • Customer Support Systems: Experience applying ML/AI to customer support workflows (e.g., agent assist, classification/routing, resolution recommendation, QA).
  • Infrastructure & Quality at Scale: Experience building robust evaluation platforms for agent behavior validation, safety/guardrails, and continuous improvement.
  • Agile Practice for Applied AI: Proven ability to take evaluation and data flywheel work from incubation to production, iterating quickly while maintaining scientific rigor.

Your Location:

This position is US - Remote Eligible. The role may include occasional work at an Airbnb office or attendance at offsites, as agreed to with your manager. While the position is Remote Eligible, you must live in a state where Airbnb, Inc. has a registered entity. Click here for the up-to-date list of excluded states. This list is continuously evolving, so please check back with us if the state you live in is on the exclusion list. If your position is employed by another Airbnb entity, your recruiter will inform you what states you are eligible to work from.

How We'll Take Care of You:

Our job titles may span more than one career level. The actual base pay is dependent upon many factors, such as: training, transferable skills, work experience, business needs and market demands. The base pay range is subject to change and may be modified in the future. This role may also be eligible for bonus, equity, benefits, and Employee Travel Credits.

Pay Range

$244,000—$305,000 USD

A Note on Recruiting Scams: Scammers sometimes pose as Airbnb recruiters to get money or personal information from candidates. A few things that will always be true for Airbnb’s hiring process: our open roles are posted on Airbnb’s Career’s Page at careers.airbnb.com, and our recruiters correspond only from @airbnb.com or @ext.airbnb.com email addresses. Our recruiters will never ask for your Social Security number, bank account details, passport, or payment app information while you are interviewing. We’ll also never ask you to pay a fee, send money, deposit or cash a check, or purchase work-related equipment (such as a company laptop) during the interview process. We encourage candidates to remain vigilant of these recruiting scams and not share sensitive information if you do not believe an individual is actually affiliated with Airbnb.

Equal Employment Opportunity:

Airbnb values diversity & belonging and is proud to be an Equal Employment Opportunity employer. All individuals seeking employment at Airbnb are considered without regard to race, color, religion, national origin, age, gender, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, gender expression, sexual orientation, or any other legally protected characteristic.

Reasonable Accommodations: We are committed to providing reasonable accommodations throughout the recruitment process for applicants with disabilities. If you need assistance, or an accommodation, please let your recruiter know once you are contacted about a role.

Something wrong with this job? Report it · This flag looks wrong

Similar remote jobs

See more like this