Software Engineer jobs

Senior Applied AI Engineer

QualitateNew York City, NYFull-timeHybrid

$200k–$240k/yr listed by the employer

Above the typical $203k a year for senior Software Engineer jobs in New YorkHow we compare

How we compare. Typical is the median yearly pay of 456 senior Software Engineer jobs in New York that list pay, from the employers' own postings. About means within 5%.

Looks real. 1 thing to check before you apply.

On Qualitate’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 3 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 89 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Own the LLM orchestration layer for real-time, multi-turn AI Moderator interviews, including prompt and context design, model routing, and latency optimization.
  • Instrument evaluations, monitoring, and guardrails for production AI systems.
  • Build APIs and services that turn interview data into structured, queryable customer insights, and improve retrieval and post-processing through iterative evaluation.
  • Partner with research, product, data engineering, and the founding team to ship AI-powered research assets and shape the retrieval and generation stack, including an MCP over proprietary data.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 5+ years building and shipping applied AI/ML systems in production
  • Experience handling unstructured data such as audio, transcripts, and financial documents, and building pipelines to prepare it for models
  • Experience at a startup or in another fast-paced environment (preferred)
  • Experience as an applied AI/ML engineer at a technology startup (preferred)
  • Production experience with agentic or multi-turn conversational systems, voice or chat, used by real users (preferred)
  • Experience building or hardening evaluation frameworks for production AI systems, including offline and online evaluations, LLM-as-judge, and regression testing (preferred)
  • ETL and data engineering experience with messy unstructured data and automated ingestion and transformation into a relational database (preferred)
  • Background in fintech, research, or another AI-driven, data-intensive startup environment (preferred)

Skills

  • Hands-on LLM experience with prompt/system design, retrieval-augmented generation, tool/function calling, and structured output design across multiple model providers
  • Strong backend engineering in Python or TypeScript, including designing and shipping APIs and microservices
  • Vector database and search experience, including optimizing retrieval quality and latency at scale
  • Familiarity with speech-to-text transcription models (preferred)

Other requirements

  • Full-time, in-person work in New York City

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Health insurance
Health coverage
Paid time off
Flexible PTO, Paid holidays
Equity
Meaningful early-stage grant with upside tied to company growth
Bonus
Competitive base salary plus bonus

Full job description

From Qualitate’s careers site

ABOUT QUALITATE

Qualitate is building the AI-native primary intelligence platform for enterprises, investment firms, and asset managers. We automate the entire primary research workflow, from study design and expert recruitment through AI-moderated interviews and insight synthesis - replacing weeks of manual research with hours of high-signal output.

Expert research is a broken industry dominated by legacy incumbents with expensive, manual, and slow services. Investment and corporate strategy teams spend hundreds of hours coordinating intermediaries, scheduling expert calls, and synthesizing insights across fragmented providers. The process takes weeks to produce short-lived intelligence.

Show full descriptionShow less

Qualitate transforms this workflow by autonomously conducting expert discussions at scale using AI. Qualitate’s AI Moderator interviews thousands of experts simultaneously each month, capturing forward-looking purchasing plans, competitive displacement, and real-time ROI signals. The platform turns these interviews into quantified, time-series outputs tracking over 10,000 public and private companies, queryable instantly in natural language. Our customers include some of the world’s largest enterprises, hedge funds, PE firms, credit investors, and venture capital firms.

THE ROLE

We’re looking for a Senior Applied AI Engineer to help build the LLM and retrieval systems at the core of our AI Moderator - the engine that conducts thousands of real-time expert interviews and turns them into structured, queryable intelligence. You’ll work at the intersection of applied research and production engineering: designing agentic pipelines, scaling retrieval over proprietary data, and shipping systems that directly power what our customers see. This is a senior, high-ownership role on a small founding team - your architecture decisions and code will define how the product scales.

THE DAY-TO-DAY:

  • Own the LLM orchestration layer that drives our AI Moderator’s real-time, multi-turn expert interviews, from prompt and context design to model routing and latency optimization.
  • Instrument evals, monitoring, and guardrails for AI systems running in production, so quality issues surface before customers see them.
  • Build production APIs and services that turn raw interview data into novel, structured, queryable insights for customers, in partnership with product and data engineering.
  • Work cross-functionally with our research team to build and ship AI-powered research assets that are surfaced to customers.

WHO THIS ROLE IS FOR

  • 5+ years building and shipping applied AI/ML systems in production, ideally including time at a startup or other fast-paced environment.
  • Deep, hands-on LLM experience - prompt/system design, retrieval-augmented generation, tool/function calling, and structured output design across multiple model providers (Anthropic, OpenAI, open-weight models).
  • Strong backend engineering in Python/TypeScript with experience designing and shipping APIs and microservices.
  • Vector database and search know-how and experience optimizing retrieval quality and latency at scale.
  • Comfort with unstructured, messy data - audio, transcripts, financial documents - and the pipelines needed to turn it into model-ready input.
  • A bias for ownership - you can take a problem from rough prototype to production without needing a detailed spec.

WHAT SETS YOU APART

  • Prior experience as an applied AI/ML engineer for a technology startup.
  • Experience with agentic or multi-turn conversational systems - voice or chat - operating in production with real users.
  • Experience building or hardening evaluation frameworks (offline + online evals, LLM-as-judge, regression testing on prompt/model changes) for production AI systems.
  • Familiarity with speech-to-text transcription models (Deepgram, AssemblyAI, Whisper, or similar).
  • ETL and data engineering experience for dealing with messy unstructured data and setting up automated processes to ingest and transform the data in a relational database
  • A background in fintech, research, or another AI-driven, data-intensive startup environment.

WHAT YOU’LL BUILD IN THE FIRST 90 DAYS

  • Ship improvements to the LLM orchestration layer powering live AI Moderator interviews, with real customer traffic on the line.
  • Help improve our downstream LLM processes — retrieval relevance, or post-processing — by iterating labeled data → eval → prompt/code → re-eval.
  • Partner with the founding team to define what parts of the retrieval and generation stack to build versus buy as usage scales.
  • Help architect an MCP that sits on top of our proprietary dataset and the relevant tools that make up the orchestration layer

COMPENSATION & BENEFITS

  • Compensation: Competitive base salary + bonus
  • Equity: Meaningful early-stage grant with upside tied to company growth.
  • Benefits: Health coverage, flexible PTO, paid holidays.
  • Location: Full-time, in-person in New York City.

HOW WE WORK WITH AI AT QUALITATE

A lot of companies are cutting back on AI. We’re going the other way. At Qualitate, we think the people who learn to work well with AI now will set the pace for the next decade, and we want you to be one of them.

  • Experimenting is part of the job. New tools come out every week. Try them, test them, and tell the team what works. Nobody here waits for permission to find a better way.
  • A big token budget. Use AI as much as the work needs. We don’t ration it, and we won’t ask you to justify every prompt.
  • Use the tools that work best for you. Whatever makes you most effective, whether that’s a coding agent, a research assistant or something you built yourself, is welcome here.
  • We build systems that build on each other. We create and maintain our own internal tools, and each one is designed to make the others stronger. You’ll get to build and improve the systems the whole team runs on.

If you’re ambitious and want your AI skills to grow instead of being held back, you’ll fit in here.

WHAT TO EXPECT WORKING AT QUALITATE

  • Excellence. You’ll be held accountable to an exceptionally high bar and impact
  • High velocity. Fast paced work environment in terms of growth, decision making, and time to impact
  • High agency. Ownership over how you work, not just what you work on. We hire people we trust to figure it out.
  • A builder’s mindset. You’ll create the processes, playbooks, and systems that define how this company scales.
  • Shared ambition. A small, early team disrupting the multi-billion dollar market research industry with AI.
  • Joy. We work hard and genuinely enjoy working with each other. Dedication to our work is matched only by our passion for fostering an enjoyable, collaborative work environment.

Qualitate is an equal opportunity employer. We encourage candidates of all backgrounds to apply.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs in New York

Senior Software Engineer

Podium Automation · New York City, NY

$150k–$190k/yrFull-timeOn-site

StrongChecked open todayPosted 9 days ago

Senior Backend Engineer

Rain Technologies Inc. · New York City, NY

$180k–$220k/yrFull-timeHybrid

StrongChecked open todayPosted 10 days ago

See more like this