Product Manager jobs

Lead Product Manager, Agent Runtime & Evals

JobgetherRemote · US onlyFull-time7+ years experiencePosted 23 days ago

$200k–$230k/yr listed by the employer

Above the typical $196k a year for manager-level remote Product Manager jobsHow we compare

How we compare. Typical is the median yearly pay of 143 manager-level remote Product Manager jobs that list pay, from the employers' own postings. About means within 5%.

This job is closed.

The employer took it down 22 days ago, after 0 days open. Here are similar open jobs.

  • Checked: On the company's own careers siteLever board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundLever board
  • Note: Last seen open 22 days agoLive check
  • Trust score 100 / 100 · Strong
See all 5 checksShow fewer
How we check

Similar open jobs

Senior Product Manager

Packback · Remote · US only

$130k–$150k/yrFull-time

StrongChecked open todayPosted 11 days ago

See more like this

What you’ll do

  • Own the vision, strategy, and roadmap for an agent runtime platform and its lifecycle management capabilities.
  • Develop evaluation, governance, memory, learning, and model-routing capabilities for AI agents and automations.
  • Define secure integrations, agent identities, permissions, oversight, and monitoring for fleets of agents.
  • Partner with workflow orchestration leadership, engineering, and design to establish standards and tools for governed agent workflows.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 7+ years of product management experience in B2B or enterprise SaaS, including at least 3 years in AI-focused product roles
  • Experience owning complex products end to end, from vision and strategy through roadmap, execution, launch, and continuous improvement
  • Experience building and scaling AI agents or automation solutions that replace manual business processes
  • Leadership ability to influence multiple teams, establish shared product direction, and mentor or manage product managers
  • Experience building products in a zero-to-one environment (preferred)
  • Experience with managed agent platforms such as Claude Managed Agents or Workato Genies, particularly evaluations, connectors, permissions, versioning, and deployments (preferred)

Skills

  • Practical understanding of AI evaluation methodologies, including golden datasets, data labeling, automated judges, regression testing, performance monitoring, and human-in-the-loop evaluation
  • Ability to work with engineering teams on agent-to-workflow contracts, structured outputs, deterministic execution, agent memory, and model orchestration
  • Hands-on fluency with modern AI tools and large language models, including Claude and Cursor, and ability to independently prototype solutions
  • Ability to make informed decisions about AI models, evaluation frameworks, integrations, access controls, and deployment strategies

Other requirements

  • Ability to balance technical complexity with practical business outcomes and end-user improvements

Benefits

Health insurance
Medical coverage for employees and their families
Dental & vision
Dental and vision coverage for employees and their families
Retirement plan
Access to retirement savings plans
Paid time off
Flex Time Off and company-paid holidays
Parental leave
Family leave, medical leave, and bereavement leave policies
Equity
Opportunity to participate in the company's long-term growth through equity compensation
Learning & tuition
Annual stipend for continued learning, training, and career growth
Remote work stipend
Allowance to personalize home office and technology setup
Wellness & mental health
Mental health coverage and resources to support overall well-being
Flexible schedule
Remote-first collaboration with opportunities to work alongside colleagues across multiple locations
Life & disability insurance
Life insurance
Family & fertility support
Fertility benefits and support for growing families

Full job description

From Jobgether’s careers site

Accountabilities: Own the vision, product strategy, and roadmap for the agent runtime platform, enabling non-technical users to create, test, publish, deploy, monitor, improve, and retire AI agents and automations without requiring traditional engineering support. Lead the development of agent lifecycle management capabilities, including version control, deployment processes, runtime management, observability, and continuous improvement. Define and evolve evaluation and model governance frameworks, including golden datasets, data labeling, automated evaluation, regression testing, and quality gates to ensure automations meet required standards before production. Establish evaluation mechanisms for AI agents that assess other agents, ensuring quality, reliability, and measurable performance across large-scale automation environments. Shape agent memory, learning, and self-improvement capabilities, ensuring that agents can retain relevant context and improve through experience while delivering measurable user value. Lead the development of a model-agnostic platform, defining model routing, cost attribution, and pricing strategies that allow teams to select appropriate models for different use cases. Define integration strategies that enable agents to securely access internal systems, business data, and third-party applications through reusable connectors, credentials, permissions, and access controls. Establish agent identity and execution models, including user-based permissions and independent service identities, while ensuring appropriate governance and auditability. Develop oversight and monitoring experiences that allow business teams to manage large fleets of agents through dashboards, execution histories, traces, metrics, logs, audit trails, and emergency controls. Partner with workflow orchestration product leadership to define agent registries, structured contracts, and integration standards that allow agents to operate reliably within governed workflows. Collaborate with engineering and design teams to create intuitive configuration tools that enable business specialists to build, test, and optimize automations independently. Monitor emerging AI technologies, managed agent platforms, orchestration frameworks, and automation solutions to inform strategic build-versus-buy decisions and partnership opportunities. Provide product leadership across adjacent teams, guide complex technical and strategic decisions, mentor product colleagues, and establish high standards for product execution and customer experience. Requirements: 7+ years of product management experience in B2B or enterprise SaaS environments, including at least 3 years working in AI-focused product roles. Proven experience owning complex products end to end, from product vision and strategy through roadmap development, execution, launch, and continuous improvement. Experience building and scaling AI agents or automation solutions that replace manual business processes, ideally in a zero-to-one product environment. Strong practical understanding of AI evaluation methodologies, including golden datasets, data labeling, automated judges, regression testing, performance monitoring, and human-in-the-loop evaluation. Ability to work closely with engineering teams on technical concepts such as agent-to-workflow contracts, structured outputs, deterministic execution, agent memory, and model orchestration. Strong understanding of how non-technical users build, configure, deploy, and supervise automated systems, with a focus on usability and operational reliability. Hands-on fluency with modern AI tools and large language models, including platforms such as Claude and Cursor, with the ability to independently prototype solutions and accelerate product development. Ability to make informed decisions about AI models, evaluation frameworks, integrations, access controls, and deployment strategies. Experience with managed agent platforms such as Claude Managed Agents or Workato Genies is a strong advantage, particularly regarding evaluations, connectors, permissions, versioning, and deployments. Excellent written and verbal communication skills, with the ability to explain complex product decisions to engineers, designers, executives, and customers. Demonstrated leadership and ability to influence multiple teams, establish shared product direction, and mentor or manage other product managers. Strong product judgment, analytical thinking, customer empathy, and comfort navigating ambiguity in a rapidly evolving technology environment. Ability to balance technical complexity with practical business outcomes, ensuring that platform capabilities translate into meaningful improvements for end users. Benefits: Competitive compensation: US salary range of USD $200,000–$230,000+, with final compensation depending on experience, skills, and location. Canadian compensation details can be confirmed with the recruiter. Equity: Opportunity to participate in the company's long-term growth through equity compensation. Comprehensive healthcare: Medical, dental, and vision coverage for employees and their families. Life insurance: Financial protection and additional security. Mental wellness: Coverage and resources to support mental health and overall well-being. Family support: Fertility benefits and support for growing families. Flexible time off: Flex Time Off in addition to company-paid holidays. Paid leave: Family leave, medical leave, and bereavement leave policies. Retirement planning: Access to retirement savings plans. Remote work support: Allowance to personalize your home office and technology setup. Professional development: Annual stipend to support continued learning, training, and career growth. Flexible work environment: Remote-first collaboration with opportunities to work alongside talented colleagues across multiple locations.

Show full descriptionShow less

How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether?

#LI-CL1

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

Something wrong with this job? Report it