Engineering Manager jobs

AI Engineering Experienced Manager, Digital Transformation & Innovation

BDO USAGrand Rapids, MI +1 placePosted 20 days ago

$150k–$180k/yr listed by the employer

Key facts

Where
Grand Rapids, MI · Woodbridge, NJ
Experience
8+ years

This job is closed.

The employer took it down 17 days ago, after 1 day open. Here are similar open jobs.

  • Checked: On the company's own careers siteOracle Recruiting board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundOracle Recruiting board
  • Note: Last seen open 18 days agoLive check
  • Trust score 100 / 100 · Strong
See all 5 checksShow fewer
How we check

Similar open jobs

VP, Platform Engineering

Kobie Marketing · Remote · US only

Full-time

Looks goodPay not listedChecked open todayPosted 3 days ago

See more like this

What you’ll do

  • Design, build, deploy, and operate agent architectures, orchestrations, and LLM-powered features across the product portfolio.
  • Implement RAG pipelines, prompt strategies, reusable AI components, and integrations with the .NET and React product stack.
  • Establish model selection, evaluation, guardrails, observability, and LLMOps practices to improve accuracy, reliability, cost, and performance.
  • Ensure AI systems meet governance, security, regulatory, data-handling, and auditability requirements, coordinating reviews and documentation.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Licenses and certifications

  • Microsoft Certified: Azure AI Engineer Associate, or equivalent (preferred)

Experience

  • 8+ years of experience in software engineering or related technology fields
  • 4+ years of hands-on experience building and deploying production AI/ML or LLM applications
  • 4+ years of experience with AI orchestration frameworks and Azure AI services
  • Demonstrated delivery of agent, RAG, or function-calling features to production
  • Experience with AI orchestration and agent frameworks such as Semantic Kernel, LangChain, or Durable Functions
  • Strong development experience with .NET 10/C# and React
  • Experience with RAG, structured outputs, function calling, and vector technologies
  • Proven ability to serve as a hands-on AI engineer while providing team leadership and engineering governance
  • Experience delivering enterprise-scale AI solutions in professional services, assurance, or accounting (preferred)
  • Experience establishing AI evaluation, responsible-AI, or model governance frameworks (preferred)
  • Experience leading technical teams or directing complex engineering initiatives (preferred)
  • Experience delivering enterprise AI solutions in professional services, assurance, accounting, financial services, or another regulated industry (preferred)
  • Experience with Azure DevOps CI/CD pipelines, Git, Docker, and Azure Container Apps (preferred)
  • Experience with AI evaluation and observability tooling and agentic development tooling such as Claude Code (preferred)

Education

  • High School Diploma/GED
  • Bachelor’s degree focused in Computer Science, Information Systems, Engineering, or Information Technology (preferred)

Skills

  • Expert-level knowledge of Azure AI Foundry, Azure OpenAI, and Azure AI Search
  • Strong understanding of the AI application lifecycle, including prompt design, evaluation, deployment, monitoring, and observability
  • Expert-level knowledge of applied LLM and agent engineering for production systems
  • Deep knowledge of agent architectures, orchestration, RAG, and responsible-AI practices
  • Working knowledge of AI governance frameworks and designing, building, and operating systems aligned with AI governance policies

Full job description

From BDO USA’s careers site

Job Summary:

Show full descriptionShow less

The AI Engineering Experienced Manager serves as the senior development lead for the AI capabilities embedded across the DT&I product portfolio, with primary emphasis on the architecture, development, production operation, and continuous improvement of agent architectures, orchestration, and LLM-powered application features. This role owns the hands-on engineering of intelligent capabilities—from model selection and prompt and tooling design through evaluation, deployment, and observability—ensuring AI features are accurate, reliable, secure, and cost-effective within BDO’s Azure ecosystem. The AI Engineering Experienced Manager works within and ensures that all AI capabilities are designed, deployed, and operated in alignment with BDO’s AI Governance Policies and responsible-AI standards. The role works in close coordination with the Context Engineering Experienced Manager and Data & Analytics Lead to deliver well-integrated, properly provisioned AI solutions. The AI Engineering Experienced Manager partners closely with cross-functional teams to align AI engineering practices and product delivery with BDO’s AI Governance Policies, responsible AI standards, assurance service delivery objectives, firm policies, and security standards.

Job Duties:

Agent & Orchestration Engineering

  • Designs and builds agent architectures and multi-agent orchestrations using Semantic Kernel and Azure AI Foundry across the DT&I portfolio
  • Serves as the principal technical authority on agent design, planners, tool and function calling, memory, and workflow patterns
  • Architects and implements production-grade AI features, turning prototypes into reliable capabilities within the product stack
  • Evaluates and integrates emerging models, frameworks, and orchestration tools to continuously improve capability and efficiency
  • Designs agent observability, error handling, retry, and fallback strategies to ensure resilient, fault-tolerant AI workflows
  • Sets engineering quality standards for AI application design, testing, deployment, monitoring, and operational support.

LLM Application Engineering

  • Implements retrieval-augmented generation (RAG) pipelines, structured outputs, function calling, and application-layer prompt strategies
  • Integrates AI capabilities into the .NET 10 / React product stack across the Advantage portfolio (PEM, RevRec, GCM 2.0, Centralized Billing, and others)
  • Partners with the Context Engineering Experienced Manager to integrate governed knowledge, context and grounding within agent and application runtimes
  • Builds reusable AI service components, SDKs, and patterns for use across DT&I product teams
  • Establishes development practices that promote maintainability, extensibility, testability, and operational readiness
  • Guides engineering teams through technical challenges, design decisions, production incident, and remediation activities
  • Ensures AI capabilities meet defined product, assurance service delivery, performance, usability, and reliability objectives

Model Strategy, Evaluation & Responsible AI

  • Establishes model selection, routing, fallback, and cost and latency optimization across AI features
  • Designs evaluation harnesses, regression tests, and guardrails to measure and defend accuracy, cost, and latency targets
  • Implements responsible-AI controls including safety, bias mitigation, accuracy, and human-in-the-loop appropriate to a regulated assurance context
  • Works within and maintains alignment with BDO’s AI Governance Policies—including approved-model, acceptable-use, data-handling, and responsible-AI requirements—across all AI features
  • Owns LLMOps including deployment, monitoring, observability, and drift detection for AI systems within the Advantage SDLC and CI/CD pipeline

Risk Management

  • Ensures AI systems comply with BDO’s AI Governance Policies, firm security policies, regulatory requirements, and responsible-AI standards (SOC 2, PCAOB AS 2201, QC 1000, ISO 27001)
  • Coordinates AI governance reviews and approvals and maintains the documentation and evidence required to demonstrate alignment with BDO’s AI Governance Policies
  • Manages data handling, auditability, and the controls required for AI-assisted assurance work
  • Partners with risk and compliance stakeholders to maintain alignment between AI systems and firm governance requirements

Performs other duties as assigned

  • Willingly accepts share of less desirable assignments

Supervisory Responsibilities:

  • Acts as a direct supervisor to engineering and development team members, as assigned
  • Acts as a career advisor and mentor to engineering and development team members as assigned

Qualifications, Knowledge, Skills, and Abilities:

Education:

  • High School Diploma/GED, required
  • Bachelor’s degree with a focus in Computer Science, Information Systems, Engineering, Information Technology, preferred

Experience:

  • Eight (8) or more years of experience in software engineering or related technology fields, required
  • Four (4) or more years of hands-on experience building and deploying production AI/ML or LLM applications, required
  • Four (4) or more years of experience with AI orchestration frameworks (Semantic Kernel, LangChain, or equivalent) and Azure AI services, required
  • Demonstrated delivery of agent, RAG, or function-calling features to production, required
  • Experience delivering enterprise-scale AI solutions in professional services, assurance, or accounting industries, preferred
  • Experience establishing AI evaluation, responsible-AI, or model governance frameworks, preferred
  • Experience leading technical teams or directing complex engineering initiatives, preferred
  • Experience delivering enterprise AI solutions within professional services, assurance, accounting, financial services, or another regulated industry, preferred

License/Certifications:

  • Microsoft Certified: Azure AI Engineer Associate, or equivalent, preferred

Technology:

  • Expert-level knowledge of Microsoft Azure AI services including Azure AI Foundry, Azure OpenAI, and Azure AI Search, required
  • Strong understanding of the AI application lifecycle including prompt design, evaluation, deployment, monitoring, and observability, required
  • Experience with AI orchestration and agent frameworks (Semantic Kernel, LangChain, Durable Functions), required
  • Strong development experience with .NET 10 / C# and React, required
  • Experience with RAG, structured outputs, function calling, and vector technologies (Azure AI Search, embeddings), required
  • Experience with CI/CD pipelines (Azure DevOps), source control (Git), and containerization (Docker, Azure Container Apps), preferred
  • Experience with AI evaluation and observability tooling and agentic development tooling (Claude Code or equivalent), preferred

Knowledge, Skills, & Abilities:

  • Expert-level knowledge of applied LLM and agent engineering for production systems
  • Proven ability to serve as a hands-on AI engineer while providing team leadership and engineering governance
  • Deep knowledge of agent architectures, orchestration, RAG, and responsible-AI practices
  • Working knowledge of AI governance frameworks and the ability to design, build, and operate AI systems in alignment with BDO’s AI Governance Policies
  • Familiarity with data architecture and the ability to collaborate with context, infrastructure, and data engineering teams on dependencies
  • Excellent problem-solving skills with a focus on accuracy, reliability, cost optimization, and operational efficiency
  • Strong communication skills with the ability to convey technical concepts to non-technical stakeholders and leadership
  • Experience presenting AI architecture and governance recommendations to governance bodies (e.g., Architecture Review Boards)
  • Strong knowledge of AI security, data privacy, and compliance considerations in regulated environments
  • Ability to work across technical teams and business stakeholders in complex, fast-paced enterprise environments
  • Self-directed with the ability to manage multiple priorities and competing deadlines
  • Collaborative mindset with the ability to build relationships across all levels of the organization
  • Ability to travel up to 10%, no relocation required

Individual salaries that are offered to a candidate are determined after consideration of numerous factors including but not limited to the candidate’s qualifications, experience, skills, and geography.

National Range: $150,000 - $180,000

Maryland Range: $150,000 - $180,000

NYC/Long Island/Westchester Range: $150,000 - $180,000

Something wrong with this job? Report it