Build OpenTelemetry instrumentation standards for agents, tools and platform services.
Integrate AgentCore Observability and CloudWatch with the client’s observability tools, and set up end-to-end tracing across agents, model calls, tools and handoffs.
Define SLOs and build alerting for latency, availability and errors.
Track token usage and compute spend by team, agent and environment; create dashboards, budgets and alerts for unexpected cost spikes.
Summarised from the posting by AI and can be wrong. The full description is the source.
Requirements
Experience
6+ years in SRE, platform reliability or observability engineering
Experience with Amazon CloudWatch and at least one enterprise observability platform
Experience defining and running SLOs, error budgets and alerting strategies
Experience building cost visibility and FinOps reporting on AWS, including Cost Explorer, CUR and tagging strategies
Prior experience in consulting
Experience monitoring LLM or agent-based applications, including token usage, latency and quality signals (preferred)
Experience with infrastructure as code using Terraform or AWS CDK for monitoring resources (preferred)
Experience in pharma, life sciences or another regulated industry (preferred)
Prior experience and connections in the Life Sciences industry (preferred)
Skills
Strong hands-on AWS experience
Hands-on experience with OpenTelemetry SDKs, collectors and exporters
Experience with distributed tracing
Scripting and coding skills in Python, Go or TypeScript
Familiarity with LLM observability tools such as Langfuse, Arize or LangSmith, or AgentCore Observability (preferred)
Your match
Sign in and add your resume to see which of this job's requirements it shows.
Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Full job description
From Appnovation Technologies’ careers site
About us
Appnovation is the AI Product Company. Headquartered in Vancouver, Canada, with teams around the world, Appnovation builds AI products, delivers enterprise AI and digital solutions, and co-invests with clients to bring new AI products to market.
Open source has been the foundation of Appnovation's work since 2007, and it remains the foundation underneath every AI product the company builds today. Appnovation teams work with organizations in highly regulated industries, including global leaders such as Pfizer, Google and AstraZeneca, to help them adopt AI they can govern and own.
Show full descriptionShow less
We’re putting together a dedicated delivery pod to build and run an internal agent platform on AWS Bedrock AgentCore for a global life sciences client. The pod works as one team with the client’s engineers to deliver the platform other teams will build their agents on.
In this role you make the platform visible and dependable. You’ll build the OpenTelemetry instrumentation, observability integrations, tracing, SLOs, alerting, and cost monitoring for token and compute spend. When an agent is slow, wrong or expensive, your work is how the team finds out and fixes it. This is a named-team engagement, so the person we propose is the person who starts.
ROLE RESPONSIBILITIES
Instrumentation: Build OpenTelemetry instrumentation standards for agents, tools and platform services, and make it easy for teams to adopt.
Observability Integration: Connect AgentCore Observability and CloudWatch with the client’s existing observability tools (e.g., Datadog, Splunk, Grafana).
Tracing: Set up end-to-end tracing across agent steps, model calls, tool calls and multi-agent handoffs so issues can be traced to their source.
SLOs and Alerting: Define SLOs for latency, availability and error rates, and build alerting that is useful and not noisy.
Cost Monitoring: Track token usage and compute spend by team, agent and environment, with dashboards, budgets and alerts for unexpected spikes.
Incident Readiness: Write runbooks, support incident response and run post-incident reviews that lead to real fixes.
QUALIFICATIONS
6+ years in SRE, platform reliability or observability engineering, with strong hands-on AWS experience.
Hands-on experience with OpenTelemetry (SDKs, collectors, exporters) and distributed tracing.
Experience with Amazon CloudWatch and at least one enterprise observability platform (Datadog, Splunk, Grafana, New Relic or similar).
Experience defining and running SLOs, error budgets and alerting strategies.
Experience building cost visibility and FinOps reporting on AWS (Cost Explorer, CUR, tagging strategies).
Scripting and coding skills in Python, Go or TypeScript.
Clear communication and able to turn data into decisions for technical and non-technical audiences.
PREFERRED QUALIFICATIONS
Experience monitoring LLM or agent-based applications, including token usage, latency and quality signals.
Familiarity with LLM observability tools (e.g., Langfuse, Arize, LangSmith) or AgentCore Observability.
Experience with infrastructure as code (Terraform or AWS CDK) for monitoring resources.
Experience in pharma, life sciences or another regulated industry.
WHO YOU ARE
You want to know what’s happening in a system before a user tells you
You build alerts people trust and dashboards people actually open
You treat cost as a reliability concern, not an afterthought
You stay calm in incidents and focus on learning afterward
You work well inside a client team and build trust quickly
You have prior experience in consulting
Prior experience and connections in the Life Sciences industry is preferred
Thank you for your interest in a career with Appnovation Technologies! Please note that only those selected for an interview will be contacted.
Accommodations are available upon request throughout the recruitment process.
At Appnovation, we recognize that diverse teams are the strongest teams. Diversity, Equity & Inclusion is not only something that we embrace - we celebrate it! We are proud to be an Equal Opportunity Employer and we encourage applicants from all backgrounds, lived experiences and industries to apply. Come join us at Appnovation, and learn more about how we stay true to our company values as we build better lives through better digital.