Software Engineering jobs

Senior AI Platform and Harness Engineer

LTSRemote · US only

$164k–$200k/yr listed by the employer

Key facts

Experience
7+ years
Degree
Bachelor's degree
Clearance
PUBLIC TRUST clearance required

Looks real. 1 thing to check before you apply.

On LTS’s own careers site (Greenhouse), checked open today.

  • To check: Posted by the employer 3 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: Checked open todayGreenhouse board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundGreenhouse board
  • Trust score 90 / 100 · Strong
See all 6 checksShow fewer
How we check

What you’ll do

  • Build and maintain enterprise AI platforms, reusable testing harnesses, evaluation frameworks, and LLMOps pipelines.
  • Prepare data and train, fine-tune, evaluate, and optimize LLMs; develop workflows for prompt testing, retrieval evaluation, and model benchmarking.
  • Implement AI observability, monitoring, security, governance, and Responsible AI controls.
  • Build backend services and APIs, integrate AI models and related systems, and deploy AI services across cloud platforms using containerized architectures.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 7+ years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering
  • 2+ years building or supporting Generative AI, LLM, or machine learning applications
  • Experience implementing LLMOps or MLOps platforms and deployment pipelines
  • Hands-on experience training, fine-tuning, and optimizing LLMs, including data preparation, SFT, LoRA/QLoRA, model evaluation, and performance optimization
  • Experience developing APIs, backend services, and distributed systems
  • Experience with AWS, Azure, or Google Cloud Platform
  • Experience deploying applications using Docker and Kubernetes
  • Experience with Git, CI/CD pipelines, Infrastructure as Code, and infrastructure automation
  • Experience building scalable, production-grade software platforms
  • Experience with LangChain, LangGraph, LlamaIndex, Semantic Kernel, or AutoGen (preferred)
  • Experience with LangSmith, OpenTelemetry, Prometheus, Grafana, Evidently AI, or Arize AI (preferred)
  • Experience with Pinecone, Qdrant, Weaviate, Azure AI Search, or pgvector (preferred)
  • Experience with OpenAI, Azure OpenAI, AWS Bedrock, Anthropic Claude, Google Vertex AI, or similar enterprise AI platforms (preferred)
  • Experience implementing Responsible AI, AI governance, model security, and AI safety practices (preferred)
  • Experience supporting Federal Government or other regulated environments (preferred)
  • Experience evaluating AI systems for quality, reliability, accuracy, explainability, latency, and cost optimization (preferred)
  • Familiarity with healthcare, enterprise modernization, or mission-critical systems (preferred)

Education

  • Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical field

Skills

  • Strong programming experience in Python and TypeScript
  • Strong understanding of LLMs, RAG, prompt engineering, embeddings, vector databases, AI agents, and agentic workflows
  • Familiarity with AI evaluation techniques, automated testing, benchmarking, regression testing, and model validation

Other requirements

  • Ability to obtain and maintain a Public Trust

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Other
Comprehensive benefits for eligible employees, with options intended to meet employees' and their families' needs.

Full job description

From LTS’s careers site

Location: United States – Remote Clearance: Ability to obtain and maintain a Public Trust

LTS is seeking a Senior AI Platform and Harness Engineer to develop and maintain the infrastructure, tooling, and evaluation frameworks that power enterprise AI solutions. This role is responsible for building the AI platform and reusable "AI harnesses" that enable Large Language Models (LLMs), AI agents, Retrieval-Augmented Generation (RAG), and Generative AI applications to be securely developed, tested, evaluated, monitored, and deployed at scale.

Show full descriptionShow less

The ideal candidate has experience with AI platforms, LLMOps, software engineering, cloud-native technologies, and backend systems, along with a passion for building reliable, observable, and production-ready AI solutions. You will work closely with AI architects, software engineers, data scientists, and product teams to ensure AI solutions are scalable, secure, cost-effective, and continuously improving.

What You'll Do:

  • Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications.
  • Lead data preparation, training and fine-tuning of LLMs
  • Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance.
  • Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance.
  • Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics.
  • Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement.
  • Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking.
  • Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity.
  • Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations.
  • Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications.
  • Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures.
  • Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment.
  • Apply security, governance, and Responsible AI controls throughout the AI development lifecycle.
  • Evaluate emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to improve engineering productivity.
  • Troubleshoot production AI issues and continuously improve platform reliability, scalability, security, and user experience.
  • Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices.

What We're Looking For:

  • Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical field.
  • 7+ years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering.
  • 2+ years building or supporting Generative AI, Large Language Model (LLM), or machine learning applications.
  • Experience implementing LLMOps or MLOps platforms and deployment pipelines
  • LLM Training and Fine-Tuning with demonstrated hands-on experience training, fine-tuning, and optimizing Large Language Models (LLMs), including data preparation, supervised fine-tuning (SFT), parameter-efficient techniques (LoRA/QLoRA), model evaluation, and performance optimization using frameworks such as PyTorch, Hugging Face, and related ML tools.
  • Strong programming experience in Python/Typescript
  • Experience developing APIs, backend services, and distributed systems.
  • Experience with cloud platforms including AWS, Azure, or Google Cloud Platform.
  • Experience deploying applications using Docker and Kubernetes.
  • Experience working with Git, CI/CD pipelines, Infrastructure as Code (IaC), and infrastructure automation.
  • Strong understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Prompt engineering, Embeddings, Vector databases, AI agents and agentic workflows
  • Familiarity with AI evaluation techniques, automated testing, benchmarking, regression testing, and model validation.
  • Experience building scalable, production-grade software platforms.
  • Strong problem-solving, debugging, and performance optimization skills.

Nice to Have:

  • Experience with AI orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, or AutoGen.
  • Experience with AI observability tools such as LangSmith, OpenTelemetry, Prometheus, Grafana, Evidently AI, or Arize AI.
  • Experience with vector databases including Pinecone, Qdrant, Weaviate, Azure AI Search, or pgvector.
  • Experience with OpenAI, Azure OpenAI, AWS Bedrock, Anthropic Claude, Google Vertex AI, or similar enterprise AI platforms.
  • Experience implementing Responsible AI, AI governance, model security, and AI safety best practices.
  • Experience supporting Federal Government or other regulated environments.
  • Experience evaluating AI systems for quality, reliability, accuracy, explainability, latency, and cost optimization.
  • Familiarity with healthcare, enterprise modernization, or mission-critical systems.

What’s In It for You?

  • The Opportunity to support high-visibility federal missions
  • A culture that values innovation, growth, and collaboration
  • Access to cutting-edge tools and technologies
  • Comprehensive benefits for you and your family
  • A career path that rewards ambition and performance

If you’re ready to push boundaries, sharpen your skills, and join a team that is passionate about building what’s next, we’d love to meet you. Apply today and let’s build a future together!

Pay Range

$164,000—$200,000 USD

LTS shares salary ranges to promote transparency. Compensation ranges are provided for informational purposes, and final compensation may vary based on experience, skills, location, and role requirements.

LTS is committed to offering eligible employees comprehensive benefits that will provide them with options intended to meet their needs and the needs of their family.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs

AI Engineer III

Vericast · Remote · US only

$120k–$140k/yrFull-time

StrongChecked open todayPosted 3 days ago

Senior Product Owner

Privia Health · Remote · US only

$110k–$130k/yrFull-time

StrongChecked open todayPosted 3 days ago

See more like this