DevOps, SRE & Cloud Engineer jobs

Senior Cluster Site Reliability Engineer

The Voleon GroupRemote · US onlyFull-time

$205k–$235k/yr listed by the employer

Above the typical $176k a year for senior remote DevOps, SRE & Cloud Engineer jobsHow we compare

How we compare. Typical is the median yearly pay of 177 senior remote DevOps, SRE & Cloud Engineer jobs that list pay, from the employers' own postings. About means within 5%.

Key facts

Experience
5+ years
Degree
Bachelor's degree

Looks real. 1 thing to check before you apply.

On The Voleon Group’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 7 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundAshby board
  • Trust score 80 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Respond to cluster outages and urgent operational issues; triage and resolve them.
  • Track cluster uptime and define and monitor reliability SLAs.
  • Diagnose recurring problems and engineer systemic fixes with engineering teams.
  • Build cluster health metrics and observability, including custom monitoring where needed.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 5+ years of experience in SRE or DevOps roles
  • Experience with cloud infrastructure (AWS or GCP)
  • Experience with distributed storage technologies (Lustre, Ceph, S3)
  • Senior engineer or tech lead experience (preferred)
  • Hands-on experience with HPC frameworks (Slurm, Grid Engine) and Kubernetes-based job orchestrators (Airflow, Kueue, Kubeflow Pipelines), along with distributed computing frameworks (Ray, Modin, Dask, (preferred)
  • Familiarity with hybrid/on-prem environments (preferred)
  • Experience with containerization (Docker, Podman, Singularity), particularly for HPC/batch compute environments (preferred)
  • Experience with HPC networking (InfiniBand, RDMA) (preferred)

Education

  • Bachelor degree in computer science

Skills

  • Knowledge of HPC/batch compute frameworks (Slurm, Kueue, AWS/GCP Batch) and/or machine learning training systems (Kubeflow, MLflow, Horovod)
  • Ability to develop scripts and utilities of moderate complexity in a common scripting language (Python, Ruby, etc.)
  • Familiarity with infrastructure-as-code and configuration management tools (Terraform, Ansible)
  • Familiarity designing and implementing modern observability stacks (Prometheus, Grafana, Loki, ELK, OpenTelemetry)
  • Familiarity with ML frameworks (PyTorch/Tensorflow, JAX, Horovod, DeepSpeed) (preferred)
  • Solid security/IAM foundations (identity management systems, AWS/GCP IAM, Zero Trust) (preferred)

Other requirements

  • A system engineer rather than system administrator mindset, with systematic thinking and use of automation

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Learning & tuition
Technology talks by company experts
Meals
Daily catered lunches
Other
Highly competitive compensation and benefits packages

Full job description

From The Voleon Group’s careers site

Voleon is a technology company that applies state-of-the-art AI and machine learning techniques to real-world problems in finance. For nearly two decades, we have led our industry and worked at the frontier of applying AI/ML to investment management. We have become a multibillion-dollar asset manager, and we have ambitious goals for the future.

Show full descriptionShow less

Your colleagues will include internationally recognized experts in artificial intelligence and machine learning research as well as highly experienced finance and technology professionals. In addition to our enriching and collegial working environment, we offer highly competitive compensation and benefits packages, technology talks by our experts, a beautiful modern office, daily catered lunches, and more.

As a Senior Cluster Site Reliability Engineer (SRE), you will help scale our research compute cluster to meet our growing needs, and you will leverage engineering skills to ensure high degrees of uptime, reliability, and robustness. Our research clusters are at the core of our R&D, and you will be directly responsible for keeping this key resource available and performant.  Your work will provide a world-class HPC platform for researchers to focus on cutting-edge machine learning problems at scale.  You will support both on-prem and cloud infrastructure, and work to provide the best experience to our technical staff.  You will leverage IaC, Automation, and SRE principles to refine and hone a product that operates 24/7 to support Voleon.

The Cluster Operations team works on the frontline to triage and mitigate real-time operational issues. You will be an integral member of this team, solving day-to-day issues with high urgency, while also engineering systemic improvements and architectural fixes to prevent recurring issues. You will collaborate with engineering teams to develop improvements to monitoring/telemetry. You will help design and oversee operational frameworks to ensure the cluster operates within a set of rigorous SLAs.

RESPONSIBILITIES

  • Be a first responder in the event of cluster outages or issues. Triage and resolve urgent issues as they arise
  • Ensure a high degree of cluster uptime (measured in multiple nines), and define + track SLAs to quantify reliability
  • Diagnose systemic/recurring patterns of problems, and engineer precision solutions to them in collaboration with engineering teams
  • Develop robust metrics and observability for cluster health and use those metrics to inform your work. Build out custom observability mechanisms when off-the-shelf ones won't do
  • Help software and research teams design policies around fair cluster usage, and help develop enforcement mechanisms for said policies
  • Assist in forecasting cluster growth, and help select appropriate scale-up strategies. Help optimize operations across dimensions of cost and usability

REQUIREMENTS

  • 5+ years of experience in SRE or DevOps roles, preferably working as a senior engineer or tech lead
  • Knowledge of HPC/batch compute frameworks (Slurm, Kueue, AWS/GCP Batch) and/or machine learning training systems (Kubeflow, MLflow, Horovod)
  • Ability to develop scripts and utilities of moderate complexity in a common scripting language (Python, Ruby, etc.)
  • Familiarity with infrastructure-as-code and configuration management tools (Terraform, Ansible)
  • Experience with cloud infrastructure (AWS or GCP)
  • Familiarity designing and implementing modern observability stacks (Prometheus, Grafana, Loki, ELK, OpenTelemetry)
  • Experience with distributed storage technologies (Lustre, Ceph, S3)
  • Embodies a "system engineer" rather than "system administrator" mindset, thinking systematically and leveraging automation
  • Bachelor degree in computer science

PREFERRED QUALIFICATIONS

  • Hands-on experience with HPC frameworks (Slurm, Grid Engine) and Kubernetes-based job orchestrators (Airflow, Kueue, Kubeflow Pipelines), along with other distributed computing frameworks (Ray, Modin, Dask, Spark)
  • Familiarity with ML frameworks (PyTorch/Tensorflow, JAX, Horovod, DeepSpeed)
  • Familiarity with hybrid/on-prem environments
  • Experience with containerization (Docker, Podman, Singularity), particularly for HPC/batch compute environments
  • Experience with HPC networking (InfiniBand, RDMA)
  • Solid security/IAM foundations (Identity management systems, AWS/GCP IAM, Zero Trust)

“FRIENDS OF VOLEON” CANDIDATE REFERRAL PROGRAM

If you have a great candidate in mind for this role and would like to have the potential to earn $15,000 if your referred candidate is successfully hired and employed by The Voleon Group, please use this form https://voleon.com/referrals/ to submit your referral. For more details regarding eligibility, terms and conditions please make sure to review the Voleon Referral Bonus Program https://voleon.com/assets/Voleon-Referral-Bonus-Program.pdf.

EQUAL OPPORTUNITY EMPLOYER

The Voleon Group is an Equal Opportunity employer. Applicants are considered without regard to race, color, religion, creed, national origin, age, sex, gender, marital status, sexual orientation and identity, genetic information, veteran status, citizenship, or any other factors prohibited by local, state, or federal law.

Something wrong with this job? Report it · This flag looks wrong

Similar remote jobs

Senior Cloud Engineer

Earnest · Remote · US only

$184k–$230k/yr

StrongChecked open todayPosted 8 days ago

Senior Platform Engineer

Doma Technology LLC · Remote · US only

$160k–$190k/yrFull-time

StrongChecked open todayPosted 9 days ago

See more like this