Data Engineer jobs

Senior Data Engineer, AI Systems

Movable InkMovable Ink +2 places

$165k–$215k/yr US pay range, listed by the employer

Key facts

Where
Movable Ink · New York, NY · Toronto, Canada
Experience
5+ years

Looks real. 1 thing to check before you apply.

On Movable Ink’s own careers site (Greenhouse), checked open today.

  • To check: Posted by the employer 3 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: Checked open todayGreenhouse board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundGreenhouse board
  • Trust score 89 / 100 · Looks good
See all 6 checksShow fewer
How we check

What you’ll do

  • Build, maintain, and optimize production data pipelines for AI-driven personalization.
  • Own and scale Spark batch pipelines, including cluster configuration, tuning, and performance optimization on GCP Dataproc.
  • Build and maintain the ML Data Lake, ensuring data quality, accessibility, and efficient storage.
  • Support ML engineers and scientists with data for model development, training, and evaluation.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 5+ years of data engineering experience
  • Experience with large-scale data processing, cluster configuration, optimization, and tuning
  • Experience with data storage formats such as Parquet and Delta Lake
  • Experience with event streaming data
  • Experience with cloud computing platforms
  • Experience with advanced query optimization
  • Familiarity with software development lifecycle practices, including CI/CD and automated deployment

Skills

  • Deep expertise with Apache Spark, including the PySpark DataFrame API and experience solving challenging scaling problems
  • Strong software development skills in Python, including unit testing, Git, code review, and CI/CD

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Health insurance
Medical benefits are included as part of a full range of benefits.
Bonus
Additional bonus may be included depending on the position ultimately offered.

Full job description

From Movable Ink’s careers site

Movable Ink scales content personalization for marketers through data-activated content generation and AI decisioning. The world’s most innovative brands rely on Movable Ink to maximize revenue, simplify workflow and boost marketing agility. Headquartered in New York City with close to 600 employees, Movable Ink serves its global client base with operations throughout North America, Central America, Europe, Australia, and Japan.

Show full descriptionShow less

The AI Systems team owns the core recommendations engine and ML platform that powers billions of AI-driven marketing decisions daily across some of the world's largest consumer brands. As a Senior Data Engineer, you will own the Spark-based data pipelines and data infrastructure at the heart of this system - building, scaling, and optimizing the data layer that feeds our production ML models. You will work alongside ML engineers and scientists in a collaborative environment, contributing data pipelines and products to power our core recommender systems and our DaVinci Personalization product. This is an opportunity to work end-to-end on large-scale data systems that touch millions of customers, on a team working at the intersection of data engineering and machine learning.

This role will be reporting to the Director of Engineering (AI/ML).

Responsibilities:

  • Build, maintain, and optimize production data pipelines that power AI-driven personalization at scale across content selection, send-time optimization, subject line personalization, and frequency capping
  • Own and scale Spark-based batch pipelines, including cluster configuration, tuning, and performance optimization across GCP Dataproc
  • Build and maintain our ML Data Lake, ensuring data quality, accessibility, and efficient storage
  • Support the data needs of ML Engineers and Scientists for model development, training, and evaluation
  • Identify and resolve performance bottlenecks and scaling limitations in data pipelines and infrastructure
  • Collaborate with distributed systems engineers on the platform's architectural evolution, ensuring data layer continuity throughout
  • Continuously improve data infrastructure for greater scalability and reliability
  • Release features and data products that deliver measurable and tangible business value

Qualifications:

  • 5+ years of data engineering experience
  • Deep expertise with Apache Spark, including the PySpark DataFrame API and experience solving challenging scaling problems
  • Experience with large-scale data processing, cluster configuration, optimization, and tuning (we use GCP Dataproc)
  • Strong software development skills in Python (unit testing, git, code review, CI/CD)
  • Experience with data storage formats (we use Parquet, Delta Lake)
  • Experience with event streaming data (we use Kafka)
  • Experience with cloud computing platforms (we use Google Cloud Platform)
  • Experience with advanced query optimization
  • Familiar with Software Development Lifecycle practices, such as continuous integration/continuous delivery and automated deployment (we use Docker, Kubernetes, and GitHub Actions)
  • Ability to collaborate with technical partners - you'll be working closely with ML engineers, scientists, and other teams to determine requirements and make design decisions
  • Enjoys working in a fast-paced, goal-driven environment

The base pay range for this position is $165K - $215K USD/year, which can include additional bonus depending on the position ultimately offered, in addition to a full range of medical, financial, and/or other benefits. The base pay offered may vary depending on job-related knowledge, skills, and experience.

Studies have shown that women, communities of color, and historically underrepresented people are less likely to apply to jobs unless they meet every single qualification. We are committed to building a diverse and inclusive culture where all Inkers can thrive. If you’re excited about the role but don’t meet all of the abovementioned qualifications, we encourage you to apply. Our differences bring a breadth of knowledge and perspectives that makes us collectively stronger.

We welcome and employ people regardless of race, color, gender identity or expression, religion, genetic information, parental or pregnancy status, national origin, sexual orientation, age, citizenship, marital status, ethnicity, family or marital status, physical and mental ability, political affiliation, disability, Veteran status, or other protected characteristics. We are proud to be an equal opportunity employer.

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

Senior Data Engineer

Pontera · New York, NY

$180k–$220k/yrHybrid

StrongChecked open todayPosted 21 days ago

Data Engineer

K Health · New York, NY

$150k–$175k/yrHybrid

StrongChecked open todayPosted 1 month ago

See more like this