Data Engineer jobs

Lead Data Warehouse Engineer - Artificial Intelligence-Ready Mount Sinai (AIR.MS)

Mount Sinai Health SystemNew York, NYOn-siteBachelor's degree

Pay not listed

This job is closed.

The employer took it down 3 days ago, after 0 days open. Here are similar open jobs.

  • To check: Posted by the employer 6 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • To check: No pay range listed, though one is likely required in NYAsk for the pay range before you interview. The law here likely requires one.
  • Checked: On the company's own careers siteOracle Recruiting board
  • Note: Last seen open 3 days agoLive check
  • Trust score 70 / 100 · Looks good
See all 5 checksShow fewer
How we check

Similar open jobs

Data Engineer

K Health · New York, NY

$150k–$175k/yrHybrid

StrongChecked open todayPosted 1 month ago

Data Engineer II

Garner Health · New York City, NY +1 place

$138k–$160k/yrHybrid

StrongChecked open todayPosted 1 month ago

Sr. Data Engineer

NBCUniversal · New York, NY

$115k–$145k/yrFull-timeHybrid

StrongChecked open todayPosted 2 days ago

Data Engineer

Mindex · Remote · US only

$90k–$140k/yrFull-time

StrongChecked open todayPosted 7 days ago

Staff Data Engineer

Fanatics Betting & Gaming · Remote · US only

$181k–$238k/yrFull-time

StrongChecked open todayPosted 16 days ago

See more like this

What you’ll do

  • Assess data integration needs and priorities, plan integrations, and build pipelines that bring multi-modal data and metadata into the AIR.MS warehouse.
  • Design, test, maintain, and tune databases, data pipelines, and flat-file loading processes.
  • Lead technical planning, release management, design sessions, code reviews, documentation, and mentoring for junior engineers.
  • Monitor warehouse and pipeline operations, troubleshoot failures, and coordinate with DBAs and system administrators on maintenance and performance.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 12–15 years of related experience, including 7 years designing, developing, and maintaining relational databases, data pipelines, and dimensional/OLAP warehouses (preferred)
  • Experience working with healthcare data (preferred)
  • Experience with healthcare data such as EHR, billing/claims, and cost accounting; Epic Clarity/Caboodle; and data models such as OMOP, i2b2, and PCORnet (preferred)
  • Experience with database administration, including configuration, performance tuning, partitioning, materialized views, permissions, backups, and restorations (preferred)

Education

  • Bachelor’s degree in a technical discipline
  • Master’s degree (preferred)

Skills

  • Expert knowledge of data warehousing, including 3NF and dimensional modeling, change data capture, incremental loads, data lineage, source-to-target mappings, and pattern-based and parameter-driven 개발 (preferred)
  • Expert-level experience with SQL, indexing, stored procedures, UDFs, sequences, dynamic SQL, data transformation tools, and job orchestration tools (preferred)
  • Experience with DevOps/SDLC best practices, Agile (Scrum, Kanban), JIRA, Confluence, and Git (preferred)
  • Knowledge of Hadoop, Spark, Kafka, other big data technology stacks, and streaming tools (preferred)

Full job description

From Mount Sinai Health System’s careers site

The Scientific Computing and Data team at the Icahn School of Medicine at Mount Sinai partners with scientists and clinicians to accelerate scientific discovery. The AI-Ready.Mount Sinai (AIR.MS) platform contains patient data generated from the clinical care processes at the Mount Sinai Health System. AIR·MS is a cloud-based, high-performance SAP HANA data platform with electronic health record (EHR) data in an OMOP data format. It also contains metadata from and links to raw data sets in other modalities, such as radiology, genomics and pathology. Researchers can access the AIR.MS data through direct database access, AI agents, cohort query tools and the Minerva high-performance computer. AIR.MS ingests OHDSI’s Observational Medical Outcomes Partnership (OMOP)-formatted EHR data from the Mount Sinai Data Warehouse. AIR.MS is integrated with Minerva, a high-performance computer with >21 petaflops of raw computational power and the raw radiology, genomics and pathology data sets. An expert team of 20+ PhD/MD computational scientists, biomedical informaticists and computer scientists partner with researchers and clinicians to effectively and efficiently utilize these resources for translational science.

Show full descriptionShow less

The Lead Data Warehouse Engineer is a senior technical specialist responsible for leading the ongoing integration of multi-modal clinical data into the AIR.MS data warehouse. The Lead Data Warehouse Engineer will assess the state of data integration, identify opportunities for data integration based on researcher’s priorities, develop a plan to integrate metadata and data, and execute on the data integration. The incumbent will work collaboratively with other members of the AIR.MS data platform and Mount Sinai Data Warehouse teams to lead technical efforts for the integration of multi-modal data sets resulting in expanded AIR.MS functionality. The AIR.MS data warehouse is built on the SAP HANA technology stack and MSDW is built on MySQL.

  • Design databases and pipelines that balance functionality, performance, cost, and development time; evaluate technical options with the product manager.
  • Design, build, test, and maintain data pipelines that extract/capture data from source systems, transform and augment those data, and integrate it into a multi-modal data repository.
  • Serve as a team leader; contribute to project planning, work breakdown, dependency sequencing, and release management.
  • Develop and promote standards, conventions, design patterns, DevOps/SDLC best practices, and operational procedures for pipelines and warehouse maintenance.
  • Mentor junior engineers in data warehousing, data engineering skills, and operational support.
  • Design, build, and maintain data management processes, including loading flat files (csv, tsv, pipe-delimited, JSON).
  • Lead design sessions, code walkthroughs, peer reviews, and produce technical documentation.
  • Tune database objects, stored procedures, and pipelines to optimize performance and minimize compute and storage costs.
  • Monitor database and pipeline operations; lead troubleshooting and remediation of failures; provide occasional after-hours on-call support.
  • Collaborate with DBAs and system administrators on backups, performance tuning, statistics/index maintenance, and patching.
  • Provide high-quality customer service to researchers, clinicians, and internal partners; maintain a science‑driven, customer-focused approach.
  • Ensure patient privacy and data security in compliance with IRB & cybersecurity policies, HIPAA, 42 CFR Part 2, NYS Article 27-F, and other regulations.
  • Stay current with emerging technologies to improve capabilities, efficiency, quality, or cost.
  • Identify improvements in procedures, technology, compliance, and data privacy/security.
  • Periodically assist DBAs with user provisioning, backups, restorations, capacity planning, and performance monitoring.
  • Perform related duties as assigned.
  • Bachelors degree in a technical discipline; Masters degree preferred
  • 12-15 years preferred of related experience, including 7 years of experience designing, developing, and maintaining relational databases, data pipelines, and dimensional/OLAP warehouses.

Preferred:

  • Expert knowledge of data warehousing: 3NF & dimensional modeling (fact table types, SCDs), change data capture, incremental loads, data lineage, source-to-target mappings, pattern-based & parameter-driven development.
  • Experience working with healthcare data
  • Expert-level experience with data engineering technologies: SQL, indexing, stored procedures, UDFs, sequences, dynamic SQL, data transformation tools, job orchestration tools for data processing.
  • Experience with DevOps/SDLC best practices; Agile (Scrum, Kanban) with JIRA and Confluence; version control with git.
  • Strong communication and customer service skills for working with researchers, clinicians, administrators, and IT staff.
  • Excellent critical thinking, problem-solving, multitasking, and collaboration skills; ability to work independently in a fast-paced environment.
  • Preferred experience with healthcare data (EHR, billing/claims, cost accounting), Epic Clarity/Caboodle, data models (OMOP, i2b2, PCORnet).
  • Experience with database administration: configuration, performance tuning, partitioning, materialized views, permissions, backups & restorations.
  • Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.

Something wrong with this job? Report it · This flag looks wrong