Data Engineer jobs

Apache Spark Developer

Bright Vision TechnologiesRemote · US onlyFull-timeFirst seen 20 days ago

$125k–$185k/yr listed by the employer

About the typical $155k a year for senior remote Data Engineer jobsHow we compare

How we compare. Typical is the median yearly pay of 139 senior remote Data Engineer jobs that list pay, from the employers' own postings. About means within 5%.

Key facts

Experience
6+ years
US visa
No US visa sponsorship

This job is closed.

The employer took it down 19 days ago, after 1 day open. Here are similar open jobs.

  • Checked: On the company's own careers siteJazzHR board
  • Checked: Pay listedJob posting
  • Checked: No reposts foundJazzHR board
  • Note: Last seen open 20 days agoLive check
  • Trust score 100 / 100 · Strong
Posting history
  1. Posted Nov 1, 2026 · taken down Nov 2, 2026
  2. Reposted #1 Nov 2, 2026 · taken down Nov 2, 2026
See all 5 checksShow fewer
How we check

Similar open jobs

Senior Data Engineer

Vericast · Remote · US only

$130k–$150k/yrFull-time

StrongChecked open todayPosted yesterday

Senior Data Engineers

Movement Labs · Remote · US only

$120k–$150k/yrFull-time

StrongChecked open todayPosted 7 days ago

Data Engineer

Mindex · Remote · US only

$90k–$140k/yrFull-time

StrongChecked open todayPosted 8 days ago

Senior DataOps Engineer

Voyager Technologies, Inc. · Remote · US only

$140k–$180k/yr

StrongChecked open todayPosted 9 days ago

Data Integration Engineer

D.A. Davidson · Remote · US only

$74k–$106k/yrFull-time

StrongChecked open todayPosted 10 days ago

Lead Analytics Engineer

The Knot Worldwide · Remote · US only

$135k–$231k/yr

StrongChecked open todayPosted 11 days ago

See more like this

What you’ll do

  • Design, develop, and maintain distributed Spark applications and scalable batch and real-time ETL/ELT pipelines.
  • Optimize Spark jobs for memory use, partitioning, shuffle performance, and execution efficiency.
  • Process structured, semi-structured, and streaming data; build reusable processing frameworks and metadata-driven ingestion pipelines.
  • Implement data quality checks, monitoring, error handling, and integrations with warehouses, lakehouses, and reporting platforms.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Licenses and certifications

  • Cloud certifications in Azure, AWS, Databricks, or Apache Spark-related technologies (preferred)

Experience

  • 6+ years of professional software or data engineering experience
  • 4+ years of hands-on Apache Spark development experience in enterprise production environments
  • Experience with distributed computing concepts, including partitioning, shuffling, caching, broadcast joins, and fault tolerance
  • Experience with Hadoop ecosystem technologies, including Hive, HDFS, YARN, and Parquet
  • Experience processing streaming data using Spark Structured Streaming, Apache Kafka, or Event Hubs
  • Hands-on experience with cloud platforms including Azure Databricks, AWS EMR, AWS Glue, Azure Synapse Analytics, or Google Dataproc
  • Experience integrating Spark applications with Delta Lake, Apache Iceberg, or Apache Hudi
  • Experience using Git, CI/CD pipelines, Azure DevOps, GitHub Actions, or Jenkins
  • Experience working in Agile Scrum development environments
  • Experience building enterprise Lakehouse architectures using Databricks or Delta Lake (preferred)
  • Experience with machine learning workflows using Spark MLlib, MLflow, or feature engineering pipelines (preferred)
  • Experience implementing data quality frameworks using Great Expectations or Deequ (preferred)
  • Experience working with cloud object storage including Amazon S3, Azure Data Lake Storage (ADLS Gen2), or Google Cloud Storage (preferred)
  • Experience with enterprise monitoring tools including Prometheus, Grafana, Datadog, or OpenTelemetry (preferred)

Skills

  • Proficiency in PySpark, Scala, or Spark SQL for distributed data processing
  • Understanding of Apache Spark architecture, including RDDs, DataFrames, Datasets, Catalyst Optimizer, DAG execution, and Tungsten engine
  • Advanced SQL skills with databases such as SQL Server, Oracle, PostgreSQL, Snowflake, or Teradata
  • Understanding of data warehousing concepts, dimensional modeling, and data lake architecture
  • Strong debugging, troubleshooting, and Spark performance tuning skills
  • Familiarity with Apache Airflow, Azure Data Factory, AWS Step Functions, or Control-M for workflow orchestration (preferred)
  • Knowledge of Kubernetes, Docker, and containerized Spark deployments (preferred)
  • Familiarity with Apache NiFi, Apache Flink, Trino, or Presto (preferred)
  • Knowledge of Infrastructure as Code using Terraform or ARM templates (preferred)

Full job description

From Bright Vision Technologies’ careers site

Apache Spark Developer – Remote

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States.

This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential.

Job Title: Apache Spark Developer Location: 100% Remote (U.S.) Position Type: Full-time, Direct W2 Salary Range: $125,000–$185,000 Annually Experience Required: 6+ years

Sponsorship: U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position.

Show full descriptionShow less

Job Summary We are seeking an experienced Apache Spark Developer to design, develop, and optimize large-scale distributed data processing applications supporting enterprise analytics, machine learning, real-time reporting, and cloud-based data platforms. This role focuses on building high-performance Spark applications capable of processing billions of records across structured and semi-structured data sources while delivering scalable, reliable, and cost-efficient data pipelines. You will work closely with data architects, data engineers, cloud platform teams, machine learning engineers, and business intelligence developers to build modern data processing solutions leveraging Apache Spark, cloud-native technologies, and distributed computing frameworks. The ideal candidate possesses deep expertise in Spark architecture, distributed systems, performance optimization, and cloud-based big data ecosystems.

Key Responsibilities

  • Design, develop, and maintain high-performance distributed data processing applications using Apache Spark.
  • Build scalable batch and real-time ETL/ELT pipelines processing large volumes of enterprise data.
  • Develop Spark applications using PySpark, Scala, or Spark SQL for data transformation, aggregation, and analytics.
  • Optimize Spark jobs for memory utilization, partitioning strategies, shuffle performance, and execution efficiency.
  • Process structured, semi-structured, and streaming data from enterprise databases, APIs, Kafka, cloud storage, and data lakes.
  • Develop reusable Spark libraries, data processing frameworks, and metadata-driven ingestion pipelines.
  • Collaborate with cloud engineering teams to deploy Spark workloads on Databricks, EMR, Azure Synapse, or Kubernetes.
  • Implement data quality validation, reconciliation, monitoring, and automated error handling across distributed pipelines.
  • Integrate Spark applications with enterprise data warehouses, lakehouses, and reporting platforms.
  • Participate in architecture reviews, code reviews, technical design discussions, and Agile development activities.
  • Troubleshoot production issues involving distributed processing, cluster performance, resource utilization, and data quality.
  • Support cloud migration initiatives by modernizing legacy ETL workloads into Spark-based architectures.

Required Skills

  • Six or more years of professional software or data engineering experience.
  • Four or more years of hands-on Apache Spark development experience in enterprise production environments.
  • Strong proficiency in PySpark, Scala, or Spark SQL for distributed data processing.
  • Deep understanding of Apache Spark architecture including RDDs, DataFrames, Datasets, Catalyst Optimizer, DAG execution, and Tungsten engine.
  • Strong experience with distributed computing concepts including partitioning, shuffling, caching, broadcast joins, and fault tolerance.
  • Advanced SQL skills with databases such as SQL Server, Oracle, PostgreSQL, Snowflake, or Teradata.
  • Experience working with Hadoop ecosystem technologies including Hive, HDFS, YARN, and Parquet.
  • Experience processing streaming data using Spark Structured Streaming, Apache Kafka, or Event Hubs.
  • Hands-on experience with cloud platforms including Azure Databricks, AWS EMR, AWS Glue, Azure Synapse Analytics, or Google Dataproc.
  • Experience integrating Spark applications with Delta Lake, Apache Iceberg, or Apache Hudi.
  • Strong understanding of data warehousing concepts, dimensional modeling, and data lake architecture.
  • Experience using Git, CI/CD pipelines, Azure DevOps, GitHub Actions, or Jenkins.
  • Strong debugging, troubleshooting, and Spark performance tuning skills.
  • Experience working in Agile Scrum development environments.

Preferred Qualifications

  • Experience building enterprise Lakehouse architectures using Databricks or Delta Lake.
  • Familiarity with Apache Airflow, Azure Data Factory, AWS Step Functions, or Control-M for workflow orchestration.
  • Experience with machine learning workflows using Spark MLlib, MLflow, or feature engineering pipelines.
  • Knowledge of Kubernetes, Docker, and containerized Spark deployments.
  • Experience implementing Data Quality frameworks using Great Expectations or Deequ.
  • Familiarity with Apache NiFi, Apache Flink, Trino, or Presto.
  • Experience working with cloud object storage including Amazon S3, Azure Data Lake Storage (ADLS Gen2), or Google Cloud Storage.
  • Knowledge of Infrastructure as Code using Terraform or ARM templates.
  • Experience with enterprise monitoring tools including Prometheus, Grafana, Datadog, or OpenTelemetry.
  • Cloud certifications in Azure, AWS, Databricks, or Apache Spark-related technologies are highly desirable.

Project Environment You will be joining a modern data engineering team responsible for building cloud-native big data platforms supporting enterprise analytics, AI, and business intelligence initiatives. Current projects include:

  • Enterprise data lakehouse implementation using Databricks and Delta Lake
  • Real-time streaming analytics processing billions of daily events
  • Large-scale customer analytics and behavioral data platforms
  • Financial risk modeling and fraud detection pipelines
  • Healthcare clinical and operational analytics solutions
  • Cloud migration of legacy Hadoop and ETL workloads
  • Machine learning feature engineering and model training pipelines
  • Enterprise reporting platforms supporting executive dashboards and self-service analytics
  • Distributed data processing infrastructure deployed on Azure and AWS

This is a hands-on engineering role where you will contribute to distributed system architecture, Spark application development, cloud migration, performance optimization, production support, and continuous improvement of enterprise-scale data processing platforms.

How to Apply

Would you like to know more about this opportunity? For immediate consideration, please send your resume to Harry@bvteck.com or contact us at (908) 676-4399. Learn more about Bright Vision Technologies at www.bvteck.com.

Bright Vision Technologies is an Equal Opportunity Employer.

Something wrong with this job? Report it