Data Engineer jobs

Data Consultant

Particle41Remote · US only

Pay not listed

Key facts

Experience
3+ years
Degree
Bachelor's degree

This job is closed.

The employer took it down 8 days ago, after 11 days open. Here are similar open jobs.

  • To check: Posted by the employer 12 months agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • To check: No pay range listedAsk for the pay range before you interview.
  • Checked: On the company's own careers siteGreenhouse board
  • Checked: No reposts foundGreenhouse board
  • Trust score 70 / 100 · Looks good
See all 5 checksShow fewer
How we check

Similar open jobs

Senior Data Engineer

Vericast · Remote · US only

$130k–$150k/yrFull-time

StrongChecked open todayPosted today

Data Engineer

Mindex · Remote · US only

$90k–$140k/yrFull-time

StrongChecked open todayPosted 7 days ago

Data Integration Engineer

D.A. Davidson · Remote · US only

$74k–$106k/yrFull-time

StrongChecked open todayPosted 9 days ago

Principal Data Engineer

Constant Contact · Remote · US, Canada

$138k–$172k/yrFull-time

StrongChecked open todayPosted 11 days ago

Lead Data Engineer

Oddball · Remote · location not stated

$110k–$145k/yr

StrongChecked open todayPosted 15 days ago

Staff Data Engineer

Fanatics Betting & Gaming · Remote · US only

$181k–$238k/yrFull-time

StrongChecked open todayPosted 16 days ago

See more like this

What you’ll do

  • Design, build, and maintain scalable ETL pipelines and data storage solutions.
  • Integrate, validate, cleanse, and transform data from internal and external sources.
  • Gather stakeholder requirements and translate them into technical solutions.
  • Test and debug applications, write unit and integration tests, and resolve defects and performance bottlenecks.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • Over 3 years of data engineering experience

Education

  • Bachelor's degree in Computer Science, Engineering, or a related field

Skills

  • Proficiency in Python
  • Experience with SQL databases such as MySQL or PostgreSQL, and NoSQL databases such as MongoDB
  • Understanding of frameworks and technologies including Flask, API frameworks, data warehousing/lakehouse, databases and ORM, data analysis, Databricks, pandas, Spark, PySpark, machine learning, OpenCV
  • Experience with scikit-learn, logging, requests, subprocess, regex, pytest, ELK stack, Redis, and distributed task queues
  • Understanding of data warehousing/lakehousing principles and concurrent or parallel processing concepts
  • Familiarity with at least one cloud data engineering stack (Azure, AWS, or GCP), and ability to adapt to ETL/ELT tools across cloud providers
  • Competence working on Linux and creating shell scripts
  • Understanding of software engineering principles, design patterns, and best practices

Full job description

From Particle41’s careers site

Data Consultant

Particle41 is seeking a talented and versatile Data Consultant to join our innovative team. As a Data Consultant, you will play a key role in designing, building, and maintaining robust data pipelines and infrastructure to support our clients' data needs. You will work on end-to-end data solutions, collaborating with cross-functional teams to ensure high-quality, scalable, and efficient data delivery. This is an exciting opportunity to contribute to impactful projects, solve complex data challenges, and grow your skills in a supportive and dynamic environment.

In This Role, You Will:

Software Development

Show full descriptionShow less
  • Design, develop, and maintain scalable ETL (Extract, Transform, Load) pipelines to process large volumes of data from diverse sources.
  • Build and optimize data storage solutions, such as data lakes and data warehouses, to ensure efficient data retrieval and processing.
  • Integrate structured and unstructured data from various internal and external systems to create a unified view for analysis.
  • Ensure data accuracy, consistency, and completeness through rigorous validation, cleansing, and transformation processes.
  • Maintain comprehensive documentation for data processes, tools, and systems while promoting best practices for efficient workflows.

Requirements Gathering and Analysis

  • Collaborate with product managers, and other stakeholders to gather requirements and translate them into technical solutions.
  • Participate in requirement analysis sessions to understand business needs and user requirements.
  • Provide technical insights and recommendations during the requirements-gathering process.

Agile Development

  • Participate in Agile development processes, including sprint planning, daily stand-ups, and sprint reviews.
  • Work closely with Agile teams to deliver software solutions on time and within scope.
  • Adapt to changing priorities and requirements in a fast-paced Agile environment.

Testing and Debugging

  • Conduct thorough testing and debugging to ensure the reliability, security, and performance of applications.
  • Write unit tests and validate the functionality of developed features and individual elements.
  • Writing integration tests to ensure different elements within a given application function as intended and meet desired requirements.
  • Identify and resolve software defects, code smells, and performance bottlenecks.

Continuous Learning and Innovation

  • Stay updated with the latest technologies and trends in full-stack development.
  • Propose innovative solutions to improve the performance, security, scalability, and maintainability of applications.
  • Continuously seek opportunities to optimize and refactor existing codebase for better efficiency.
  • Stay up-to-date with cloud platforms such as AWS, Azure, or Google Cloud Platform.

Collaboration

  • Collaborate effectively with cross-functional teams, including testers, and product managers.
  • Foster a collaborative and inclusive work environment where ideas are shared and valued.

Skills and Experience We Value:

  • Bachelor's degree in Computer Science, Engineering, or related field.
  • Proven experience as a Data Engineering, with over of 3 years of experience.
  • Proficiency in Python programming language.
  • Experience with database technologies such as SQL (e.g., MySQL, PostgreSQL) and NoSQL (e.g., MongoDB) databases.
  • Strong understanding of Programming Libraries/Frameworks and technologies such as Flask, API frameworks, datawarehousing/lakehouse, principles, database and ORM, data analysis databricks, panda's, Spark, Pyspark, Machine learning, OpenCV, scikit-learn.
  • Utilities & Tools: logging, requests, subprocess, regex, pytest
  • ELK stack, Redis, distributed task queues
  • Strong understanding of data warehousing/lakehousing principles and concurrent/parallel processing concepts.
  • Familiarity with at least one cloud data engineering stack (Azure, AWS, or GCP) and the ability to quickly learn and adapt to new ETL/ELT tools across various cloud providers.
  • Familiarity with version control systems like Git and collaborative development workflows.
  • Competence in working on Linux OS and creating shell scripts.
  • Solid understanding of software engineering principles, design patterns, and best practices.
  • Excellent problem-solving and analytical skills, with a keen attention to detail.
  • Effective communication skills, both written and verbal, and the ability to collaborate in a team environment.
  • Adaptability and willingness to learn new technologies and tools as needed.

About Particle41 Our core values of Empowering, Leadership, Innovation, Teamwork, and Excellence drive everything we do to achieve the ultimate outcomes for our clients. Empowering Leadership for Innovation in Teamwork with Excellence ( ELITE )

  • E - Empowering: Enabling individuals to reach their full potential.
  • L - Leadership: Taking initiative and guiding each other toward success.
  • I - Innovation: Embracing creativity and new ideas to stay ahead.
  • T - Teamwork: Collaborating with empathy to achieve common goals.
  • E - Excellence: Striving for the highest quality in everything we do.

We seek team members who embody these values and are committed to contributing to our mission. Particle41 welcomes individuals from all backgrounds who are committed to our mission and values. We provide equal employment opportunities to all employees and applicants, ensuring that hiring and employment decisions are based on merit and qualifications without discrimination based on race, color, religion, caste, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, local, or international laws. This policy applies to all aspects of employment and hiring.

We appreciate your interest and encourage applicants from these regions to apply. If you need any assistance during the application or interview process, please feel free to reach out to us at careers@Particle41.com. This is specifically for USA location only.

Something wrong with this job? Report it · This flag looks wrong