Support requirements for new system capabilities, bulk data pipeline extensions and maintenance, and application enhancements in customer cloud environments.
Build and maintain ETL pipelines that transform structured and unstructured data into enriched, query-friendly indexed data.
Process system and data logs into analytic reports, metrics, and dashboards.
Review data quality and produce ETL design documentation, source and target mappings, and data dictionaries.
Summarised from the posting by AI and can be wrong. The full description is the source.
Requirements
Experience
Demonstrated experience with Spark, PySpark, and Python for big data processing
Demonstrated experience with data mapping, extraction, transformation, and loading
Demonstrated experience building analytic reports in CloudWatch and Kibana or similar tools
Demonstrated experience processing and converting OS and data logs into reports and metrics dashboards
Demonstrated experience with Regular Expressions (RegEx)
Demonstrated experience with SQL, MySQL, and PostgreSQL
Demonstrated experience processing XML and JSON data files
Demonstrated experience with IDEs and data modeling through Notebooks and Visual Studio
Demonstrated experience ETL’ing structured and unstructured data into enriched, query-friendly structured data in indexed files
Demonstrated experience performing extensive data review and data quality analysis
Demonstrated experience developing ETL design documentation, including source and target mapping and data dictionary information
Demonstrated experience interfacing with customers and integration partners to gain and clarify objectives
Demonstrated experience supporting Agile development by contributing to tasking definition, scope, and review
Demonstrated experience deploying capabilities on the Databricks unified analytics platform (preferred)
Demonstrated experience with Elastic MapReduce (EMR) for big data workloads (preferred)
Demonstrated experience joining multiple complex data sets using Spark (preferred)
Demonstrated experience tuning Spark streaming and batch jobs for cluster utilization and speed (preferred)
Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Benefits
Health insurance
Employer contributions to the health care plan
Dental & vision
Medical, dental, and vision coverage
Retirement plan
401(k) and IRA retirement plan with a matching program
Paid time off
Paid vacation, sick time, and public holidays
Learning & tuition
Training and development
Wellness & mental health
Employee Assistance Program
Life & disability insurance
Basic and voluntary life insurance, AD&D, and short-term and long-term disability
Full job description
From Avalore, LLC’s careers site
As a Data Scientist at Avalore, you will support the requirements for new system capabilities, the extension and maintenance of bulk data pipeline and enhancements for multiple applications in the customers' cloud computing environment.
Requirements
Active Security Clearance with appropriate Polygraph
Demonstrated experience utilizing big data processing tech such as Spark, Pyspark, and Python.
Demonstrated experience data mapping, extraction, transformation and loading.
Demonstrated experience building analytic reports in tools such as CloudWatch and Kibana.
Demonstrated experience processing and converting of OS and Data Logs into reports and metrics Dashboards.
Show full descriptionShow less
Demonstrated experience with Regular Expressions (RegEx).
Demonstrated experience with SQL, MySQL, PostgresSQL.
Demonstrated experience with Data file type processing XML, JSON.
Demonstrated experience with IDEs and Data Modeling through Notebooks and Visual Studio.
Demonstrated experience ETL’ing data from disparate structured & unstructured data formats into enriched, query-friendly structured data in indexed files.
Demonstrated experience performing extensive data review and data quality analysis.
Demonstrated experience developing ETL design documentation including source and target mapping and data dictionary information.
Demonstrated experience interfacing with customers and integration partners for gaining and clarifying detailed objectives.
Demonstrated experience supporting Agile development by contributing to tasking definition, scope and review.
Desired:
Demonstrated experience deploying capabilities on the Databricks unified analytics platform.
Demonstrated experience with Elastic Map Reduce to run big data workloads.
Demonstrated experience in joining multiple complex data sets using Spark.
Demonstrated experience in tuning Spark streaming and batch jobs for cluster utilization and speed.
Demonstrated experience in deploying complex, notebook-based pipelines.
Demonstrated experience in Python data analysis libraries such as Pandas.
Demonstrated experience utilizing Cloud services such as Lambda, SNS/SQS, or EC2.
Demonstrated experience with DevOps tools to include Cloudwatch, Lambda, SQS, Dynamo and RDS.
Demonstrated experience working with Elastic, ElasticSearch/Logstash/Kibana (ELK stack).
Work Environment:
Moderate noise (i.e. business office with computers, phone and printers) and /or occasional Loud noise (airfield, large equipment).
Ability to sit at a computer terminal for an extended period of time.
Physical Demands:
While performing the responsibilities of the job, the employee is required to sit, stand, talk, and hear.
Employee is often required to sit and use their hands and fingers to operate a computer.
Travel:
Up to 10%/ Occasional travel.
Benefits
Eligibility requirements apply.
Employer Contributions to Health Care Plan (Medical, Dental & Vision)
Retirement Plan (401k, IRA) with a generous matching program
Life Insurance (Basic, Voluntary & AD&D)
Paid Time Off (Vacation, Sick & Public Holidays)
Short Term & Long Term Disability
Training & Development
Employee Assistance Program
Salary at Avalore is determined by various factors, including but not limited to location, the individual's particular combinaition of education, knowledge, skills, competencies, and experience, as well as contract-specific affordability and organizational requirements.
Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
Avalore will provide reasonable accommodations to applicants who are unable to utilize our online application system due to a disability. Please send your request to hr@avalore.ai.
Avalore is an Equal Opportunity Employer - We not discriminate based on age, race, national or ethnic origin, gender, veterans status, disability, sexual orientation, gender identity or any other legally protected class. Applicants have rights under Federal Employment Laws.