Design, implement, and optimize data ingress and egress pathways.
Support efficient, scalable, and reliable processing of analytics workloads.
Use Apache Airflow to orchestrate, schedule, and monitor complex workflows.
Summarised from the posting by AI and can be wrong. The full description is the source.
Requirements
Experience
Extensive expertise in dataflow design, data transport mechanisms, and Apache Spark-based distributed processing
Experience using the Linux CLI and Linux tools
Experience developing Bash scripts to automate manual processes
Recent software development experience using Python and Java
Experience using Apache Airflow for DAG design, scheduling, operators, and sensors
Experience with distributed big data processing engines, including Apache Spark
Experience using Jupyter Notebook
Experience with data wrangling and preprocessing using tools such as pandas and NumPy
Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, and XML
Experience with Git source control
Experience using the Atlassian Tool Suite, including JIRA and Confluence (preferred)
Education
Master’s degree in computer science or a related discipline from an accredited college or university, plus five years of SWE experience on programs and contracts of similar scope, type, and complexity
Bachelor’s degree in computer science or a related discipline from an accredited college or university, plus seven years of SWE experience on programs and contracts of similar scope, type, and depth
Skills
Familiarity with SQL technologies such as MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets
Familiarity with data quality concepts, data validation, and anomaly detection
Familiarity with HPC job scheduling tools, including Slurm (preferred)
Other requirements
Active TS/SCI with Polygraph
Your match
Sign in and add your resume to see which of this job's requirements it shows.
Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.
Full job description
From Red Arch Solutions’ careers site
**Requires an active TS/SCI with Polygraph**
Red Arch Solutions is hiring a Software Engineer on a priority program in Annapolis Junction, MD. The Software Engineer shall possess extensive expertise in dataflow design, data transport mechanisms, and Apache Spark based distributed processing. In this role, the Software Engineer shall be responsible for designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable, and reliable processing of the organization’s analytics workloads.
Position Required Skills:
Experience using the Linux CLI and Linux tools
Experience developing Bash scripts to automate manual processes
Show full descriptionShow less
Recent software development experience using Python and Java
Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
Experience with Distributed Big Data processing engines including Apache Spark
Familiar with SQL technologies such as MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets
Experience using Jupyter Notebook
Experience with data wrangling and preprocessing using tools such as pandas, NumPy
Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML
Familiarity with data quality concepts, data validation, and anomaly detection
Experience with Git Source Control System
Position Desired Skills:
Familiar with HPC Job Scheduling tools including Slurm
Experience using the Atlassian Tool Suite (JIRA, Confluence)
Basic Qualifications:
Active TS/SCI with Polygraph.
Qualifications: Master’s degree in computer science or related discipline from an accredited college or university, plus five (5) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity.
OR Bachelor’s degree in computer science or related discipline from an accredited college or university, plus seven (7) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity OR Nine (9) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity.
Salary range for this position: $205,000-$225,000 #CJ