- Analyze and modernizelegacy ETL processesand translate them into scalableETL/ELT solutions.
- Design, develop, and optimizePySpark and Scala-based Spark jobs.
- Interpret and supportSqoop jobs, including JDBC configurations, incremental load logic, and syntax.
- Develop and manageAWS Glue jobs, crawlers, and workflows.
- Orchestrate Glue jobs usingtriggers, workflows, and scheduling mechanisms.
- Implement complexdata transformation logicincluding joins, filters, aggregations, cleansing, and formatting.
- ApplyProtegrity integrationfor data masking and tokenization of sensitive data during ETL processing.
- Optimize Glue job performance usingpartitioning strategies, parallel execution, and resource tuning.
- Handleschema evolutionand manage changes in source data structures.
- Manage and maintainGlue Data Catalog, including metadata, table definitions, and schema versioning.
- Performunit testing and integration testingfor data pipelines.
- Leaddata migration initiatives, includingS3 to S3 migrations using AWS DataSync.
- Conductsource system analysis, including schema, data types, dependencies, and relationships.
- Design and documentsystem architecturefor individual components and the overall data pipeline.
- Produce and maintaintechnical documentationand manage development lifecycle.
- Support deployed code inproduction environmentsand troubleshoot issues as needed.
We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.
#J-18808-Ljbffr