Job Summary
We are seeking a Data Engineer to support the development and enhancement of a Python-based, AWS-native data ingestion platform responsible for moving financial data from source systems into a third-party application. This role focuses on building scalable, secure, and highly reliable data pipelines while ensuring compliance with enterprise data governance, security, and regulatory requirements.
Key Responsibilities
- Design, develop, and maintain data pipeline components across the full data lifecycle, including ingestion, validation, tokenization, transformation, and publishing.
- Build and support scalable data processing solutions using AWS-native services and modern data engineering frameworks.
- Develop and execute unit, integration, end-to-end, and system testing strategies.
- Partner with cross-functional teams during system and user acceptance testing to ensure data quality and platform reliability.
- Ensure compliance with enterprise data governance, security, and regulatory requirements, including the secure handling of sensitive and highly sensitive data.
- Identify, troubleshoot, and remediate application vulnerabilities and security findings across applications, infrastructure, and third-party dependencies.
- Contribute to CI/CD best practices, source control management, and deployment automation.
- Monitor, troubleshoot, and optimize cloud-native data pipelines and processing platforms.
- Support secure, scalable, and reliable cloud-based data engineering solutions.
Required Qualifications
- Strong proficiency in Python.
- Experience building and maintaining production-grade data processing applications.
- Hands-on experience with AWS services, including Lambda, ECS, S3, and DynamoDB.
- Experience with event-driven and streaming architectures.
- Experience designing and developing data ingestion and transformation pipelines.
- Experience implementing CI/CD pipelines and deployment automation.
- Experience with unit, integration, and end-to-end testing methodologies.
- Strong understanding of data security, governance, and compliance requirements.
- Experience supporting the secure handling of sensitive financial or regulated data.
- Ability to troubleshoot complex cloud-native data pipeline and platform issues.
Preferred Qualifications
- Experience with AWS Flink.
- Experience building event-driven architectures and large-scale streaming platforms.
- Experience supporting financial services or other highly regulated environments.
- Familiarity with Infrastructure as Code (IaC) and cloud-native application development.
#J-18808-Ljbffr