As a Data Engineer at QuintaGroup, you will design and optimize data pipelines within the AWS ecosystem for a US-based B2B marketplace platform. The platform simplifies and accelerates business operations by providing seamless data solutions and advanced analytics. You’ll collaborate with data scientists, analysts, and cross-functional teams to deliver innovative results.
Key Responsibilities:
• Develop, implement, and optimize data pipelines using PySpark in AWS environments.
• Utilize AWS services such as S3, Glue, Lambda, EMR to create scalable and efficient data solutions.
• Enhance PySpark workflows for performance, reliability, and cost-effectiveness.
• Maintain data quality through rigorous testing and monitoring processes.
• Apply data governance, security, and compliance best practices.
• Document workflows, processes, and designs to support team collaboration and maintenance.
Requirements:
• 3+ years of experience in data engineering, with a focus on PySpark.
• Strong experience with AWS services.
• Proficiency in Python and related frameworks or libraries.
• Solid understanding of distributed computing and Apache Spark.
• Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation) is a plus.
• Strong analytical and problem-solving skills with attention to detail.
• Excellent communication skills and ability to work in dynamic, team-oriented environments.
• Upper-Intermediate level of English.
Tech Stack:
• Programming Languages: Python (with Pandas and PySpark).
• AWS Services: S3, Glue (Glue Data Catalog, Glue Crawler, Glue Jobs with PySpark), Lambda, ECS, Athena, Aurora (RDS), AppConfig, API Gateway, Step Functions, Quicksight, EventBridge.
• Infrastructure: Terraform for Infrastructure-as-Code.
We Offer:
• Flexible working format: remote, office-based, or hybrid.
• Competitive salary and compensation package.
• Personalized career growth and mentorship programs.
• Professional development tools, including tech talks and training sessions.
• Access to active tech communities and regular knowledge sharing.
