Requirements

• 8+ years of hands-on experience in delivering data engineering solutions
• Experience leading a team of 5+ team members, responsible for technical project management including stakeholder management, design review, code reviews and mentoring
• Strong hands-on experience in AWS, with experience in data engineering related services and technologies including AWS Glue, Lambda, Amazon Redshift, EMR, Amazon Kinesis etc.
• Expertise in database concepts, SQL and relational modelling
• Good understanding and experience working with NoSQL databases
• Strong hands-on experience in implementing data pipelines using Python and PySpark
• Good understanding of data lake and data warehouse concepts, modern data architecture patterns and industry trends in data engineering space
• Hands-on experience in data modelling, with a good understanding of both ER modelling and dimensional modelling
• Good understanding and experience implementing data quality management solutions
• Good understanding of metadata management for data engineering solutions and data platforms
• Good understanding of data governance concepts – including data stewardship, data security and data privacy management
• Experience with version control tools like Git for managing version control for source code
• Experience with DevOps processes, including experience implementing CI/CD pipelines for data engineering solution components
• Strong analytical and problem-solving skills
• Excellent communication and collaboration skills

Responsibilities

• Lead technical architecture and design efforts for data engineering solution components for delivering business solutions
• Design and implement scalable, secure and resilient data engineering solutions on AWS, with a good understanding of data architecture patterns
• Evaluate and select appropriate technology stack for data engineering solution components – cloud native services, open-source tools, data storage, data processing frameworks etc
• Lead, mentor and manage a team of dynamic data engineers for development, testing and implementation of data pipelines ensuing quality, performance and on-time delivery
• Responsible for detailed technical design and data mapping specification
• Ensure adherence to industry best practices, coding standards and guidelines driving continuous improvement in code quality and delivery excellence
• Optimize data solutions for performance, cost-efficiency and scalability
• Ensure data quality and integrity for downstream systems and applications
• Plan, prioritize and manage data engineering projects, ensuring timely delivery with quality
• Identify and mitigate risks, troubleshoot issues, handle customer escalations
• Ensure alignment with business requirements and priorities, working closely with stakeholders – both internal and customer
• Maintain comprehensive technical documentation, including architecture diagrams, high level / low level data integration design, data mapping specifications, database design etc
• Working closely with peers, BI engineers and data scientists to enable advanced analytics, machine learning models and reporting solutions