GCP Data Engineer

Key Responsibilities

• Pipeline Development: Design, develop, and maintain robust end-to-end data pipelines and orchestration workflows using Python, Apache Airflow, and Cloud Composer on GCP.
• Data Modernization: Bridge the gap between legacy/on-premise database environments and modern cloud architectures, ensuring seamless data ingestion and migration.
• Optimization & Performance: Architect and tune analytics solutions at scale within BigQuery, utilizing partitioning, clustering, and materialized views to maximize efficiency and control costs.
• CI/CD & Software Best Practices: Embed rigorous software engineering principles, including Git/GitHub version control, strict PR disciplines, automated unit/integration testing, and deployment via CI/CD pipelines.
• Data Governance & Compliance: Embed data quality controls, access management, data lineage, and privacy frameworks required to meet stringent prudential/regulatory banking compliance standards.
• Collaboration: Partner with cross-functional Agile teams, including Product Owners, DevOps Engineers, and Business SMEs, to translate complex technical requirements into business-ready assets.

Required Technical Skills & Experienc

• eGCP Core Stack: Proven production-grade experience building and managing cloud data solutions on Google Cloud Platfor m (specifically BigQuery, Cloud Composer/Airflow, Cloud Storag e, and Dataflo w)
• .Programming & Scripting: Strong hands-on coding capability in Pytho n for data engineering, service development, and pipeline automation
• .Data Transformation & Modeling: Proficient in SQL, data modeling (Star Schema, Data Vault, normalisation), and data transformation frameworks like db t
• .Regulated Environment Experience: Prior experience working within UK financial services, banking, or a similarly heavily regulated framewor k handling sensitive/critical data assets is highly preferred
• .Containerization (Desirable): Experience or familiarity with containerizing and operating services using Docke rand Kubernetes (GKE)
• .Legacy Systems (Desirable): Familiarity with enterprise legacy platforms (e.g., Informatica, Teradata, Mainframe) to assist with migration patterns is an added advantage