3 of 3 PySpark Jobs in East Anglia

support reporting, analytics, and business operations. This includes developing ETL/ELT workflows using both low-code tools and code-based approaches (SQL, Python, PySpark), ensuring performance, scalability, and reliability across the data platform. You will take ownership of data modelling and architecture , designing scalable data warehouse and lakehouse … understanding of data modelling (star schema, dimensional modelling, lakehouse design) Experience building and optimising ETL/ELT pipelines using SQL, Python, and/or PySpark Knowledge of data governance, quality, and security best practices Experience implementing CI/CD pipelines and DevOps practices Ability to work independently and manage ...

high-quality outcomes. Your Responsibilities: Lead the design and implementation of scalable, secure, and cost-efficient ETL/ELT pipelines using AWS Glue, Python (PySpark), and MWAA (Airflow). Define solution architectures, data models, orchestration patterns, and CI/CD for data workflows. Own the technical roadmap, decomposition …/data engineering, with 5+ years leading delivery of production solutions in an AWS data engineering environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs ...

experience to Design & Develop a ETL pipelines in aws Glue for ingesting data from sql server or other sources Hands on experience on PySpark & Python Experience of implementing change data capture pipelines using aws dms for near-real time or batch ingestion Experience of developing incremental loads Glue … experience to Design & Develop a ETL pipelines in aws Glue for ingesting data from sql server or other sources Hands on experience on PySpark & Python Experience of implementing change data capture pipelines using aws dms for near-real time or batch ingestion Experience of developing incremental loads Glue ...