9 of 9 PySpark Jobs in the East of England

added flexibility for diverse migration and integration projects. Prior experience with tools such as MuleSoft, Boomi, Informatica, Talend, SSIS, or custom scripting languages (Python, PySpark, SQL) for data extraction and transformation. Prior experience with Data warehousing and Data modelling (Star Schema or Snowflake Schema). Skilled in security frameworks ...

data workflows using Data Lakes and Logic Apps Write and maintain complex T-SQL queries, stored procedures, and optimised database logic Use Python and PySpark for distributed data processing and large-scale transformation within Fabric and cloud BI pipelines Ensure data accuracy, quality, and adherence to BI governance standards ...

data modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

support reporting, analytics, and business operations. This includes developing ETL/ELT workflows using both low-code tools and code-based approaches (SQL, Python, PySpark), ensuring performance, scalability, and reliability across the data platform. You will take ownership of data modelling and architecture , designing scalable data warehouse and lakehouse … understanding of data modelling (star schema, dimensional modelling, lakehouse design) Experience building and optimising ETL/ELT pipelines using SQL, Python, and/or PySpark Knowledge of data governance, quality, and security best practices Experience implementing CI/CD pipelines and DevOps practices Ability to work independently and manage ...

experience to Design & Develop a ETL pipelines in aws Glue for ingesting data from sql server or other sources Hands on experience on PySpark & Python Experience of implementing change data capture pipelines using aws dms for near-real time or batch ingestion Experience of developing click apply for full ...

high-quality outcomes. Your Responsibilities: Lead the design and implementation of scalable, secure, and cost-efficient ETL/ELT pipelines using AWS Glue, Python (PySpark), and MWAA (Airflow). Define solution architectures, data models, orchestration patterns, and CI/CD for data workflows. Own the technical roadmap, decomposition …/data engineering, with 5+ years leading delivery of production solutions in an AWS data engineering environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs ...

maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...

orchestrate data pipelines across ingestion, transformation, and modelling layers Optimise and monitor data workflows using Data Lakes and Logic Apps Advanced SQL, Python & PySpark Write and maintain complex T-SQL queries, stored procedures, and optimised database logic Use Python and PySpark for distributed data processing and large-scale … Microsoft Fabric (Lakehouse, pipelines, notebooks) Strong understanding of Data Lakes and Logic Apps Advanced metadata-driven ETL/ELT pipeline development Strong Python and PySpark capabilities Advanced T-SQL and SQL performance optimisation Experience with Azure DevOps (Boards, Repos, Deployment Pipelines) Desirable skills Experience with SSIS and SSAS Tabular ...

experience to Design & Develop a ETL pipelines in aws Glue for ingesting data from sql server or other sources Hands on experience on PySpark & Python Experience of implementing change data capture pipelines using aws dms for near-real time or batch ingestion Experience of developing incremental loads Glue … experience to Design & Develop a ETL pipelines in aws Glue for ingesting data from sql server or other sources Hands on experience on PySpark & Python Experience of implementing change data capture pipelines using aws dms for near-real time or batch ingestion Experience of developing incremental loads Glue ...