4 of 4 Permanent PySpark Jobs in the West Midlands

pipelines and working with lakehouse architectures Knowledge of Azure tools such as Data Factory and/or Fabric Data Pipelines Experience with Python or PySpark for data transformation Understanding of data modelling (dimensional modelling, medallion architecture) Strong communication skills and ability to work with non-technical stakeholders Desirable: Experience ...

mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena ...

data pipelines within Azure Databricks and Azure Lakehouse environments. Work within established medallion architecture patterns. Develop data flows using Azure Data Factory , Python/PySpark, SQL, and Azure data services. Support data modelling, data integration, and data product development. Implement data quality, validation, lineage, and integrity controls. Work with … Strong hands-on experience with Azure Databricks . Experience with lakehouse platforms and medallion architecture. Strong Azure Data Factory, Azure Data Lake, Python/PySpark, SQL, MS SQL/Azure SQL experience. Good understanding of data modelling, ETL/ELT, data quality, governance, and lineage. Experience working in financial ...

scalable, high-quality data platform from the ground up. The Role Design and build scalable ETL/ELT pipelines in Microsoft Fabric using PySpark and SQL Develop transformation logic within Fabric Lakehouses using Delta Lake and medallion architecture principles Integrate data from ecommerce, ticketing, and marketing platforms into … best practices in data engineering, documentation, and version control Requirements Strong hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, pipelines) Advanced SQL and strong PySpark experience Proficiency in Python for data processing and API integration Experience building Power BI models and dashboards for business reporting Strong understanding ...