8 of 8 Permanent PySpark Jobs in the West Midlands

using frameworks like FastAPI, Flask, or Django Writing efficient, scalable, and maintainable code following best practices Working with data processing libraries such as Pandas, PySpark, and NumPy Implementing data validation, logging, and error handling mechanisms Experience with asynchronous programming and performance optimization Building microservices-based architectures Cloud Technologies Proven ...

complex environments Strong experience with Microsoft Fabric (including OneLake, Lakehouse, Data Pipelines, Dataflows, Power BI models) Proficiency in SQL (including optimisation) and Python/PySpark for data transformation Experience with ETL/ELT processes using Fabric Data Pipelines and/or Azure Data Factory Solid understanding of data modelling ...

pipelines and working with lakehouse architectures Knowledge of Azure tools such as Data Factory and/or Fabric Data Pipelines Experience with Python or PySpark for data transformation Understanding of data modelling (dimensional modelling, medallion architecture) Strong communication skills and ability to work with non-technical stakeholders Desirable: Experience ...

mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena ...

engineering within distributed systems Proven ability to build and optimise data pipelines at scale Solid Python skills with exposure to tools such as Spark, PySpark, and Hadoop Strong understanding of data ingestion and batch processing methodologies Experience with text or unstructured data is advantageous but not essential A hands ...

scalable, high-quality data platform from the ground up. The Role Design and build scalable ETL/ELT pipelines in Microsoft Fabric using PySpark and SQL Develop transformation logic within Fabric Lakehouses using Delta Lake and medallion architecture principles Integrate data from ecommerce, ticketing, and marketing platforms into … best practices in data engineering, documentation, and version control Requirements Strong hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, pipelines) Advanced SQL and strong PySpark experience Proficiency in Python for data processing and API integration Experience building Power BI models and dashboards for business reporting Strong understanding ...

optimise end-to-end data pipelines Act as Databricks SME partnering with stakeholders to gather requirements and drive continuous improvement Develop solutions using PySpark, Spark SQL and Python Implement and maintain CI/CD practices Enhance and optimise existing data pipelines About You Proven experience in Databricks, Delta Lake … PySpark and Spark SQL Excellent communication skills - ability to partner with the business and key stakeholders to champion best practice in data engineering Strong Python and SQL skills Knowledge of CI/CD practices On Offer Salary - up to £75,000 Bonus - up to 15% 27 days annual leave ...

high-quality, governed, and observable data platform . 🔧 What You’ll Be Doing Build and maintain Azure & Databricks pipelines Develop scalable ELT processes using PySpark Implement data quality, lineage, and security controls Own CI/CD pipelines and Infrastructure as Code (Terraform) Ensure pipelines are testable, observable, and easy … best practices 🧠 What We’re Looking For Strong experience with Azure Data Platform & Databricks Proven track record building scalable data pipelines Hands-on with PySpark/Spark-based processing Experience with Terraform & CI/CD pipelines Solid understanding of data governance, quality, and lineage Familiarity with Git-based workflows ...