17 of 17 PySpark Jobs in the Midlands

data modelling and data warehousing concepts (star/snowflake schemas, SCD2, Dimensions/Facts) Proficient in SQL and Python, with hands-on experience using PySpark, Spark SQL, and/or Pandas. Experience in technical documentation of migration processes, data mappings, data quality checks, and testing outcomes. Ability to work ...

pipelines and working with lakehouse architectures Knowledge of Azure tools such as Data Factory and/or Fabric Data Pipelines Experience with Python or PySpark for data transformation Understanding of data modelling (dimensional modelling, medallion architecture) Strong communication skills and ability to work with non-technical stakeholders Desirable: Experience ...

similar Demonstrable experience in programming full scale data and analytic pipelines, storage solutions and process automation functions Highly proficient coding skills in SQL, Python & PySpark Proven knowledge in at least one of the following big data technologies – HIVE, Hadoop, MongoDB, Spark Experience within the Life Science, Medical Research ...

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...

mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena ...

modelling techniques Understanding of metadata management, data governance, and data quality frameworks Proficiency with SQL and ideally one or more programming languages (e.g., Python, PySpark) Experience with database technologies and data warehousing principles (star schema, snowflake, data vault) Experience working with cloud data services specifically with AWS (including Redshift ...

commercial or Agile environment.To be successful in this role it’s essential that you will: Have significant experience of AWS , Python, SQL, Git and PySpark Desirable experience needed will be: SISS or SAS experience Quality Assurance and Test Automation experience Experience of Database technologies Experience in Financial Services organisation ...

engineering within distributed systems Proven ability to build and optimise data pipelines at scale Solid Python skills with exposure to tools such as Spark, PySpark, and Hadoop Strong understanding of data ingestion and batch processing methodologies Experience with text or unstructured data is advantageous but not essential A hands ...

frameworks Engineering Delivery Design and evolve scalable data pipelines (batch and incremental watermark-based ingestion) using: Microsoft Fabric Pipelines Azure Data Factory PySpark/Python Fabric Notebooks Implement and maintain a medallion lakehouse architecture using OneLake and Fabric Warehouses Ensure Direct Lake-ready semantic models for analytics Lead incident … Hands-on experience with Microsoft Fabric, including: Workspaces, OneLake, Fabric Warehouses, Fabric Notebooks and Data Pipelines and Direct Lake semantic models Strong Python and PySpark development capability Solid SQL skills for warehouse development and metadata layers Proven ownership of production data systems end-to-end Wider Azure data platform ...

scalable, high-quality data platform from the ground up. The Role Design and build scalable ETL/ELT pipelines in Microsoft Fabric using PySpark and SQL Develop transformation logic within Fabric Lakehouses using Delta Lake and medallion architecture principles Integrate data from ecommerce, ticketing, and marketing platforms into … best practices in data engineering, documentation, and version control Requirements Strong hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, pipelines) Advanced SQL and strong PySpark experience Proficiency in Python for data processing and API integration Experience building Power BI models and dashboards for business reporting Strong understanding ...

optimise end-to-end data pipelines Act as Databricks SME partnering with stakeholders to gather requirements and drive continuous improvement Develop solutions using PySpark, Spark SQL and Python Implement and maintain CI/CD practices Enhance and optimise existing data pipelines About You Proven experience in Databricks, Delta Lake … PySpark and Spark SQL Excellent communication skills - ability to partner with the business and key stakeholders to champion best practice in data engineering Strong Python and SQL skills Knowledge of CI/CD practices On Offer Salary - up to £75,000 Bonus - up to 15% 27 days annual leave ...

deploying the Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring … logging and alerting for platform reliability. YOUR SKILLS AND EXPERIENCE: Deep expertise in Databricks, PySpark, Python and SQL. Proven experience designing and delivering Lakehouse solutions. Strong DevOps capabilities, including CI/CD and automated testing. Experience with Infrastructure as Code and cloud automation. Familiarity with monitoring, logging and alerting ...

data models within the Lakehouse Owning the transformation from enriched to curated datasets, enabling trusted reporting and insight Developing and maintaining robust SQL and PySpark transformation pipelines in Databricks Embedding data quality, testing, reliability and performance into the curated layer Working closely with data engineers, BI teams and business … Engineer within a modern data platform Excellent data modelling capability, including dimensional and semantic modelling Advanced SQL skills and strong hands on experience with PySpark Experience working with Databricks and Lakehouse architectures A solid grounding in engineering best practices, testing and data quality Confidence mentoring others and taking ownership ...

technical leadership and taking ownership. Key responsibilities include: Designing and building curated datasets using dimensional and semantic modelling techniques Developing and maintaining SQL and PySpark transformation pipelines Partnering closely with Data Engineers Translating business requirements into clear, well-documented data models YOUR SKILLS AND EXPERIENCE: Advanced SQL, including complex … transformations and modelling Databricks PySpark and working within a Lakehouse or modern data platform Dimensional modelling and analytics engineering best practices Communicating effectively with both technical and non-technical stakeholders THE BENEFITS: You will receive a salary, dependent on experience. Salary ...

technical leadership and taking ownership. Key responsibilities include: Designing and building curated datasets using dimensional and semantic modelling techniques Developing and maintaining SQL and PySpark transformation pipelines Partnering closely with Data Engineers Translating business requirements into clear, well-documented data models YOUR SKILLS AND EXPERIENCE: Advanced SQL, including complex … transformations and modelling Databricks PySpark and working within a Lakehouse or modern data platform Dimensional modelling and analytics engineering best practices Communicating effectively with both technical and non-technical stakeholders THE BENEFITS: You will receive a salary, dependent on experience. Salary ...

technical leadership and taking ownership. Key responsibilities include: Designing and building curated datasets using dimensional and semantic modelling techniques Developing and maintaining SQL and PySpark transformation pipelines Partnering closely with Data Engineers Translating business requirements into clear, well-documented data models YOUR SKILLS AND EXPERIENCE: Advanced SQL, including complex … transformations and modelling Databricks PySpark and working within a Lakehouse or modern data platform Dimensional modelling and analytics engineering best practices Communicating effectively with both technical and non-technical stakeholders THE BENEFITS: You will receive a salary, dependent on experience. Salary ...

business. Design and own analytics data models, transforming data from enriched layers into curated, analytics ready datasets. Develop and maintain robust SQL and PySpark transformations within Databricks. Work closely with data engineers, BI teams and stakeholders to translate complex requirements into trusted reporting. SKILLS AND EXPERIENCE The successful Analytics … following skills and experience: Strong commercial experience as an analytics focused engineer, rather than an analyst. Advanced data modelling capability, including dimensional modelling. Strong PySpark and SQL skills in production environments. Hands on experience with Databricks, including pipelines and governed data environments. A solid grounding in engineering best practices ...