1 to 25 of 77 PySpark Jobs in England

/version control Strong communication skills with both technical and non-technical stakeholders Desirable Skills: Microsoft Fabric Azure Synapse Analytics Databricks Apache Spark/PySpark dbt Apache Airflow Power BI Azure DevOps CI/CD pipelines Terraform What's On Offer: £65k+ salary package Hybrid working model Company bonus ...

years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data ...

. Strong background in ETL/ELT , data warehousing concepts, dimensional modelling and modern data architecture patterns. Hands-on skills across T-SQL , Python , PySpark , and developing robust data processing solutions. Experience optimising performance across databases, transformation workloads and distributed data platforms. Familiarity with CI/CD , Azure DevOps ...

Azure Databricks, Databricks, Azure Data Lake, ADLS, Azure SQL, Azure Synapse, Synapse Analytics, Azure PaaS, ETL, ELT, Data Pipelines, Data Integration, SQL, Python, PySpark, Azure DevOps, CI/CD, Data Governance, Data Quality, Cloud Security, Microsoft Fabric, Power BI, Power Platform. ...

Experience working across cross-functional teams including Architecture, Product, Engineering, and Business stakeholders. Data Engineering & Platforms Databricks Delta Lake Unity Catalog dbt Spark/PySpark Data pipeline development and optimisation Lakehouse architecture Data integration and ETL/ELT frameworks Data platform design and implementation Cloud Technologies AWS S3 Glue ...

architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps & Platform ...

skills (Conceptual, Logical, Physical and Dimensional)Extensive Azure Databricks experience within large-scale enterprise environmentsDatabricks Lakehouse Architecture and Medallion Design PatternsDatabricks Data Engineering using PySpark and Spark SQLDelta Live Tables (DLT)Unity Catalog implementation and governanceDatabricks WorkflowsDatabricks Asset Bundles (DABs)Databricks Solution AcceleratorsMetadata-Driven Ingestion and Orchestration FrameworksMLflow, Feature ...

Python/Data Engineering professionals with a solid Azure foundation and the ability to learn and work across the wider platform. Desirable Databricks/PySpark NoSQL/Cosmos DB Infrastructure as Code/DevOps automation AI/ML data integration Automotive or insurance industry experience If this sounds ...

APIs, Prometheus, or Azure Monitor.• Maintain best practices for data engineering performance, testing, and documentation.Technical Skills Required• Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake.• Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies.• Experience tuning job configurations (e.g., memory management, parallelism ...

analytics libraries (e.g., pandas, numpy, matplotlib)Exposure to DevOps, DataOps, and MLOps concepts and best practicesExperience with core technologies such as Python, PySpark, SQL, Airflow, Databricks, Kedro, Dask/RAPIDS, Docker, Kubernetes, and cloud services (AWS/GCP/Azure)Experience with Generative AI (GenAI) or agentic systems ...

clear technical tasks. Mentorship: A proactive mindset, eager to provide guidance to less experienced developers and support collaborative pair-programming. Desirable Skills Experience with PySpark or PySQL. Experience in Dimensional Modelling and data warehousing principles. Experience with SQL Server SSAS Tabular and DAX. Exposure to SQL Server RDBMS ...

Looking ForProven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions.Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks.Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern Data Lakehouse ...

pipelines. Experience working with cloud data platforms (AWS preferred). Exposure to data warehousing, data lakes and modern data architecture. Python experience (including PySpark) is beneficial but not essential. Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models support ...

platforms (AWS preferred). Exposure to data warehousing, data lakes and modern data architecture. Understanding of workflow orchestration and automation tools. Python experience (including PySpark) is beneficial but not essential. Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models ...

designs & builds data solutions and mentor junior engineers. Were looking for someone who can offer the following skills & experience. Strong experience in Python, PySpark/Spark, Databricks, Azure Data Factory, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest. Experience designing, building and supporting scalable data pipelines ...

Databricks Azure Data Lake/Lakehouse Good understanding of data architecture and data flows . Practical SQL experience. Experience with Python and/or PySpark . Experience with CI/CD and DevOps . Experience with Power BI and/or Power Apps . Strong understanding of data ingestion ...

written and spoken English skills, with the ability to ask questions, learn from feedback, and discuss technical ideas. Desirable (Nice to Have): Familiarity with PySpark or relational database management systems (SQL Server RDBMS). An interest in data warehousing principles or dimensional modeling. A passion for exploring how Artificial ...

support migration planning, solution design and adoption. Data Engineering and Integration Design, build and optimise scalable data ingestion and transformation solutions using Python, PySpark, SQL and Delta Lake. Implement batch and incremental processing patterns, including Change Data Capture (CDC), schema evolution, reconciliation processes, data quality controls and error handling. ...

supporting ETL/ELT frameworks. Strong understanding of data warehousing concepts and dimensional modelling. AWS experience, including services such as S3, Glue, EMR, or PySpark . Knowledge of data governance, quality controls, and testing practices. Experience working within large-scale enterprise environments. Desirable Knowledge of Iceberg or modern data ...

Microsoft Fabric, Azure Data Platform technologies, and modern cloud data architectures Hands-on experience with Databricks, Unity Catalog, and Delta Lake Advanced SQL and PySpark development skills Experience implementing Medallion Architecture principles Strong data modelling expertise, including Star Schema and Data Vault methodologies Solid understanding of data governance, security ...

with Databricks SQL endpoints for data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/ ...

fields. Proven track record of designing, developing, and deploying AI systems in production environments. Proficient in Python and key ML libraries (e.g. PyTorch, PySpark, scikit-learn, Hugging Face Transformers). Hands-on experience with modern data platforms and AI tooling such as Azure ML, Databricks, MLflow, LangChain, LangGraph. Proven ...

deliver dashboards that meet Barclays standards and business needs.Some other highly valued skills may include:Experience with SQL and database technologies.Knowledge of Python/PySpark and data analytics.Experience in a Call Centre or Contact Centre environment.Strong communication and stakeholder management skills.Ability to collaborate effectively across technical and business teams.You ...

deliver dashboards that meet Barclays standards and business needs.Some other highly valued skills may include:Experience with SQL and database technologies.Knowledge of Python/PySpark and data analytics.Experience in a Call Centre or Contact Centre environment.Strong communication and stakeholder management skills.Ability to collaborate effectively across technical and business teams.You ...

Sciences value chain. Core Architecture: Expertise in Data Modernization, Microservices, Cloud Architecture, and Lakehouse patterns. Data & AI: Deep hands-on proficiency in Databricks (Spark, PySpark, DLT), GenAI (Agent Core, Copilot Studio, Amazon Bedrock), and Graph Databases (Neo4j). Cloud & Tools: Strong command of AWS (Lambda, Glue, EMR, MWAA ...