1 to 25 of 149 PySpark Jobs in England

/version control Strong communication skills with both technical and non-technical stakeholders Desirable Skills: Microsoft Fabric Azure Synapse Analytics Databricks Apache Spark/PySpark dbt Apache Airflow Power BI Azure DevOps CI/CD pipelines Terraform What's On Offer: £65k+ salary package Hybrid working model Company bonus ...

looking for an experienced Data Engineer with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark or Scala, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta ...

/AI services). Desirable Experience Big Data and Cloud Processing: Hands-on experience with AWS (EMR, S3, Lambda, Redshift, Athena, CloudWatch) and PySpark/Python for large-scale batch processing. DevOps and CI/CD: Familiarity with automated testing pipelines, Infrastructure as Code (Terraform), and Git/GitLab ...

Azure, AKS, Azure Container Apps, Azure Monitor, Azure SQL, Cosmos DB, Azure Data Lake, Fabric/OneLake.Data & Analytics: Databricks, Unity Catalog, Azure Data Factory, PySpark, Python, SQL, Power BI, data pipelines, transformations, reconciliation, and data quality.Application & Integration: .NET/.NET Core, Angular, REST APIs, microservices, authentication, authorization, and secure ...

skills (Conceptual, Logical, Physical and Dimensional)Extensive Azure Databricks experience within large-scale enterprise environmentsDatabricks Lakehouse Architecture and Medallion Design PatternsDatabricks Data Engineering using PySpark and Spark SQLDelta Live Tables (DLT)Unity Catalog implementation and governanceDatabricks WorkflowsDatabricks Asset Bundles (DABs)Databricks Solution AcceleratorsMetadata-Driven Ingestion and Orchestration FrameworksMLflow, Feature ...

data quality best practice.Strong problem-solving skills and attention to detail.The ability to communicate effectively with both technical and business stakeholders.Desirable ExperiencePython or PySpark development within Microsoft Fabric.CI/CD practices for data engineering and data product delivery.Experience within legal services or another professional services environment.Exposure to systems such ...

Python/Data Engineering professionals with a solid Azure foundation and the ability to learn and work across the wider platform. Desirable Databricks/PySpark NoSQL/Cosmos DB Infrastructure as Code/DevOps automation AI/ML data integration Automotive or insurance industry experience If this sounds ...

have To be considered for this role, we would love you to have: Extensive experience in programming/query language skills including Python/PySpark and SQL. Data engineering experience - via data preparation, transformation & conversion to allow for rapid processing of system data across a number of different formats ...

will bring strong expertise in Power BI, Microsoft Fabric and associated languages such as DAX, SQL and Power Query. Familiarity with Azure DevOps and PySpark/Python would be beneficial as would experience of working with cloud-based data solutions. Principal Responsibilities Report Development: Develop and deploy Power ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

analytics libraries (e.g., pandas, numpy, matplotlib)Exposure to DevOps, DataOps, and MLOps concepts and best practicesExperience with core technologies such as Python, PySpark, SQL, Airflow, Databricks, Kedro, Dask/RAPIDS, Docker, Kubernetes, and cloud services (AWS/GCP/Azure)Experience with Generative AI (GenAI) or agentic systems ...

best practices, including DevOps, DataOps, and MLOps, will be advantageousWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such ...

skills to influence stakeholders as well as present complex findings clearly to both technical and non-technical audiencesStrong coding skills in Python (e.g. OOP, PySpark, Pandas, NumPy, etc) and experience with SQLStrong statistical & analytical skills (e.g. sampling methods), attention to detail, and a proactive empirical mindsetFamiliarity with data modelling ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...

Looking ForProven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions.Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks.Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern Data Lakehouse ...

data feed pipeline migrations repointing from an existing Data Warehouse to a new AWS Data Lakehouse architecture, underpinned with AI ETL accelerators, Python/PySpark, SQL and YAML configs.Test initial bulk then deltas from legacy to new.Execute automated test suites as part of sprint-based delivery and regression cyclesDefect ...

Knowledge of data warehousing and modern data architectures Strong problem-solving and stakeholder communication skills Desirable Skills Experience with Azure Data Factory Python or PySpark development experience Experience working within cloud data transformation programmes Understanding of data governance and security best practices Contract Details 6-Month Contract ...

such as S3, Kinesis, DMS, SNS, Glue, Athena, RDS • Experience with Data Warehouse databases like Snowflake/Redshift/BigQuery • Experience with Python/PySpark • Experience with implementing Rest APIs. • Experience with using and integrating BI tools such as QuickSight, Looker, Power BI, Tableau • Business Intelligence (BI) knowledge/ ...

pipelines. Experience working with cloud data platforms (AWS preferred). Exposure to data warehousing, data lakes and modern data architecture. Python experience (including PySpark) is beneficial but not essential. Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models support ...

platforms (AWS preferred). Exposure to data warehousing, data lakes and modern data architecture. Understanding of workflow orchestration and automation tools. Python experience (including PySpark) is beneficial but not essential. Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models ...

hands-on experience with modern cloud data platforms (e.g. Databricks or Snowflake), ideally within AWS environments. Advanced programming and data engineering skills in Python, PySpark and SQL, with a solid understanding of ETL pipelines and data transformation. Proven experience in data modelling (end-to-end), including performance optimisation using ...

with cloud data platforms (AWS preferred).Exposure to data warehousing, data lakes and modern data architecture.Understanding of workflow orchestration and automation tools.Python experience (including PySpark) is beneficial but not essential.Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models support ...

experience, including design and delivery of complex pipelines and data models in a cloud environment Strong proficiency in SQL; solid Python and/or PySpark capabilityDemonstrable experience with Databricks, Apache Spark, and Delta Lake architectureProven ability to design and own enterprise-grade canonical data models across multiple upstream source ...

collaborative environment.Key ResponsibilitiesDesign, develop and maintain modern data platform solutions.Build, optimise and manage data ingestion pipelines.Develop data transformation and modelling solutions using Python, PySpark and SQL.Support the implementation of data lakes, data warehouses and modern analytics architectures.Ensure high standards of data quality, security and performance.Work closely with stakeholders ...

Data Platforms: Hands-on experience with Databricks and Azure Data Lake . ETL: Proven ability to build efficient pipelines (ADF, SSIS, APIs). Desirable: PySpark, Dimensional Modelling, DAX/SSAS, or an interest in AI/ML applications. Interested in learning more? Send me a DM or comment below ...