51 to 75 of 90 PySpark Jobs in the UK

governance , schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs . Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns. Why Apply? Fully remote contract ...

complex operational challenges. You'll design and implement scalable generative AI workflows, often using technologies like Palantir AIP, while building robust data pipelines with PySpark, Python, and SQL. A key responsibility is executing sophisticated data integrations across distributed systems and enterprise environments, including ERPs and CRMs. You'll collaborate ...

strategy. About you - Extensive Technical Expertise : Strong knowledge of Microsoft Fabric components including OneLake , Lakehouse/Warehouse, Delta Lake, Direct Lake, Data Factory, Spark ( PySpark/Scala ) and Power BI (DAX and semantic modelling). Advanced Programming and Data Engineering Skills : Proficient in Python, SQL and T-SQL with … experience in PySpark ; familiarity with KQL for real-time analytics and robust ELT design using medallion architectures. Proven Experience in Modern Data Platforms : Track record of building and operating Azure-based data platforms, ideally in greenfield or transformational contexts, with integration across on-premise , cloud and third-party sources. ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Worki click apply for full job details ...

Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

TensorFlow Serving, TorchServe, ONNX, and TensorRT. Build and manage ML pipelines using MLflow, Kubeflow, and Azure ML Pipelines. Work with large-scale data using PySpark and integrate models into production environments. Monitor model performance and retrain as needed to ensure accuracy and efficiency. Collaborate with cross-functional teams ...

quality and structure of data ·Translate business needs into clear technical plans Experience required: ·Strong skills in data modelling ·SQL with either Python or PySpark ·Databricks or similar cloud data platform ·Experience communicating with both technical and non-technical users ·Leadership or mentoring experience ·Insurance or regulated sector experience ...

quality and structure of data Translate business needs into clear technical plans Experience required: Strong skills in data modelling SQL with either Python or PySpark Databricks or similar cloud data platform Experience communicating with both technical and non-technical users Leadership or mentoring experience Insurance or regulated sector experience ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

Sales, Customer Care and Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze ? Silver ? Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee … Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines, CI/CD Analytics: Power BI, Fabric What We're Looking For Experience Commercial and proven ...

Sales, Customer Care and Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze Silver Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee … Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines, CI/CD Analytics: Power BI, Fabric What We're Looking For Experience Commercial and proven ...

engineers and influence data architecture across high-consequence programmes Build and optimise ontologies, pipelines, and semantic models Apply strong data engineering skills (Python, SQL, PySpark) Bring cloud expertise (AWS, Azure, or GCP) and proven team leadership If you have Palantir Foundry experience and want to take ownership of delivery ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines click apply for full job details ...

development and optimisation of scalable data and pipelines in a cloud-based environment, building and managing ETL processes using tools such as Azure, Databricks, PySpark and SQL click apply for full job details ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven ...

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven ...

. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

data-driven decisions, while learning from experienced team members and developing your technical and business skills. Key Responsibilities: Build and maintain data pipelines, leveraging PySpark and/or Typescript within Foundry, to transform raw data into reliable, usable datasets. Familiarity with Palantir Foundry, PySpark, Kafka, TypeScript, PowerBI preferable. … years of experience building data pipelines at work or through internships is helpful. You can write clear and reliable Python/PySpark code. You are familiar with popular analytics tools (like pandas, numpy, matplotlib), big data frameworks (like Spark), and cloud services (like Palantir, AWS, Azure, or Google Cloud ...

deliver scalable, high-quality data solutions. Key Responsibilities: Develop, optimise, and maintain ETL/ELT pipelines within Databricks Build reliable data ingestion frameworks using PySpark and Spark SQL Design well-structured data models across medallion/lakehouse architecture Work with DevOps teams to automate deployments using CI/… Ensure compliance with best practices around data governance, security, and quality Requirements: Strong experience with Databricks (jobs, notebooks, workflows, Delta Live Tables) Proficiency in PySpark and Python Hands-on experience with Azure Data Lake, Azure Data Factory, or similar cloud services Understanding of Delta Lake, streaming pipelines, and lakehouse ...

that form the backbone of our Lakehouse platform, enabling accurate analytics, reporting, and regulatory compliance. You'll work with cutting-edge technologies including Databricks , PySpark , and Azure Data Factory , applying best practices in data engineering and governance to support this critical programme. Lead Databricks Engineer: Key Responsibilities Build … maintain Databricks pipelines (batch and incremental) using PySpark and SQL. Orchestrate end-to-end workflows with Azure Data Factory . Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming). Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets. Apply robust monitoring, logging ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong ...

experienced in building high-quality data pipelines and optimizing large-scale workloads on the Databricks Data Intelligence Platform. The ideal candidate is strong in PySpark, Delta Lake, Azure services, workflow automation, and performance tuning. Key Responsibilities Build and maintain scalable batch and streaming pipelines using Databricks notebooks, Jobs … ensure workloads are cost-efficient and reliable. Apply data quality, lineage, and access controls using Unity Catalog and governance best practices. Develop reusable PySpark ETL and data transformation code. Manage Delta Lake tables with ACID transactions, schema evolution, and time-travel capabilities. Integrate Databricks with Azure ADLS, Key Vault ...

Your new company This is a pivotal opportunity to join the Data and Innovation division of a large complex organisation leading the delivery of SAM (Supervisory Analytics and Metrics)-a transformative programme enhancing supervisory decision ...