1 to 25 of 40 Remote/Hybrid PySpark Jobs in the North of England

optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow Experience with containerisation ...

optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow Experience with containerisation ...

optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow Experience with containerisation ...

Databricks ecosystem. What you'll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

data platform from scratch, with responsibility for architecture and delivery in a business with limited existing data capability Strong SQL and Python or PySpark skills, with practical experience of ELT pipelines, dimensional modelling, source control and CI/CD Experience putting data governance and self-service BI into practice ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

API Gateway, and CloudWatch. Design, develop, and maintain data pipelines and ETL workflows using technologies such as Glue, Spark/PySpark, Kafka, or similar platforms. Implement and maintain Infrastructure as Code (IaC) using Terraform or equivalent tools. Drive observability, monitoring, logging, and operational readiness across software platforms and services. ...

API Gateway, and CloudWatch. Design, develop, and maintain data pipelines and ETL workflows using technologies such as Glue, Spark/PySpark, Kafka, or similar platforms. Implement and maintain Infrastructure as Code (IaC) using Terraform or equivalent tools. Drive observability, monitoring, logging, and operational readiness across software platforms and services. ...

Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls ...

responsibilities. Ensure solutions are secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines … duties as required. Skills and Experience Strong understanding of data engineering principles, data architecture, and cloud-based data platforms. Proficient in Python (PySpark) and SQL for building and optimising data transformations at scale. Strong hands-on Databricks experience. Experience with Azure data storage technologies (Azure SQL, Data Lakes, Blob ...

Implement automated testing, CI/CD patterns + deployment best practices for Foundry-based data assets. Technical Environment: Data Engineering, Palantir Foundry, Python, SQL, PySpark, Spark SQL + Cloud (AWS, Azure, GCP), CI/CD, Git based workflows. Benefits: £75,000 - £80,000 (DOE) + Remote/Hybrid Working ...

engineering best practicesYou have experience with Databricks, ideally also AzureYou have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and PandasYou're excited to join a start-up in a role that you can shape and influenceYou're collaborative with excellent communication skillsApply ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

doing Developing and maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions … stakeholders It would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group ...

doing Developing and maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions … stakeholders It would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group ...

Delta Lake, with governance, security, and real-time processing at scale. The role sits in the Enterprise Data Team and requires strong SQL and PySpark, plus experience with cloud-native data platforms. Hybrid working available, with 2 days in the office and 3 days from home. #J-18808-Ljbffr ...

functional teams to deliver high-quality technical solutions. Support the ongoing maintenance and optimisation of platform services. Essential Skills & Experience Hands-on experience with PySpark for data transformation and processing. Experience working with AWS data services , including: AWS Glue Amazon S3 Experience using Terraform for Infrastructure as Code. Experience ...

data ecosystem. You will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J ...

machine learning and analytical solutions from discovery through to deployment and business impact. Advanced proficiency in Python and relevant data science libraries, including NumPy, PySpark, Scikit-learn, TensorFlow, and/or PyTorch. Strong experience using Databricks, including Jobs, Asset Bundles, Delta Lake, and MLflow. Experience working with business stakeholders ...

machine learning and analytical solutions from discovery through to deployment and business impact. Advanced proficiency in Python and relevant data science libraries, including NumPy, PySpark, Scikit-learn, TensorFlow, and/or PyTorch. Strong experience using Databricks, including Jobs, Asset Bundles, Delta Lake, and MLflow. Experience working with business stakeholders ...

machine learning and analytical solutions from discovery through to deployment and business impact. Advanced proficiency in Python and relevant data science libraries, including NumPy, PySpark, Scikit-learn, TensorFlow, and/or PyTorch. Strong experience using Databricks, including Jobs, Asset Bundles, Delta Lake, and MLflow. Experience working with business stakeholders ...