1 to 25 of 26 Remote/Hybrid PySpark Jobs in the North of England

Databricks ecosystem. What you'll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls ...

Implement automated testing, CI/CD patterns + deployment best practices for Foundry-based data assets. Technical Environment: Data Engineering, Palantir Foundry, Python, SQL, PySpark, Spark SQL + Cloud (AWS, Azure, GCP), CI/CD, Git based workflows. Benefits: £75,000 - £80,000 (DOE) + Remote/Hybrid Working ...

practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence You're collaborative with excellent ...

responsibilities. Ensure solutions are secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines … duties as required. Skills and Experience Strong understanding of data engineering principles, data architecture, and cloud-based data platforms. Proficient in Python (PySpark) and SQL for building and optimising data transformations at scale. Strong hands-on Databricks experience. Experience with Azure data storage technologies (Azure SQL, Data Lakes, Blob ...

functional teams to deliver high-quality technical solutions. Support the ongoing maintenance and optimisation of platform services. Essential Skills & Experience Hands-on experience with PySpark for data transformation and processing. Experience working with AWS data services , including: AWS Glue Amazon S3 Experience using Terraform for Infrastructure as Code. Experience ...

Delta Lake, with governance, security, and real-time processing at scale. The role sits in the Enterprise Data Team and requires strong SQL and PySpark, plus experience with cloud-native data platforms. Hybrid working available, with 2 days in the office and 3 days from home. #J-18808-Ljbffr ...

doing Developing and maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions … stakeholders It would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group ...

data ecosystem. You will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

Configure data ingestion processes and solutions. Lead the execution of Microsoft Fabric migration activities. Lead and guide migration engineers and QA resources. Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks. Create data integration test cases and support testing activities. Implement and amend data … Databricks o Azure Functions o Azure App Service o Logic Apps o Azure Kubernetes Service (AKS) o Web Apps Hands-on expertise in PySpark, SQL and Python. Good knowledge of real-time streaming applications, preferably including Kafka, Azure Functions or Azure Service Bus. Strong understanding of Spark processing ...

Data Engineer in the United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

Hybrid, 3 days onsite). This 12-month fixed-term contract role focuses on building and supporting modern cloud-based data platforms using AWS, PySpark, and IaC. You will work with Terraform, GitLab CI/CD, and collaborate with engineers to deliver secure, scalable cloud solutions. You’ll join ...

cases, implementing data-driven solutions, and developing predictive maintenance strategies. The ideal candidate holds an Aerospace Engineering Degree and possesses strong skills in Python, PySpark, and SQL. A hybrid working model is offered, and numerous career development opportunities await you. #J-18808-Ljbffr ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

purpose is unique. We'd love you to join us. What skills, knowledge and experience will you need? Experience developing and managing PySpark workflows within AWS environments. Knowledge of AWS Lambda and event-driven solutions to support automated data processing. Experience provisioning and supporting cloud infrastructure using Amazon … platforms - here you will design, build, maintain, and optimise scalable data products used by our analysts and data scientists. You will develop and orchestrate PySpark-based ETL pipelines, using technologies such as Amazon EMR, AWS Lambda, and EC2 to process large volumes of structured and unstructured data efficiently. Your ...

purpose is unique. We'd love you to join us. What skills, knowledge and experience will you need? Experience developing and managing PySpark workflows within AWS environments. Knowledge of AWS Lambda and event-driven solutions to support automated data processing. Experience provisioning and supporting cloud infrastructure using Amazon … platforms - here you will design, build, maintain, and optimise scalable data products used by our analysts and data scientists. You will develop and orchestrate PySpark-based ETL pipelines, using technologies such as Amazon EMR, AWS Lambda, and EC2 to process large volumes of structured and unstructured data efficiently. Your ...

champion best-practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … problem solving capabilities with the ability to apply market standard pricing techniques to non-standard products. Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Technical expertise in predictive modelling techniques e.g. GLMs. Experience with WTW Radar ...