1 to 25 of 26 Remote/Hybrid PySpark Jobs in the North of England

Databricks ecosystem. What you'll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls ...

Implement automated testing, CI/CD patterns + deployment best practices for Foundry-based data assets. Technical Environment: Data Engineering, Palantir Foundry, Python, SQL, PySpark, Spark SQL + Cloud (AWS, Azure, GCP), CI/CD, Git based workflows. Benefits: £75,000 - £80,000 (DOE) + Remote/Hybrid Working ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

Delta Lake, with governance, security, and real-time processing at scale. The role sits in the Enterprise Data Team and requires strong SQL and PySpark, plus experience with cloud-native data platforms. Hybrid working available, with 2 days in the office and 3 days from home. #J-18808-Ljbffr ...

doing Developing and maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions … stakeholders It would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group ...

doing Developing and maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions … stakeholders It would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

Data Engineer in the United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

Hybrid, 3 days onsite). This 12-month fixed-term contract role focuses on building and supporting modern cloud-based data platforms using AWS, PySpark, and IaC. You will work with Terraform, GitLab CI/CD, and collaborate with engineers to deliver secure, scalable cloud solutions. You’ll join ...

cases, implementing data-driven solutions, and developing predictive maintenance strategies. The ideal candidate holds an Aerospace Engineering Degree and possesses strong skills in Python, PySpark, and SQL. A hybrid working model is offered, and numerous career development opportunities await you. #J-18808-Ljbffr ...

/ETL workflows within a Fabric Lakehouse environment Work with OneLake, Lakehouse architecture, and modern data engineering best practices Utilise PySpark, SQL and Databricks for complex data transformation and engineering workloads Create robust data ingestion, orchestration and monitoring processes Support the development of enterprise-wide analytics solutions and Power … with Microsoft Fabric, including Data Factory, Dataflows, Notebooks and Lakehouse solutions Experience building and supporting production-grade ELT/ETL pipelines Strong skills in PySpark and SQL Experience working with OneLake, Delta formats or modern Lakehouse architectures Exposure to Databricks and distributed data processing environments Understanding of data modelling ...

modern data platform. Develop and maintain Medallion Architecture data layers, ensuring data is available for reporting and analytics. Create high-performance data transformations using PySpark, Databricks and Fabric Notebooks. Implement data governance and access controls through Databricks Unity Catalog. Support CI/CD deployment processes using Git integration … within a Data Engineering environment, delivering reliable and scalable data solutions. Strong hands-on experience with Microsoft Fabric and Databricks. Strong programming capability across PySpark and T-SQL. Experience working with Delta Lake, Lakehouse architectures and modern cloud data platforms. Understanding of data quality frameworks, testing approaches and governance ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

purpose is unique. We'd love you to join us. What skills, knowledge and experience will you need? Experience developing and managing PySpark workflows within AWS environments. Knowledge of AWS Lambda and event-driven solutions to support automated data processing. Experience provisioning and supporting cloud infrastructure using Amazon … platforms - here you will design, build, maintain, and optimise scalable data products used by our analysts and data scientists. You will develop and orchestrate PySpark-based ETL pipelines, using technologies such as Amazon EMR, AWS Lambda, and EC2 to process large volumes of structured and unstructured data efficiently. Your ...

purpose is unique. We'd love you to join us. What skills, knowledge and experience will you need? Experience developing and managing PySpark workflows within AWS environments. Knowledge of AWS Lambda and event-driven solutions to support automated data processing. Experience provisioning and supporting cloud infrastructure using Amazon … platforms - here you will design, build, maintain, and optimise scalable data products used by our analysts and data scientists. You will develop and orchestrate PySpark-based ETL pipelines, using technologies such as Amazon EMR, AWS Lambda, and EC2 to process large volumes of structured and unstructured data efficiently. Your ...

champion best-practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … problem solving capabilities with the ability to apply market standard pricing techniques to non-standard products. Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Technical expertise in predictive modelling techniques e.g. GLMs. Experience with WTW Radar ...