1 to 25 of 609 Data Pipeline Jobs in England

Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

Data Architect Inside IR35 up to £600+ Nottingham Hybrid Summary We're undertaking a fast paced data transformation into Databricks at E.ON Next using best practice data governance and architectural principles, and we are growing our data engineering capability within the Data … data storage concepts and logical data structures i.e. data warehousing. - Able to make recommendations for data pipeline design based on business requirements. - Be open to new ways of working, and open to working with new technologies. - Be able to set your ...

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising … warehousing concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts ...

observable data pipelines supporting critical Commodities datasets Modernize legacy workflows, reduce technical debt, and improve performance, reliability, and maintainability. Design automated pipeline controls for validation, monitoring, schema change, exception handling, and data integrity. Develop workflow orchestration, alerting, observability, and remediation processes. Translate business and client … Finance or other STEM related field or degree‐equivalent qualifications 4+ years of experience designing and building scalable data solutions, ETL pipelines, data workflows, and monitoring frameworks. Strong hands‐on experience with Python or similar programming/scripting languages. Experience with querying structured, semi‐structured ...

About the role... Are you an experienced Data Architect looking for your next challenge? Cooper Parry is on a rapid growth trajectory, and we’re seeking a Data Architect to lead the design and evolution of our enterprise data platform. This role will play … experience with Microsoft Fabric Deep understanding of: Modern data lakehouse architectures Dimensional and analytical data modelling Data pipeline orchestration Strong SQL skills and experience with distributed data processing Experience designing enterprise‐scale analytics platforms in Azure Ability to translate business requirements ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

stringent service level agreements (SLAs) and business requirements. What experience you need to be the successful Data Engineer: Proficiency in Palantir Foundry pipeline development: Extensive experience in building, maintaining, and optimising data pipelines and transformations within Palantir Foundry, leveraging tools such as Foundry Code Workbooks … integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability across various data flows. Experience in data modelling and ontology development ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc ) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc ) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc ) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/… concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/(url removed)/Public Sector domain experience strongly preferred. Must be active SC Cleared or fully ...

complete the application to adjustments@depop.com. For any other non-disability related questions, please reach out to our Talent Partners.RoleWe’re building a Data Quality, Observability & Governance Team to improve the reliability, trust, and compliance of Depop’s data ecosystem.As a Staff Data Engineer … this team, you’ll lead the design and implementation of frameworks, tools, and processes that strengthen our data foundations - ensuring our data is accurate, observable, and compliant.Your mission will be to reduce the mean time to detection and resolution of data incidents, by establishing ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...