1 to 25 of 840 Data Pipeline Jobs in the UK

Minimum 10+ years of experience as a Data Engineer or similar role. Proven expertise in the technologies below, and data pipeline development and strong understanding of data warehousing concepts and practices. Excellent problem-solving and analytical skills and strong communication and teamwork skills. … Databricks ML and Azure ML to develop predictive models and drive business insights. Proven expertise in Databricks, Apache Spark, and data pipeline development and strong understanding of data warehousing concepts and practices. Experience with Microsoft Azure cloud platform, including Azure Data Lake Storage ...

Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

Data Architect Inside IR35 up to £600+ Nottingham Hybrid Summary We're undertaking a fast paced data transformation into Databricks at E.ON Next using best practice data governance and architectural principles, and we are growing our data engineering capability within the Data … data storage concepts and logical data structures i.e. data warehousing. - Able to make recommendations for data pipeline design based on business requirements. - Be open to new ways of working, and open to working with new technologies. - Be able to set your ...

Title: Data Engineer Location: UK Job Type: Full-time About Scrumconnect: Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services … Science, Architecture, etc.) to identify emerging and cross-community issues affecting your role and escalate as necessary. Describe technical, data, pipeline, and production issues to colleagues of different specialisms. Communicate within the team and across teams to monitor expectations around delivery of data ...

Title: Data Engineer Location: UK Job Type: Full-time About Scrumconnect: Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services … Science, Architecture, etc.) to identify emerging and cross-community issues affecting your role and escalate as necessary. Describe technical, data, pipeline, and production issues to colleagues of different specialisms. Communicate within the team and across teams to monitor expectations around delivery of data ...

Title: Data Engineer Location: UK Job Type: Full-time About Scrumconnect: Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services … Science, Architecture, etc.) to identify emerging and cross-community issues affecting your role and escalate as necessary. Describe technical, data, pipeline, and production issues to colleagues of different specialisms. Communicate within the team and across teams to monitor expectations around delivery of data ...

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising … warehousing concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts ...

observable data pipelines supporting critical Commodities datasets Modernize legacy workflows, reduce technical debt, and improve performance, reliability, and maintainability. Design automated pipeline controls for validation, monitoring, schema change, exception handling, and data integrity. Develop workflow orchestration, alerting, observability, and remediation processes. Translate business and client … Finance or other STEM related field or degree‐equivalent qualifications 4+ years of experience designing and building scalable data solutions, ETL pipelines, data workflows, and monitoring frameworks. Strong hands‐on experience with Python or similar programming/scripting languages. Experience with querying structured, semi‐structured ...

Asset Alliance Group - Head Of Data & Analytics Asset Alliance Group is part of the Arbuthnot Latham Banking Group. We serve the commercial vehicle, bus, coach, and general asset finance sectors with tailored award-winning lending, contract hire, rental and finance packages. We are a full-service finance … incidents. Providing vision and direction to the Platform development team and stakeholders throughout projects and creating customer focussed outcomes. Planning and managing the pipeline of activity for the team (both engineering, analytics, governance and change disciplines) in order to optimise resource capacity and workloads, whilst securing timely engagement ...

member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Title: Data Engineer Job Location: Preston – Hybrid- 1 day a week onsite. We offer a range of hybrid and flexible working arrangements – please speak to your recruiter about … modelling practices Designing and implementing ETL pipelines using SSIS for data integration to ensure ETL processes performs optimally Troubleshooting; identifying and fixing pipeline bottlenecks and data issues Understanding of data governance, data quality, and best practices The Data Management ...

Description & Summary We are hiring Data & AI Consultants into our Technology Consulting practice. You will deliver client-facing engagements across analytics, BI, data engineering, data governance, data science, AI and workflow automation, in sectors including financial services, public sector, healthcare and life … teams within the Consulting, Advisory department, applying expertise in project management, stakeholder management, communication and process improvement to drive impactful results. As a Data & AI Consultant, you will work alongside clients across diverse sectors, using your technical, analytical and consulting skills to deliver practical data ...

platforms so they are reliable, secure, and easy to change. You will work across engineering and operations to automate delivery, improve data pipeline observability, and embed good governance so analytics and AI workloads can run at scale. You will be part of the Data Platforms … restricted environments. Delivering data applications in an automated and continuous fashion. Utilising tools like Airflow for orchestration and automating data pipeline deployments using DevOps tools and techniques (Docker, Containers, Terraform, Kubernetes) Building data pipelines using both cloud-native and open-source data ...

About the role... Are you an experienced Data Architect looking for your next challenge? Cooper Parry is on a rapid growth trajectory, and we’re seeking a Data Architect to lead the design and evolution of our enterprise data platform. This role will play … experience with Microsoft Fabric Deep understanding of: Modern data lakehouse architectures Dimensional and analytical data modelling Data pipeline orchestration Strong SQL skills and experience with distributed data processing Experience designing enterprise‐scale analytics platforms in Azure Ability to translate business requirements ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

stringent service level agreements (SLAs) and business requirements. What experience you need to be the successful Data Engineer: Proficiency in Palantir Foundry pipeline development: Extensive experience in building, maintaining, and optimising data pipelines and transformations within Palantir Foundry, leveraging tools such as Foundry Code Workbooks … integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability across various data flows. Experience in data modelling and ontology development ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc ) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc ) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc ) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...