1 to 25 of 633 Data Pipeline Jobs in the UK

Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

pipelines, implementing data governance processes, and delivering reliable data solutions that enable critical business insights. Key Responsibilities Data Pipeline Development & Optimisation Design, develop, and maintain scalable and reliable data pipelines. Build ETL/ELT solutions to ingest, transform, and load data from multiple sources, including APIs, databases, and external data providers. Optimise pipeline performance, scalability, and cost efficiency. Implement data quality controls, validation processes, and monitoring solutions. Data Transformation & Processing Develop complex data transformation logic using Spark (PySpark or Scala ...

Data Architect Inside IR35 up to £600+ Nottingham Hybrid Summary We're undertaking a fast paced data transformation into Databricks at E.ON Next using best practice data governance and architectural principles, and we are growing our data engineering capability within the Data … data storage concepts and logical data structures i.e. data warehousing. - Able to make recommendations for data pipeline design based on business requirements. - Be open to new ways of working, and open to working with new technologies. - Be able to set your ...

Principal Data Architect Location: Remote Duration: 6 months £700 - £800 per day - Umbrella only Our client is looking for Principal Data Architect for Data Science family under DSF as part of Professional services sell. This request for NGET under the department of Customer Network … GDPR and Regulatory/CNI control environments In-depth experience in providing resolution to an extensive range of complicated data pipeline related problems, proactively and as issues surface. Demonstrable ability to troubleshoot problems across infrastructure, platform, and application domains. Controlling access to the data ...

Senior Data Pipeline Engineer Location: Hybrid – based from either our Brighton or London office, with travel to UK client sites as required Salary: £70,000–£80,000 base Role type: Full-time, permanent Department: Technology Consulting – Observability Practice Role Overview NETbuilder is building an Observability Practice … this is a senior delivery hire into it. As a Senior Data Pipeline Engineer , you will work hands-on with enterprise clients to architect, deploy and operate the observability platforms and telemetry data pipelines that power modern observability. You will work extensively with technologies such ...

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising … warehousing concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts ...

Asset Alliance Group - Head Of Data & Analytics Asset Alliance Group is part of the Arbuthnot Latham Banking Group. We serve the commercial vehicle, bus, coach, and general asset finance sectors with tailored award-winning lending, contract hire, rental and finance packages. We are a full-service finance … incidents. Providing vision and direction to the Platform development team and stakeholders throughout projects and creating customer focussed outcomes. Planning and managing the pipeline of activity for the team (both engineering, analytics, governance and change disciplines) in order to optimise resource capacity and workloads, whilst securing timely engagement ...

Description & Summary We are hiring Data & AI Consultants into our Technology Consulting practice. You will deliver client-facing engagements across analytics, BI, data engineering, data governance, data science, AI and workflow automation, in sectors including financial services, public sector, healthcare and life … teams within the Consulting, Advisory department, applying expertise in project management, stakeholder management, communication and process improvement to drive impactful results. As a Data & AI Consultant, you will work alongside clients across diverse sectors, using your technical, analytical and consulting skills to deliver practical data ...

Essential Skills Strong SQL skills, including complex joins, aggregations, subqueries and query optimisation Strong understanding of ETL/ELT concepts and data pipeline design Hands-on experience with Apache Kafka or similar streaming platforms Strong knowledge of AWS data services and managed databases Experience with … from internal and third-party APIs Strong data modelling and data warehousing knowledge Ability to troubleshoot data, pipeline, performance and reliability issues Active SC clearance OR BPSS Desirable Skills Experience working in DDaT-aligned digital delivery environments Experience with Swagger/OpenAPI ...

About the role... Are you an experienced Data Architect looking for your next challenge? Cooper Parry is on a rapid growth trajectory, and we’re seeking a Data Architect to lead the design and evolution of our enterprise data platform. This role will play … experience with Microsoft Fabric Deep understanding of: Modern data lakehouse architectures Dimensional and analytical data modelling Data pipeline orchestration Strong SQL skills and experience with distributed data processing Experience designing enterprise‐scale analytics platforms in Azure Ability to translate business requirements ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

LEAD DATA ENGINEER Salary up to £120,000 + Equity London-based role - hybrid working with 4 days per week in the office Join a fast-growing technology business building modern, cloud-based data platforms Lead the design and development of scalable data infrastructure … cloud technologies Apply online or contact Chelsea Hackett via ABOUT THE CLIENT Due to continued growth, we're recruiting for a Lead Data Engineer to join an ambitious technology business with exciting expansion plans. You'll play a key role in shaping the organisation's data ...

stringent service level agreements (SLAs) and business requirements. What experience you need to be the successful Data Engineer: Proficiency in Palantir Foundry pipeline development: Extensive experience in building, maintaining, and optimising data pipelines and transformations within Palantir Foundry, leveraging tools such as Foundry Code Workbooks … integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability across various data flows. Experience in data modelling and ontology development ...

faster, and act with confidence across complex domains. \n We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … and distributed processing technologies (e.g. Apache Spark, Hadoop).Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.Experience working with cloud‐native data platforms, with strong expertise in AWS and native data services.Familiarity with Infrastructure ...

faster, and act with confidence across complex domains. \n We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … and distributed processing technologies (e.g. Apache Spark, Hadoop).Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.Experience working with cloud‐native data platforms, with strong expertise in AWS and native data services.Familiarity with Infrastructure ...

identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/… concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/(url removed)/Public Sector domain experience strongly preferred. Must be active SC Cleared or fully ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

hiring a Data Engineer Apprentice to train and work alongside experienced Microsoft consultants on real client engagements. You will spend time designing, planning and building data integrations, data pipelines & data platforms. You will work alongside the Chief Data Offices, Data Architects and Data Integration Experts. Role Please note that this is an apprenticeship position and therefore anyone with more than six months professional experience working as a Data Engineer or who holds a degree or Master’s degree in a related subject (such ...

Data & Analytics Engineer A broad, strategic data role sitting between Data Architecture and Analytics Engineering Own the data landscape across a fast-moving manufacturing business Ballymena based, full-time permanent Salary £50,000 to £55,000 About the Company Our client … well-established manufacturing business based in Ballymena, currently undergoing a significant data transformation. With a strong appetite for innovation and a leadership team that genuinely values data and AI, this is an organisation where the right person can shape the future of an entire data ...

seeking an experienced Data Architect to join our Group Technology team on a 6-month contract. You will help realise the value of the immense Connells Group data landscape for the benefit of customers, colleagues and the overall business. You will use your expertise to define … evolve and communicate the existing Estate Agency enterprise data architecture, to democratise safe access to consolidated and unified data, modelled to help engineering teams develop single customer and property views. The data architecture will also support system migration and integration, to establish effective master ...

Director of Data and AI Department: Platform Employment Type: Permanent Location: London Reporting To: James Complin Description We’re seeking a strategic Director of Data and AI to lead, develop, and empower our Data and AI teams. This role balances leadership with technical excellence … define how data, analytics, machine learning, and generative AI enable the wider business. You’ll work closely with senior stakeholders, translating business objectives into a clear Data and AI strategy, and ensuring the team delivers scalable, trusted, secure, and commercially impactful solutions. The ultimate goal will ...