251 to 275 of 341 Data Pipeline Jobs in the UK

Databricks Data Engineer London | Senior Manager | Up to £100K + Bonus Ready to take your data engineering career to the next level? Join a global consultancy on a major transformation project within the insurance domain. This is your chance to work with cutting-edge technologies, influence … make a real impact in a collaborative, forward-thinking environment. Why This Role? Be part of a high-profile project driving innovation in data and analytics. Work with a global leader in digital transformation. Enjoy senior-level responsibilities, clear progression, and exposure to decision-makers. Competitive package ...

definition through to high-level architecture and platform design. The role covers areas such as Predictive AI, Machine Learning and Generative AI, MLOps, data pipelines and modern AI platforms, across both cloud and private infrastructure. Alongside customer work, youll contribute to the development of AI offerings, standards … other architects within the wider team. Key Responsibilities Act as the AI subject matter expert in customer engagements, working directly with business stakeholders, data teams and IT operations. Lead discovery and design of AI strategies, governance models and platform architectures aligned to business outcomes. Design ...

Shape the Future with AI — Start Your Career Here Are you fascinated by artificial intelligence, machine learning, and data-driven innovation? Are you excited by the potential of AI to solve real-world problems and build smarter systems? We’re looking for an ambitious and curious Graduate … Analyst/Engineer to join our AI & Data Science team. This is an ideal role for a recent graduate who wants to kickstart their career by working on real AI projects, learning from experts, and gaining hands-on experience with modern AI tools, models, and data ...

make an impact? We're looking for a Senior Databricks Engineer to join a forward-thinking team and take ownership of cutting-edge data solutions. This is your chance to shape the future of data strategy in a business that has been empowering companies for decades … Engineers What We're Looking For Significant experience with Databricks (including Unity Catalog) Strong skills in Python, Spark, SQL Cloud expertise Knowledge of pipeline tools (Airflow, ADF) Leadership and problem-solving ability Ready to take the next step? Apply now and join us as our Senior Databricks Engineer ...

systems that think, reason and act helping enterprises evolve from AI-enabled to truly AI-native. Were solving hard problems at scale, integrating data, AI and automation to create self-improving systems that deliver measurable business value. Youll work with as part of a Data … hands-on engineering team thats turning Agentic AI from concept to production reality. If you're ready to embrace the power of data to transform our business and embark on an epic data adventure, then join us at Kyndryl. Together, let's redefine what's possible ...

Python Data Engineer Azure & PySpark - SC Cleared Contract £400-£458pd (Inside IR35) SC Clearance is Essential Summary Were looking for a Python Data Engineer skilled in PySpark, Delta Lake, Azure services, containerized development, and Behave-based testing. Youll design and build scalable data pipelines … maintain high-quality, test-driven code in a cloud environment. What youll do Build and maintain Python/PySpark pipelines for data ingestion, processing, and validation. Write unit and BDD tests using Behave, including mocking and patching. Create and optimize Delta Lake tables for reliable, performant data ...

Palantir has become mission-critical (reference examples: major aerospace, mining, insurance, healthcare systems, or government programmes) Deep Foundry technical expertise across the full stack: Pipeline Builder, Ontology, Workshop, OSDK, Code Repositories, Actions, and AIP/agentic capabilities - you can build production-grade applications not just prototypes Production deployment experience … including performance optimisation, data pipeline architecture, permissioning at scale, audit requirements, and operational maintenance ...

Data Scientist Location: Canary Wharf Employment Type: Contract, 2 months, strong chance of extension About the Role Morgan Hunt are working with a leading government organisation to recruit a GIS Data Scientist who can blend spatial analysis, advanced analytics, and problem-solving to turn geospatial data into actionable insights. You'll work with large, complex datasets, build predictive models, and support data-driven decisions across the organisation. If you love maps, patterns, and answering real-world questions with data, this role has your name all over it. Key Responsibilities Acquire ...

with global AI teams in Canada and France as well as local engineering stakeholders in Gloucester. This is not a research or deep data science role — instead, you will apply software engineering fundamentals , support the integration and deployment of AI-based tools, and help uplift engineering capability through … that integrate AI/ML functionality. Build robust, high-quality Python applications used within engineering workflows. Contribute to the design and implementation of data pipelines and data storage solutions. Support troubleshooting, optimisation, and ongoing maintenance of AI-enabled systems. Lightweight AI/ML Integration Prototype ...

responsible for: Delivering strategic advisory, engineering delivery, and enablement services to Government and commercial clients. Solve user problems by drawing source and captured data into a single foundational data model and building operational workflows that materially change the decisions made Work closely with clients, to tailor … products and services Design solutions for client problem sets based on their descriptions and by exploration of the data landscape Assist in the design, development, and maintenance of data pipelines and ETL processes to build data and action models to address the workflow needs ...

Data Engineer Location: 70 Redcliff Street, BS1 6AL Start Date: ASAP Contract Duration: 3+ months Working Hours: Mon – Fri, 09:00 – 17:00, 37 Hours per week Pay Rate: £ 200.00 per day Job Ref: (phone number removed) Job Responsibilities Analyze and document existing data pipelines, including … debug ETL pipelines. Use SQL for querying, validation, and optimization in the Lakehouse environment. Person Specifications Must Have Advanced skills in Python for ETL pipeline development. Strong experience with Databricks and related tools. Proven ability to design and optimize ETL pipelines. Strong SQL skills. Nice to Have Experience with ...

Senior Data Engineer (GenAI) - Leading Tech Company | London (Hybrid) Hybrid: 1 day in the office/4 days WFH Permanent role only Represented by Robert Half Limited Please note: No visa sponsorship on offer, No B2B,B2C, Fully remote workers or contractors on offer. The Opportunity … looking for a Senior Data Engineer with hands-on GenAI experience to join a forward-thinking technology business that's building data-driven solutions with real-world impact. This is a highly visible, senior-level role where you'll take ownership of data pipelines ...

Senior Java Developer – Energy Data Platform Location: Teesside (Remote/Hybrid) Type: Full-time Salary: £50,000 - £60,000 Hours : 37.5 per week (3 days in office/2 working from home) About the Role Imperial Recruitment Group are recruiting for a leading provider of energy market data and analytics in the UK and Europe. Their platforms process billions of data points from power markets every day, providing actionable insights for energy traders, analysts, and operators. They are looking for a Senior Java Developer to work on services that handle large, complex datasets ...

Data Engineering Consultant Salary: £65,000 - £75,000 Location: London (4 Days a Week in Office) Work on the front lines of innovation, embedding next-gen platforms directly into the operations of world-class organisations. Your engineering will become the competitive edge others can't replicate. The Opportunity … This role offers the chance to work at the frontier of technology and real-world operations, embedding advanced data and AI systems directly into major industrial organisations. You'll operate as a problem-solver and builder, transforming complex business challenges into scalable, high-impact solutions. The position sits ...

Data Engineer 6 months initially Remote working - but visits to Swindon office once per week Inside IR35 - Umbrella only Role Overview We are seeking an experienced Data Engineer to join our team. The ideal candidate will bring 5-7 years of experience in data … engineering, with a strong focus on Azure Databricks. Key Skills & Experience Proven expertise in Azure Databricks (must-have) Hands-on experience with DBT (Data Build Tool) Strong knowledge of Snowflake Solid background in designing, building, and optimizing data pipelines Ability to collaborate effectively in hybrid working ...

position. You'll: Lead full-stack development across all three platforms Define product architecture, tech stack, and roadmap from MVP to scale Build data systems for metadata logging, audit capabilities, and transparency records Integrate third-party AI tools and compliance validation frameworks Ensure security, data integrity … operational management AI & modern tech fluency – Understanding of AI tools, Agentic AI, governance frameworks, cloud infrastructure (AWS/Azure/GCP), APIs, and data pipelines Data systems experience – You've designed metadata frameworks, audit systems, or logging platforms Technical communication skills – You can explain complex concepts ...

Partner: 167 Solutions Ltd The Opportunity 167 Solutions is working with a forward-thinking organisation that is investing heavily in artificial intelligence and data-driven products. This role sits at the heart of that growth and will suit an AI/Machine Learning Engineer who enjoys taking models … from concept through to production. You will work closely with software engineers, data teams, and product stakeholders to design, build, deploy, and maintain machine learning solutions that deliver real commercial value. Key Responsibilities Design, develop, and deploy machine learning models into production environments Work with structured and unstructured ...

Data Science & Prototyping Developer Looking for a contract that actually lets you build cool stuff This team is creating the next wave of AI-powered marketing analytics tools and they are seeking an inventive Data Scientist/Prototyper to bring ideas to life. You'll take … What you'll be doing: Creating slick, high-impact dashboards using Adobe Analytics, Tableau and custom visualisations. Automating manual workflows and building robust data pipelines from scratch. Prototyping AI/ML models that genuinely improve how insights are generated. Taking ownership of legacy analytics processes and automating them. ...

Integrations Developer - Aberdeen, Hybrid Are you a problem-solver who enjoys working with data and modern .Net technologies? Were looking for an Integrations Developer in Aberdeento join a forward-thinking company and help shape the future of a global SaaS platform used across the oil, gas, and renewables … days a week into the office Pension scheme Career progression opportunities City centre parking What you will be doing: Developing and maintaining data integrations between internal and client systems. Building reliable ETL processes to manage and transform large data sets. Working closely with clients and internal ...

Integrations Developer - Aberdeen, Hybrid Are you a problem-solver who enjoys working with data and modern .Net technologies? We’re looking for an Integrations Developer in Aberdeen to join a forward-thinking company and help shape the future of a global SaaS platform used across … days a week into the office Pension scheme Career progression opportunities City centre parking What you will be doing: Developing and maintaining data integrations between internal and client systems. Building reliable ETL processes to manage and transform large data sets. Working closely with clients and internal ...

role: Own and manage the cloud infrastructure, ensuring scalability, resilience, and security. Implement and maintain high-availability architectures across critical services and data layers. Design and optimise … infrastructure to support AI-powered workloads and data pipelines. Lead cost optimisation initiatives across the platform. Manage the end-to-end deployment pipeline to ensure fast, reliable, and low-friction releases. Maintain, optimise, and automate CI/CD processes. Ensure zero-downtime deployment strategies and effective rollback ...

role: Own and manage the cloud infrastructure, ensuring scalability, resilience, and security. Implement and maintain high-availability architectures across critical services and data layers. Design and optimise … infrastructure to support AI-powered workloads and data pipelines. Lead cost optimisation initiatives across the platform. Manage the end-to-end deployment pipeline to ensure fast, reliable, and low-friction releases. Maintain, optimise, and automate CI/CD processes. Ensure zero-downtime deployment strategies and effective rollback ...

Long term contract Remote UK/EU We're looking for a BE Engineer to design and build high-quality backend systems and data pipelines that power the analytics platform. You'll collaborate with engineers and the Product team across the full software lifecycle, leading on scalable services … that ingest, transform and deliver data to our frontend applications and customers. You'll bring strong Python skills, experience with microservices and data ingestion/ETL pipelines , and an eye for clean, maintainable code. Familiarity with AI-assisted dev tools (GitHub Copilot, Cursor, Windsurf, Devin ...

learning, working with a talented team delivering important advances for a large government department. Essential Skills and Experience; Good experience and understanding of data governance, labelling, ontologies. Building data pipelines and runners. Able to construct strict quality control and data schema checks. Good knowledge … able to produce quality, detailed cloud architectures. Hands on with containerisation technologies, good Kubernetes experience is strongly desirable. Ideally worked providing DataOps/Data Engineering to enable advanced Machine Learning and Data Science Teams. Good communicator. Team player with leadership skills. Please apply immediately ...

office Key Skills/Knowledge/Experience Required: Proven experience as a DevOps Engineer, Cloud Engineer, or DataOps Engineer working in Azure-based data platforms, supporting complex, large-scale workloads within mature Agile delivery environments. Deep hands-on expertise with Azure Databricks, including clusters, job orchestration, Workflows, Repos … Databricks resources, Azure networking, storage, key vaults, identities, private endpoints, and associated cloud infrastructure. Strong understanding of Azure cloud services such as Azure Data Lake Storage, Azure Key Vault, Azure Monitor, Azure VMs, Azure Networking, Azure Log Analytics, and Azure AD. Proficient in scripting with Python, Bash ...