101 to 125 of 560 Data Pipeline Jobs in the UK

Data Engineer to join our growing team. You will be responsible for expanding and optimizing our data, data pipeline architecture and optimizing data flow across functional teams. The Data Engineer will work with our software developers, database architects, data analysts and data scientists on data initiatives. Responsibilities for Data Engineer: Configuring Azure Event Hubs Writing Azure Function Apps in C# Creating data pipelines in Azure Data Factory Creating PowerBI reports and visualisations Analysing analytics business requirements to … determine the data models and data flows required to deliver new dashboards and reports Creating Solution Design documents for analytics solutions Documenting Architectural Decisions that are relevant for analytics solutions Designing Extract-Transform-Load processes Required Technical and Professional Expertise Development of complex PowerBI reports from more »

Salary: Up to £100,000 depending on experience Location: Hybrid working in either Swansea, Bristol, Manchester, London. Are you a skilled Data Engineer who can deliver Data Platforms? Are you familiar of working on agile delivery-led projects? Data Engineers at Made Tech help … build data platforms to enable predictive government services that make a positive impact on the people in the UK. So, if you are looking to be part of this mission, we would love to hear from you. Support in applying If you need this job description in another … interviews and any other needs. We also welcome any feedback on how we can improve the experience for future candidates. As a Lead Data Engineer or architect at Made Tech, you'll play a pivotal role in helping public sector organisations become truly data-lead, by more »

Salary: Up to £100,000 depending on experience Location: Hybrid working in either Swansea, Bristol, Manchester, London. Are you a skilled Data Engineer who can deliver Data Platforms? Are you familiar of working on agile delivery-led projects? Data Engineers at Made Tech help … build data platforms to enable predictive government services that make a positive impact on the people in the UK. So, if you are looking to be part of this mission, we would love to hear from you. Support in applying If you need this job description in another … interviews and any other needs. We also welcome any feedback on how we can improve the experience for future candidates. As a Lead Data Engineer or architect at Made Tech, you'll play a pivotal role in helping public sector organisations become truly data-lead, by more »

making and strategic planning across the organisation through the development, deployment, and maintenance of comprehensive BI solutions. This role will manage the diverse data sources and produce robust and comprehensive reports and dashboards across the organisations key operational areas: • Financial reporting • Risk and Compliance reporting • Operational reporting • Banking … Platform reporting • IT Service, resilience, and cyber reporting Responsibilities Data Management: Design, construct … install, test, and maintain highly scalable data management systems. Data Analytics: Develop analytics tools that utilise the data pipeline to provide actionable insights into key business performance metrics and operational efficiency. Interdepartmental Collaboration: Work closely with department heads and their teams to gather more »

Azure Data Engineer (SC Cleared) Location: Bristol Salary: £50,000 - £60,000 We're seeking Azure Data Engineer with active SC Clearance. This is a fantastic opportunity to join one of the most reputable digital solution provider's in the United Kingdom providing exposure to the … biggest and most ambitious programmes undertaken to keep our country safe. The SC Cleared Azure Data Engineer will be joining the Defence & Security arena, where you will help digitise defence with new thinking on technology capabilities and adoption processes, so we need someone who is sensitively able to … combine the latest thinking with traditional military functions. *Must have Active SC Clearance* Role requirements Leveraging Azure cloud technologies for tasks such as ETL pipeline development, data warehousing, data lake creation, and data movement. Utilizing Azure data and analytics services, including more »

Senior Data Engineer 6 month fixed term contract – 2 x a week in London £80,000 - £90,000 per annum + excellent benefits I am very excited to be partnering with a well-known fintech company! We're seeking a talented Senior Data Engineer to join … the data engineering team on a 6-month fixed-term contract, with the potential to extend but within a different department in the company. Role Overview: As a Senior Data Engineer, you'll play a pivotal role in maturing the data warehouse. You'll … collaborate closely with various stakeholders to gather business requirements and translate them into scalable data pipelines. Your expertise in Azure will be crucial in building robust infrastructure to support our data-driven initiatives. Key Responsibilities: with stakeholders to understand business requirements and translate them into technical more »

Senior Data Engineer - Rapidly growing highstreet brand - Hybrid - up to £100,000 Burns Sheehan is partnered with a rapidly growing high-street chain who have seen great success recently and are in the process of expanding further in the UK and abroad. This will be s a great … opportunity to have a real impact on the strategy and roadmap for data across the business. Working closely with the wider tech teams on a super greenfield project, this is the perfect opportunity to have a real impact on a scaling business. You will be working very closely … Integration: Gather data from various departments and tools, integrating it into a central data lake or warehouse. Data Pipeline Development: Design and construct data pipelines to automate data flow, involving ETL processes as needed. Modern tech stack - Python, AWS more »

Data Engineer/Scientist London (Hybrid) £50,000-£60,000 (+bens) 83DATA are partnered with a leading Internet provider seeking the skills of a Data Engineer/Scientist. The successful candidate can expect to be tasked to connect data sources to pre-existing Azure … data infrastructure. You will lead projects with a data centric approach, for this reason, a passion and aptitude for critical analysis of large and complex datasets is vital. Responsibilities: Build data engineering pipelines to ingest, transform, store, and serve high quality data for Data Scientists Writing and tuning complex data pipelines, workflows, and data structures. Identify opportunities for increasing performance, minimising costs and increase platform monitoring. Experience with components such as Azure Active Directory, Management Groups, Subscriptions, vNets, Route Tables, NSG’s and Azure more »

Senior Data Platform Engineer Up to £80,000 1 day a week in office (flexible) Xcede are delighted to be working with a well-renowned insurance company based in Hertfordshire. The company have an excellent Data Science and Machine Learning unit supported by a group of … Data Engineers who are responsible for Data Pipelines. As a bonus - Data Engineers are also able to help with ML model deployment and MLOps. Responsibilities Design and implement Data Pipelines for both structured and unstructured data. Support and improve ML infrastructure – particularly … Azure ML. Data Engineering: Guarantee data access, Data quality, ETL, Data pipelines Platform administration: Help set up accesses, infrastructure, create new storage, enforce best practices Production: Help the preparation of the data science pipelines, interact with data scientists more »

Job Title: Data Engineer Job Type: Full Time, Permanent Working location: London, Hybrid Role Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and … for you to solve. Our work is fast-paced although our team remains collaborative and flexible even when delivering fast. Your role as data engineer has a wide remit: working with product owners, subject matter experts and managers across the company, to design and deliver new components of … to the traditional data engineering type! Significant hands-on experience with AWS, especially AWS services focused on data flow, pipelines, data transformation, storage and streaming. Excellent data engineering skills, for example with SQL, Python, DBT and Airflow. Good understanding of service-oriented more »

Data Engineering Technical Lead Salary - £60,000 - £65,000 This is a great job for someone who has worked as a senior data engineer in data lake/lake house architecture platform and has hands on experience of developing data pipelines using … Pyspark and SQL. The successful individual will have working experience of AWS data eco system and toolsets, with passion to learn and explore new technology stack. A bit about the job: The Data Engineering Technical Lead will provide technical expertise and delivery capability across the development … hands on delivery. Working experience of Cloud Technologies and data eco system, AWS is preferable. Expert in development of data pipeline based on Pyspark and SQL with orchestration tools such as Airflow, Tivoli etc Knowledge of OLAP implementation on any Database technologies (Snowflake preferred) Excellent more »

Data Engineer Location: London – Hybrid Employment: Full Time All applicants must hold a DV (Developed Vetting) clearance to apply. The Opportunity: For the Ministry of Defence (MOD), maintaining National Security is a 24-hour-a-day, 365-day-a-year mission. For this reason, it partners with this … Big 4 Firm to support its multidisciplinary AI & Cloud Data Engineering Capabilities. The Role: As an AI & Cloud Data Engineering practitioner, you’ll build large-scale data pipelines and infrastructure, utilising data integration tools to optimise complex data solutions. Collaborating … with high-performing Data Science and ML Engineering teams to drive innovation and insight through AI and Data across the Defence & Security sector. Required Skills/Experience: British Passport Holder with active DV (Developed Vetting Clearance) End-to-end development with data pipelines, ETL more »

About The Company: We have partnered with a company that empowers underwriters to serve their insureds more effectively. They are using advanced data intelligence tools to rebuild the way that underwriters share and exchange risk. With a current focus on the small and medium-sized businesses that power … insurance pools, and their speciality portfolio is fully diversified with very low catastrophe, aggregation or systemic risk. The Role: Design and implement scalable data pipelines and models to ensure data quality, integrity, and accessibility. Lead the design, development, and optimization of data processing workflows … and SQL, with advanced problem-solving and data manipulation skills. Strong proficiency with DevOps practices and tools for automating data pipeline deployment, monitoring, and management. Hands on experience of designing and building CI/CD pipelines and automation using GitHub Actions, Jenkins, Azure Pipelines or more »

Azure Data Engineer | Cheltenham | £50,000-£70,000 Locke & McCloud are working with a leading Cyber Security Consultancy that is home to over 4000 specialists who support clients across manufacturing, defence and security, energy and utilities, financial services and many other industries. They have offices globally and are … recognised for the impressive high standard of work they conduct. They are looking for an Azure Data Engineer, you will play a pivotal role in this journey, leveraging your expertise to architect, design, and implement scalable data solutions on the Azure platform. You will collaborate closely … with stakeholders to understand business requirements, design data solutions, and deliver value-added insights. You will be responsible for architecting data pipelines, optimizing data storage and retrieval, and ensuring the integrity and security of our data ecosystem. Additionally, you will have the more »

Job Title: Data Analyst - Bristol - £50K We are working with a growing digital agency client based in Bristol who are looking for a talented Data Analyst to join their team. This is a hybrid role with at least 2 days a week in their local office … hub (Bristol or London). As the Data Analyst, you will be responsible for providing structure and process around data collection and analysis as the company scales. You will have a passion for data and enjoy using it to support decision making. Your key … reports that improve performance, formulating and documenting requirements for outputs, performing data analysis on datasets, troubleshooting issues in the data pipeline and establishing standard reports on key business areas. You will also need to maintain and enhance datasets, review data collection processes and more »

Software Engineer (Data) £110,000-£120,000 London, Hybrid (2x days) Permanent role I am seeking an experienced Software Engineer (Data) who can be passionate about designing, optimising, and maintaining AI-native data processing pipelines. Your role will involve tackling complex challenges with robust … data pipelines across diverse sources, enhancing the quality of data for our systems, and readiness to meet the challenges of scale. Responsibilities: Manage cloud orchestration and data infrastructure, applying your expertise with various data processing and indexing tools to ensure scalability and … reliability. Design robust data processing and indexing pipelines handling large amounts of data across several modalities, ensuring robustness and timeliness. Curate and manage large-scale data ingestion and indexing pipelines, ensure data quality and error handling. Ensure the secure handling of data more »

Senior Data Engineer Glasgow Based/Hybrid working 1 day a week in office. About the Opportunity: MBN are partnered with a growing Glasgow based company to find them a Senior data Engineer. They are entering a new phase of development aimed at refining their product … platform for the future as part of a long-term business expansion strategy. They are looking for a Data Engineer to join and lead the development and strategy of their platform and functionality. You new role : Act as the Subject Matter Expert for Data Engineering, helping … to set the standards for data engineering and analytics best-practice for the business. Lay the foundations of the new data platform as a hands-on practitioner, building on the rich data sets we have in the business for both automation and analytics. Design more »

Lead Data Engineer Permanent – Hybrid Salary up to £70,186 p/year, 35 hours p/week Join our dynamic public sector client based in Central London, where you will spearhead the technical design and implementation of transformative data integration projects. They are committed to … leveraging cutting-edge technologies and cloud components to enhance the data capabilities. As the Lead Data Engineer, you will be responsible for guiding the Engineering Team in developing and implementing data and integration solutions utilising Microsoft Azure and Integration Services, among other cloud technologies. … You will ensure the quality, performance, and security of their data integration solutions while adhering to best practices and project deadlines. Key Responsibilities the Engineering Team in developing data and integration solutions using the latest technologies and best practices. and deliver data products within more »

Lead Data Engineer Opportunity | 🌩️ Azure Cloud | 🛡️ Lloyds Insurer ✨🌟 A leading insurance business collaborates with DATAHEAD to find a talented Senior Data Engineer to join their Data Team 🚀. This is an incredible chance to work with technology and data-driven Lloyd's … offers agility and allows you to help shape its future 🌐! 🔧 You'll be joining the technology Engineering Team, focusing on designing and developing data solutions within an Azure cloud native environment 🌩️. The landscape is a greenfield setting, where a new Azure and Databricks-based platform will be … from scratch. Strong skills in Azure tooling (ADF), SQL, and Python are essential 🛠️. 🎯 Key Responsibilities: 🔹 Expertise in Azure platform architecture and developing data engineering pipelines 🌩️ 🔹 Proficiency in SQL and Python (Spark) 🐍 🔹 Provide technical assistance in data engineering, analysis, orchestrations, and enrichment 📊 🔹 Build relationships within the more »

be essential for achieving project success. Key Responsibilities: Software Development: Write high-quality, maintainable code using languages such as Python , SQL . Establish data tools like Snowflake and Azure Data Lake Services (ADLS) Gen 2 Utilize PowerBI, Tableau, or similar tools to design and create interactive … PoCs to validate new technologies, frameworks, or solutions. Drive the development of MVPs, ensuring timely delivery and alignment with business goals. Azure and Data Engineering: Leverage your Azure expertise to design and implement cloud-based solutions. Collaborate with data engineers to build robust data pipelines and ensure data quality. SQL and Data Interface: Develop and optimize SQL queries for data retrieval and manipulation. Work on data interfaces to connect various systems within the bank. Qualifications: Bachelor’s degree in Computer Science, Information Technology, or more »

A highly prestigious Investment Management firm are looking to hire a Senior Cloud Data Engineer with strong Python and Snowflake experience to join their new Data & AI team in a permanent position. The Cloud Data Engineer will join alongside Data Scientists and … Data Modellers to manage and improve the cloud-based Data Warehouse (Azure, Snowflake), and guide the organisation through an unprecedented period of data change. Your Key responsibilities will be: Engineering scalable and secure cloud data solutions for high-performance Data Science workloads and Generative AI applications Enhancing and managing the cloud-based Data Warehouse (Azure, Snowflake) Designing, building and deploying data pipelines to support advanced analytics and the development of Machine Learning algorithms and Generative AI (GenAi) applications Providing technology thought leadership and supporting more »

We are seeking a qualified Data Engineer to join our team, where your responsibilities will include: Designing, optimizing, and maintaining scalable data pipelines and ETL processes using Spark, ensuring streamlined data processing and integration. Collaborating cross-functionally to translate complex data requirements … into actionable technical solutions that drive business objectives. Leveraging Microsoft Azure and Databricks to architect robust data solutions, prioritizing performance optimization for enhanced operational efficiency. Conducting data modeling, profiling, and quality assessments to uphold the integrity and reliability of our data assets. Executing data governance, transformation, and integration initiatives to support advanced analytics and reporting requirements. Integrating machine learning models seamlessly into data pipelines to facilitate predictive and prescriptive analytics. Ensuring adherence to data security protocols and regulatory compliance standards, maintaining the highest levels of database integrity and more »

Amazon well architected frameworks, serverless solutions and improving existing platforms using AWS technologies with Agile delivery methods Demonstrable experience of Python 3, Python data management/… manipulation libraries and usage of Python with AWS components Demonstrable experience working in the Data domain, covering the full data pipeline from ingestion, mastering, management and consumption/distribution of large datasets Financial Services expertise preferred working with Equity and Fixed Income asset classes and … a working knowledge of Indices Experience with No-SQL and Relational databases and data analysis using SQL. Enter key behaviours and skills required to be successful in the role: Education & Work experience: 7+ years of AWS and Python development engineer experience within the Financial Services and Data more »

digital avatars and human clones. Role: MLOps Engineer Location: London Salary: Up to £100,000 Responsibilities: Develop ML Pipelines: Build and maintain scalable data processing systems. Ensure data quality and integrity. Automate ML Processes: Integrate machine learning solutions into existing systems and automate workflows. Deploy and … implementation practices. Ensure Security: Adhere to security and compliance standards in ML deployments. Requirements: Education: Bachelor’s or Master’s in Computer Science, Data Science, or related field. Experience: 2-3 years in ML, data engineering, or MLOps. Abilities: Strong analytical, problem-solving … and communication skills. Skills: Proficient in Python, ML frameworks like TensorFlow or PyTorch. Experienced with CI/CD, Docker, Kubernetes, and data pipeline tools like Airflow. more »

Azure Data Engineer An Azure Data Engineer is required by Whitehall Resources for our client based in the south of England. This is a permanent role and can be done from the comfort of the successful candidate’s home with travel into the office one to … two times a month. The successful candidate will come from a Data Engineering background and have experience with the latest Microsoft technologies. Strong knowledge of MS SQL Server on Azure with proficiency in SQL will also be needed. Do you have? Azure Data Engineering experience? Strong … knowledge of MS SQL Server on Azure? A background in the design, build, and maintenance of ADF data pipelines to extract, transform, and load (ETL) data from various sources into Data Warehouses? Exposure to Azure Synapse Analytics? Excellent communication skills? Would you like? Work more »