101 to 125 of 132 Data Pipeline Jobs in the UK excluding London

Senior Data Optimisation Engineer Location: Coventry (3 Days a Week) Salary: up to £80,000 + Benefits Investigo are working with a leading global technology business based in the Midlands in search of a Senior Data Optimisation Engineer to come and join their new team. They … in the Automotive sector. We are looking to assist them in growing out a new data focused team to ultimate service the pipeline of clients in need of their support and expertise. This responsibilities will include: Manage and execute data-focused projects using internal analytic … Provide expertise in AWS monitoring and optimisation, optimising databases and ETL pipelines. Utilise programming languages such as Python and PySpark to transform big data into manageable datasets. Contribute to the development of interactive dashboards and provide expert analysis across program lifecycles. Transform technical data into actionable more »

SENIOR DATA ENGINEER (Hertfordshire HQ – circa x1 day a week in office) Xcede are delighted to be working with a well-renowned insurance company based in Hertfordshire. The company have an excellent Data Science and Machine Learning unit supported by a group of Data Engineers who are responsible for Data Pipelines. As a bonus - Data Engineers are also able to help with ML model deployment and MLOps. Responsibilities Design and implement both Data Pipelines for both structured and unstructured data. Support and improve ML infrastructure – particularly … organisation. Requirements An academic background in Statistics, Mathematics, or Computer Science (or similar) A number of years of commercial work as a Senior Data Engineer (responsible for taking decisions, rather than just writing code) An ability to write production quality Python code. Strong SQL Skills Experienced creating data more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Birmingham. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP services such as BigQuery, Dataflow, and Cloud Storage. Architect data models and schemas in BigQuery to … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

Leading ecommerce client now requires a Data Engineer to join their innovative Machine Learning team. This role will focus on increasing overall data efficiency using Machine Leaning, streamlining data processes to increase overall profitability, and ensuring our clients operations and customer teams can increase … strong exposure to Machine learning principles/practices and technologies such as ML flow/Sage Maker. The role: Develop robust and scalable data pipelines. Collaborate with cross-functional teams to develop and deploy Machine Learning models and algorithms to solve business challenges and create opportunities. Contribute to … the development of our clients in-house Machine Learning operations framework. Support the data science and analytics team in deploying pipelines. Coach and mentor the team to improve development standards. Key requirements: Strong hands-on experience with Databricks, Spark, SQL or Scala. Proven experience designing and building data more »

love to see: Proficiency in developing ML models with Python or TensorFlow Familiarity with NLP and AI technologies, including LLMs Experience building robust data pipelines Experienced with DevOps practices, including CI/CD, TDD, SDLC Cloud experience – Azure or GCP This is an opportunity to take charge of more »

and we help our customers solve problems, elevate, and do existing things better. We are on a mission to help our customers connect data, software, and purpose to create extraordinary outcomes. You could say we are a digital transformation business. We specialize in software product development, analytics, data science, IoT solutions, machine learning, DevOps optimization, and modernization of applications, data, and platforms. We work with incredible clients in all types of industries such as smart home devices, space exploration, beer manufacturing, finance, ecology, and logistics. We work with some of the sharpest minds in … collaboration while inclusivity strengthens our community. As part of our team, you'll be tasked with: Here at Ascent, we believe that using data in the right way doesn't just help companies perform better; it transforms entire industries. You'll help world-leading brands transform the way more »

AWS Data Engineer | £50-60k | Fintech | Hybrid Leeds Want to work for a business that uses Data to create predictions that shape the future of fintech decision making? If so, AWS Data Engineers this one is for you... About the business 🚀 G.Digital has … how the everyday person gets closer to financial freedom! They've kicked off some exciting new projects including the development of a new Data Lakehouse and the implementation of an iPaaS solution and now they're looking to hire a Data Engineer to join their small … Python Any exposure to Terraform would be great Liaising with other areas across the business to help define requirements for new projects, including both pipeline and ETL requests What’s on offer to you? 💰 £50-60k Bonus of up to 10% Flexible working (core hours) Private health more »

a few. We are hiring for Senior Product Developer ( Full time/Fixed term contract ) Job Description:- Design, developing and testing large scale data processing systems through data pipelines and ETL's. Working closely with architects, analysts, and stakeholder to bring efficient and scalable solutions in … data processing. Skills : Specific skills we need – strong SQL, performance tuning, python for ETL frameworks and robust data warehousing experience and GCP. Strong experience in python strong knowledge of sql/plsql and ability to write performant sql queries Experience working on data pipelines … data warehouse and ETL's. GCP experience (Migration to cloud is a plus)(pub/sub, dataflow, dataproc, big query, cloud sql) knowledge in containers and container orchestration CI/CD experience version control (GIT) Orchestration tools ( airflow or cloud composer more »

team. This team is responsible for and enhancing the business’s lead management and CRM capabilities BI analysts with accurate and relevant application data business insights and predictive models developed by the BI team into self-optimising, productionised systems within the wider business. The ideal candidate will be … responsible for developing and maintaining data pipelines and queries to extract complex datasets needed for our business intelligence and reporting needs. Additionally, the candidate will be involved in writing production-level code to support the organisation’s various CRM and lead-management applications and systems. Key Responsibilities: 1. … Data Extraction and Transformation: - Design and develop data pipelines and queries, for efficient data extraction and transformation. - Collaborate with cross-functional teams to understand data requirements and ensure accurate and timely delivery of BI solutions. 2. Business Intelligence: - Work closely with business more »

Python or TensorFlow ) Familiarity with natural language processing or AI technologies ( e.g LLMs such as ChatGPT or BARD , Embeddings , Prompt Engineering ) Experience with data pipelines and preferably processing of document s ( e.g using Google Cloud Fusion/Azure Data Factory ) Strong background in either Python/… Service Orientated Architecture, API-Centric Design, Twelve-Factor App, FAIR, etc.). Experience supporting digital platforms, including Integrations, Release Management, Regression Testing, Integrations, Data Obfuscation, etc. Knowledge of Azure Data Factory/GCP Cloud Data Fusion , Microsoft Azure Machine Learning or GCP Cloud ML … Engine , Azure Data Lake , Azure Databricks or GCP Cloud Dataproc . Familiarity with big data technologies and distributed computing frameworks, such as Hadoop, Spark, or Apache Flink. Experience scaling an “API-Ecosystem ”, designing, and implementing “API-First” integration patterns. Experience working with authentication and authorisation protocols more »

Lead Data Engineer Want to help drive the Data function with one of the most reputable employers in Belfast? This company is known for it’s innovation, delivery, and culture. Their Belfast hub is at the centre of the tech community, promoting learning and diversity at … all levels. They have a new opportunity for a Lead Data Engineer – this function is still relatively young in this region, and they’re looking for a thought leader in the industry to come in to this blank page and help write a new story. The person Experienced … in designing and deploying production data pipelines within a big data architecture Strong technical skills in Python, SQL and AWS (though they’re very open to any cloud experience) Experience processing large volumes of structured and unstructured data Experience of leading teams, mentoring junior more »

acquisition of a new BI Developer. As a BI Developer, you will collaborate closely with cross-functional teams to understand business requirements, design data models, and develop robust BI reports and dashboards. This role will be based from the Leeds office on a hybrid working model to be … BI solutions using tools such as Power BI, Tableau, or similar. Collaborate with stakeholders to gather and analyse business requirements. Develop and maintain data models, ETL processes, and data pipelines. Create interactive dashboards and reports to provide insights into key business metrics. Optimise query performance and … ensure data accuracy and integrity. Stay updated with emerging BI trends and technologies to drive continuous improvement. Utilise real time analytics to have a meaningful impact on business performance. Requirements: Relevant Degree in Computer Science or similar field Advanced Power BI SQL & SSRS DAX Experience with cloud platforms more »

retail retirement and insurance businesses, we’re a fast-growing division with big, customer-centred ambitions. Job Description Legal and General Retail’s Data Operations team are currently hiring three Lead Data Engineers following the merger of internal divisions resulting in them expanding into a new … area. These positions are to focus on the retirements side of the Retail division and will build out new data pipelines utilising tools such as Synapse, DBT, Azure Devops and Snowflake. This role will see you responsible for designing, building, and implementing a variety of data … Facilitating the refinement of the tasks in the Product Backlog and guide and support your team as you break deliveries down into technical data engineering components. Converting user stories in to technical, quality, testing and documentation tasks in the chosen work-flow management tool. Liaising with the end more »

and also offer you a chance to work in a highly motivated and smart team of engineers? You'll work as a Senior Data Engineer building a data and analytics platform that's used by the hedge fund's trading desks. You'll join at a … time where the fund are diversifying into new markets and therefore you'll play a crucial part in all aspects of data operations in this new area for them. You'll work extremely closely with Portfolio Managers/Quants - You'll work with them to understand exactly what … they need from the data and work fully end to end to create solutions that present data in a way that can be consumed effectively by them. Qualifications: Strong experience with Python for data engineering. Experience building custom Data Pipelines Strong SQL more »

role demands both high technical aptitude and problem-solving abilities, motivation, and exceptional attention to detail. At Roku, we are a company of data-focused builders, particularly on the Search and Recommendations team. Every day, you'll have the opportunity to assess existing systems and find ways to … applying Machine Learning to concrete problems at large-scale. Strong CS fundamentals, with the ability to write algorithms with ease. Good understanding of data science and machine learning fundamentals like regression, classification, tree-based approach, neural networks, and sequence-based models. Experience with advanced machine learning techniques, including … and artwork. Proven familiarity with NLP approaches like Word2Vec or BERT, including identifying the right KPIs and objective functions. Experience working with big data systems (Spark, EMR, Kafka, S3, Airflow) and programming languages (Java, Scala, or Python). Experience building in-production Machine Learning systems Good understanding of more »

Principal Data Engineer - Bristol This Digital Services Partner revolutionises how people engage with technology. They pride themselves on delivering innovative solutions that enhance user experiences and drive measurable results. People are genuinely at the heart of this business, and they're looking for a Principal Data Engineer to join. This role will provide technical expertise in data engineering practices both internally and to clients, and provides great opportunity to develop and upskill. The role: Act as a technical SME to help produce user centred software Design and manage software deployment and development … teams to prioritise workload and realign where needed Be a strong communicator, present prototypes and solutions to stakeholders The person: Strong experience in Data Engineering, with python being your primary language Strong cloud exposure - AWS, GCP or Azure Excellent SQL and relational database skills - NoSQL desirable Experienced in more »

also offer you a chance to work in a highly motivated and smart team of engineers? You'll work as a Mid-Level Data Engineer building a data and analytics platform that's used by the hedge fund's trading desks. You'll join at a … time where the fund are diversifying into new markets and therefore you'll play a crucial part in all aspects of data operations in this new area for them. You'll work extremely closely with Portfolio Managers/Quants - You'll work with them to understand exactly what … they need from the data and work fully end to end to create solutions that present data in a way that can be consumed effectively by them. Qualifications: Strong experience with Python for data engineering. Experience building custom Data Pipelines Good knowledge more »

Data Engineer- Leicester I'm currently looking for a Data Engineer to join a company that is based in Leicester. They need someone to start in the next few weeks. Location: Leicester (on site 2-3 times a week) Length: 12 months Day Rate: £400 per … day. IR35 Status: Inside of IR35 Required experience will include: Expertise in designing and implementing data pipelines using Azure services such as Azure Data Factory , Azure Databricks and DBT . Hands-on experience with SQL database design Experience of product lifecycle management principles and tools (e.g. … DevOps , terraform ) and relational database manipulation and interrogation. If you are interested in this Data Engineer role please apply with your most recent CV. Alternatively email me on George . batson @ Randstad . co . uk Data Engineer- Leicester Randstad Technologies is acting as an Employment more »

our medium blog which received more than 300 claps here) API's & Gateway - Focuses on how evolve to Microservice architecture using .NET Core Data Engineering - Focuses on how to build the best backend data pipelines to manage/move our data using Python, SQL … our event driven architecture over RabbitMQ. Data Science - Focuses on how we can improve our AI assistant (Aimie) and maximise the huge amount of data that we sit on SRE - Focuses on how we best deploy/manage all the infrastructure every single day using tools more »

AI solutions that enhance, organise, and enrich their data, cost-effectively, accurately, and scalably Plan and manage their ML/AI delivery pipeline Mentor and coach team members on ML engineering best practices Advise senior management on their ML/AI strategy Key skills: Python for Machine … Learning and Data Science Frameworks used in Machine Learning and Data Science such as TensorFlow, PyTorch, and scikit-learn Data Science and Visualisation libraries including Pandas, NumPy, scikit-learn, matplotlib, Seaborn Cloud services used in machine learning and data science, such as … Azure, OpenAI, Hugging Face, AWS ML/AI Machine Learning Applications development life cycle processes and tools - MLOps, including building ETL data pipelines. Cutting-edge approaches to Machine Learning, including transformer models, diffusion models, and adversarial machine learning Experience in fine-tuning off-the-shelf Large Language Models more »

. Familiarity with natural language processing or AI technologies ( e.g LLMs such as ChatGPT or BARD, Embeddings, Prompt Engineering ) Experience with machine learning, data pipelines and preferably processing of document s (e.g using Google Cloud Fusion/Azure Data Factory ) Experience in new database technologies such … as vector databases advantageous Strong background in either Python/Typescript Experience supporting digital platforms, including Integrations, Release Management, Regression Testing, Integrations, Data Obfuscation, etc. Experience scaling an “API-Ecosystem”, designing, and implementing “API-First” integration patterns. Experience working with authentication and authorisation protocols/patterns. Other Information more »

. Familiarity with natural language processing or AI technologies (e.g LLMs such as ChatGPT or BARD, Embeddings, Prompt Engineering) Experience with machine learning, data pipelines and preferably processing of documents (e.g using Google Cloud Fusion/Azure Data Factory) Experience in new database technologies such as … automation and generative AI applications.(e.g SpaCy, NLTK, or Hugging Face). Experience supporting digital platforms, including Integrations, Release Management, Regression Testing, Integrations, Data Obfuscation, etc. Experience scaling an “API-Ecosystem”, designing, and implementing “API-First” integration patterns. Experience working with authentication and authorisation protocols/patterns. Other more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Skills/Experience: Previous experience of Enterprise ICS/network architectures … and technologies. * Working with frameworks and technologies that support data-intensive distributed applications. * Experience maintaining and administrating data analytical and SIEM platforms. * Experience using host and network-based IDS/IPS * Experience using packet capture solutions. * Skill in developing and deploying signatures. * Skill to apply cybersecurity … network tools in response to new or observed threats within the network environment or enclave. * Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. * Implement data management standards, requirements, and specifications. * Develop data more »

be able to demonstrate strong capabilities in Azure DevOps methodologies with the ability to implement CI/CD pipelines in collaboration with the data engineering team. Core responsibilities: Lead the implementation and maintenance of CI/CD pipelines for data projects, ensuring efficiency and scalability. Collaborate … for process improvement and implement solutions to streamline development workflows. Monitor system performance and implement optimizations to ensure the reliability and availability of data pipelines and services. Collaborate with cross-functional teams to troubleshoot issues, implement fixes, and ensure timely resolution of technical challenges. Stay up-to-date … with industry trends, emerging technologies, and best practices in DevOps and data engineering. Document processes, configurations, and best practices to ensure knowledge sharing and maintainability of data infrastructure and pipelines. Key technical skill-set: Proven leadership capabilities in regards to project implementation coupled with the ability more »

components of the AI infrastructure. Design and deploy scalable and resilient infrastructure solutions to support AI and machine learning initiatives. Manage and optimise data pipelines, storage systems, messaging frameworks and computing resources for efficient data processing and analysis. Integrate machine learning models into production systems, ensuring … and strong proficiency in Python development Deep knowledge of distributed and scalable systems, including proficiency with PostgreSQL, RabbitMQ, and Cassandra. Familiarity with big data technologies such as Hadoop, Spark, or Kafka . Strong problem-solving skills and the ability to troubleshoot complex issues in distributed systems. Excellent communication more »