51 to 75 of 351 Permanent Data Pipeline Jobs in the UK

knowledge on which to develop their careers and progress in the direction they choose. The Role: We are looking for an experienced Senior Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with a … fantastic team of technology experts. You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability. … Self motivated, enthusiastic and committed The ideal candidates will share our values and be aligned with our ways of working Key Responsibilities: Develop Data Engineering solutions for our clients/projects Design and build data models, schemas to support business requirements Develop and maintain data more »

and Oaktree Capital. Find out more about us by visiting our website at: http://www.hartreepartners.com/JOB OVERVIEW: As a Senior Data Scientist at Hartree, you will play a pivotal role in shaping our data strategy and driving the development of advanced machine learning … time series forecasting solutions, specifically focused on the European Power Markets. You will collaborate with cross-functional teams to design, implement, and optimize data pipelines and predictive models that inform trading decisions and enhance operational efficiency. JOB RESPONSIBILITIES: Market Data Analysis: Conduct comprehensive analysis of data … Python and its data science libraries (e.g., Pandas, NumPy, Scikit-Learn, TensorFlow, PyTorch). Solid understanding of ML and data pipeline architectures and best practices. Experience with big data technologies and distributed computing (e.g., Spark, Hadoop) is a plus. Proficient in SQL and more »

Senior Data Engineer (Azure/Data Lake/Databricks/Data Factory/SQL Data Warehouse/SQL Database) A leading energy company in the low-carbon sector is looking for a Senior Data Engineer . Base Location: Flexible, with … are also possible. Working Pattern: Permanent | Full Time | Hybrid (1 day at the office per week) What is the Role? As a Senior Data Engineer , you'll develop and optimize data pipelines on the Microsoft data stack, particularly the new Fabric data platform. You'll collaborate with data architects, developers, and stakeholders to drive data-driven decision-making. Reporting to the Lead Data Engineer, you'll be part of a team delivering innovative data solutions across the business. Key Responsibilities: Design and more »

Data Engineer £50,000 - £60,000 - Hybrid My client, industry leading EdTech organisation are currently looking to expand their Data team! become a Data Engineer with vast progression opportunities, where your work is all for a great cause! Requirements Thrive in … a fast paced and dynamic environment Exceptional SQL skills including an appreciation for performance tuning on a variety of database engines Data pipeline development experience in Python (especially using pandas, parsing API responses, posting data to API’s etc.) Excellent analytical and problem-solving skills … An appreciation for the big picture while delivering short/mid-term solutions Experience of building data warehouses, enterprise data warehouses or data lakes with ETL pipelines Experience with AWS tools including Glue, Lambda, S3, RDS, is preferred An understanding of CICD concepts including more »

our team with people just like you! We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

Test Engineer Contract Type: Inside IR35 Rate: £500 per day Duration: 6 months This is a highly back-end/data-focused testing position, so experience in this area is essential. Location: Remote with travel to Leeds or London as required Brio is working with a leading provider … Requirements:** Proven experience as an Automation Test Analyst, preferably within the healthcare or NHS sector. Strong scripting skills in SQL and Python ETL, Data Migrations, Data Pipelines Experience with test automation frameworks. AWS cloud Excellent communication and collaboration skills. Ability to work effectively in a fast more »

learning models through research, such as reviewing academic papers and adapting them for our use. Drive the development of the cutting-edge 3D AI pipeline, leveraging the latest breakthroughs in 3D data processing, 3D vision, and machine learning techniques. This entails system design and comprehensive testing. Identify … and address bottlenecks in data, pipeline, and operations, proactively devising and implementing agile solutions to enhance scalability and optimize performance. Take ownership of project management for AI/Vision initiatives, overseeing end-to-end execution and effectively coordinating data operations. Mentor and empower junior team more »

Fantastic new opportunity for a Data Engineer to join a high performing team at a leading hedge fund! You'll be vital in building and maintaining infrastructure for the efficient execution of their quantitative and discretionary trading strategies. What we'd love to see: MSc in Comp Sci … Engineering etc. from a Top University Demonstrable Data Engineering experience within a financial services company or hedge fund/commodity trading Strong Python and SQL Building and maintaining data pipelines AWS preferred but not essential What's in it for you? £80k - £100k starting salary dependent … on experience Opportunity to work with a world class team of Data professionals and portfolio managers 4 days on site more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … databases, as well as NoSQL databases Familiarity with cloud services (preferably GCP ) and understanding of its ETL frameworks. Desired : experience with data pipeline and workflow management (Airflow, Composer). Desired : Familiarity with machine learning algorithms and data science principles is a plus. Qualifications Degree in more »

Our client are a renowned name in the Property Sector, they are on the lookout for 2 AWS Data Engineers to come in on a contract basis. Key Skills/Requirements: Must have Python & Spark experience Must have strong AWS experience Must have Terraform experience SQL & NoSQL experience … Have built out Data Warehouses & built Data Pipelines Strong Databricks & Snowflake experience Docker, ECS, Kubernetes & Orchestration tools like Airflow or Step Functions are nice to have Contracts are running for 6 months initially, paying up to £450p/day (Outside IR35) and will be mostly remote more »

Data Engineer – Azure A fast growing and innovative company are looking for a Data Engineer to join their team. They are a people focused company, who since 2021 have won 3 best place to work awards, representing a fantastic working environment. Based in either Derby or … the Chief Operating Officer, the candidate will play a crucial role in driving the company's digital initiatives forward. Key Responsibilities As a Data Engineer, you will play a pivotal role in supporting the delivery and evolution of the company's digital portfolio. Collaborating with a talented team … the candidate will oversee the management and maintenance of existing data pipelines, monitor workloads, conduct root cause analysis, ensure data quality, and escalate technical issues as necessary. What you’ll be getting up to: Monitoring and recovery of Azure Data Factory and Databricks jobs more »

Data Scientist/Quant Researcher Top-tier Hedge Fund London/Hong Kong Our client, an industry-leading hedge fund, is looking to add Data Scientists to its Equities division. We will be hiring in both London and Hong Kong. You will be an early/… mid-career Data Scientist (1-7 yrs' experience) currently working in a: Hedge Fund Asset Manager Market Data Provider Alternative Data Provider or adjacent industry with exposure to Capital Markets data You should ideally have an interest in a particular Equities sector … or group of sectors (e.g. Consumer data, Healthcare data, etc) and some proven expertise in applying this to Equities markets problems. Working with the firm's world class technologists and investment teams, you will be an individual contributor focused on the analysis of 3rd party data more »

trusted partner, a leading financial services provider is currently hiring an Lead Azure DevOps Engineer who has recent experience working alongside a large data infrastructure department implementing CI/CD pipelines within an Azure environment our client transitions their legacy IT estate into the cloud. This is an … be able to demonstrate strong capabilities in Azure DevOps methodologies with the ability to implement CI/CD pipelines in collaboration with the data engineering team. Core responsibilities: Lead the implementation and maintenance of CI/CD pipelines for data projects, ensuring efficiency and scalability. Collaborate … for process improvement and implement solutions to streamline development workflows. Monitor system performance and implement optimizations to ensure the reliability and availability of data pipelines and services. Collaborate with cross-functional teams to troubleshoot issues, implement fixes, and ensure timely resolution of technical challenges. Stay up-to-date more »

ll play a key role building a Gen AI system that facilitates seamless communication between users and databases, enhancing efficiency and accuracy in data retrieval and analysis. Responsibilities include: Lead the development of systems with a specific emphasis on AI/ML applications, CI/CD pipelines, APIs … and communication layers across hybrid architectures spanning multiple public clouds and private data centers. Engineer global-scale backends using microservices or distributed systems, primarily in Python (experience in other languages is advantageous). Collaborate on PDF parsing and/or optical character recognition (OCR) projects. Construct data more »

Data Architect/Cloud Data Architect (GCP) Google Cloud/GCP, Machine Learning/Generative AI Remote-based or Hybrid should you prefer (London) £100k-£140k basic salary + bonus Benefits: 25 days holiday + bank holidays Private Healthcare Discounted Gym Membership (Nuffield + Hussle) Life … Insurance, Income Protection, Employee Assistance Program Pension Scheme (max 10%) We are working with an AI/Data Consultancy with core clients within the Retail, Financial Services, Gaming, SaaS sectors. The successful Cloud Architect will work with an array of technologies with clients from several industries & sectors meaning … devising innovative solutions to address their needs, defining project parameters, and supervising the execution of these projects. Role of the Cloud Architect/Data Architect: Utilising advanced data processing and cloud technologies effectively to stimulate innovation. Engaging with clients to create top-notch solutions tailored to more »

Data Engineer Location: London, UK Hybrid: 3x a week in office Salary: £63,000 About the Client Our client is a top-tier professional services firm known for its exceptional workplace culture and some of the most sought-after office spaces in London. Role Overview Our client is … in search of a skilled Data Engineer to manage and engineer the data warehouse. The opportunity will be amazing for a Data Engineer who is happy to work in a small team. Incredible opportunity to mentor/lead a small Data Engineering … team within a year! Main Responsibilities Build scalable data pipelines on a daily basis Code on a daily basis with Azure SQL and python Develop and maintain the logical and technical architecture to fulfil the client’s information, insights, and reporting needs. Stay informed on the latest developments more »

InstaDeep is looking for an Applied Research Scientist to join our expanding team. Applied Research Scientists at InstaDeep develop algorithms and data pipelines focused mainly, but not exclusively, on physics-based methods. As part of the BioAI team, you will be contributing to the development of new approaches … Keywords: Computational molecular Physics, Structural Biology, Computational Chemistry, Protein Interactions, Protein Design, Molecular Dynamics, Monte Carlo, Enhanced Sampling, Machine Learning, Deep Learning, Statistics, Data Science Responsibilities Design, implement and deliver complex pieces of research in the area of computational molecular physics/biology. Implement and test pioneering enhanced more »

Principal Data Engineer £80,000 - £103,000 + 15% Bonus London based - Hybrid working A consultancy at the heart of ground-breaking innovation and transformative solutions where expertise meets ambition. With a focus on collaborative brilliance shaping industries and propelling client towards their full potential whilst driving meaningful … change is seeking to add a Principal Data Engineer to the team based in London. Requirements of a Principal Data Engineer; Adeptness in utilising AWS and Azure data and analytics services (or their open-source equivalents), including but not limited to EMR, Glue, RedShift … Kinesis, Lambda, Data Factory, and Databricks. Strong experience in data engineering, data science, and product development, encompassing experience in both stream and batch processing. Proficiency in the design and deployment of production data pipelines, involving languages like Java, Python, Scala, Spark, and more »

Senior Data Engineer | Full Time | On-Site | Python | SQL | DataBricks| Azure| As a Senior Data Engineer, you will play a key technical role in design, implementation, and continuous refinement of data pipelines and architectures within Microsoft’s Azure ecosystem. This role involves leveraging Azure … s diverse data services, such as Data Factory, Databricks, Azure SQL and Synapse Analytics, among others, to build scalable and secure data solutions that support complex onward processing by both operational and analytical teams. What will this role look like? - Develop and maintain scalable … operational and analytical teams to solve complex business problems. - Monitor system performance, identify bottlenecks, and implement changes to improve data processing and pipeline/systems performance. - Work closely with Data Scientists within the Operational Analytics team to understand data needs and deliver solutions more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

experience we're looking for, please go ahead and apply. We'd love to hear from you! About the role As the Senior Data Management Analyst, you will be responsible for continuous evolution of the data quality life cycle practices. This includes defining and proposing data … and their current levels under each dimension. You will take a leading role in supporting the organisation in improving the quality of its data by acquiring data to be brought under governance and perform data profiling analysis to understand it's origination, definition, lineage … into wider data management subjects e.g. Data Quality Insight, Data Quality Management, Data Quality in the pipeline, Master Data Management etc. Design, implement & maintain data quality technology Partner with key project initiatives and act as escalation point more »

contributions are valued. We offer world class learning and career development opportunities, while we celebrate an inclusive culture. The role As a Lead Data Scientist (NLP & GenAI) at Allianz Commercial, you will work closely with our team of data scientists, data engineers, ML engineers … their strategic objectives. Research, design, and develop solutions using NLP models and algorithms to extract insights from unstructured text data. Collaborate closely with data engineers to pre-process and clean text data, ensuring data quality and compatibility with NLP models. Apply machine learning and … utilize LLMs, such as GPT, BERT or open source foundational LLM's (Llama2), for natural language understanding and generation tasks. Conduct thorough exploratory data analysis to gain insights into text data characteristics and develop effective pre-processing strategies. Collaborate with data engineers and MLops more »

SIEM environment, utilizing OpenSearch. The SIEM Onboarding Engineer will work closely with business units to identify devices for onboarding, manage the data pipeline, and assist other engineers in configuring their data sources to provide a resilient pipeline.The SIEM Onboarding Engineer will approach each system to … to identify devices and data sources that need to be onboarded to our SIEM (OpenSearch with Security Analytics). Data Pipeline Management: Configure and manage data collection agents to ensure data is reliably ingested into the SIEM. Automation: Leverage automation scripts … with security engineers and other stakeholders to ensure seamless integration and optimal performance of the SIEM. Monitoring and Troubleshooting: Monitor the data pipeline for issues, troubleshoot problems, and implement fixes to maintain data integrity and system reliability. Documentation: Maintain comprehensive documentation on the onboarding processes more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »