76 to 100 of 183 Permanent Data Pipeline Jobs in the UK

Job Title: Software Engineer (Rust/HPC/Real Time Data) Client: Elite Tech Firm Salary: Up to £200k 1st Year Compensation Location: London Sells: Cutting-edge tech, ownership of multiple greenfield projects, no red tape, a friendly/collaborative environment, beautiful offices, personal projects on Fridays An … as part of a team of engineers looking after and building a zero-down time low latency solutions for multi petabyte/exabyte data sets Working heavily in Rust Building/Optimising some of the world's most advanced data pipelines Collaborating with ML Engineers, Data … with other incredibly passionate technologists Skills: 5 years experience working as a Software Engineer ideally using Rust Outstanding CompSci Fundamentals needed Knowledge of data engineering principles/data lake architectures - Arrow/Parquet etc. Excellent knowledge of low-level concepts (e.g. compilers, x86, LLVM etc. more »

new services? We are partnering exclusively with a large organisation to bring you an exciting opportunity for a highly skilled and experienced Lead Data Engineer to be responsible for leading data engineering projects, ensuring the efficient processing and management of data across the organisation. … other benefits. Flexible, hybrid working from Swansea with options for condensed hours. You will get to lead a team working on a modern data platform, collaborating with engineering product teams delivering services and working closely with reporting teams and data scientists to help understand and optimise … their digital services. This role will be instrumental in shaping their data infrastructure, ensuring it supports the strategic objectives. You will leverage cutting-edge technologies, primarily focusing on Microsoft Azure and Databricks, to develop and maintain robust, scalable data solutions. Responsibilities: Leadership and Strategy: Lead the more »

our team with people just like you We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don t engage BJSS to … get to work with a wide range of tools and technologies and there are always new things to learn. About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

Java Data Engineer - 6 Month FT Contract About us Amber Labs was born out of the recognition that many organisations require support on their data maturity journey. Our team is passionate and experienced in delivering cutting-edge data capability. Amber Labs was established with … a vision to provide specialist services, automation, accelerators competitively to organisations across EMEA. The Project The Java Data Engineer needs to be able to programme in Java as the project uses Java to ingest the data into the platform. The Consultant would be helping to build … out the Data Lake and Lakehouse on Databricks, working with AWS. Skill Priority: AWS, Java (to support ingestion), Databricks - experience in PowerBi would also be useful. You will be a Engineer with past experience with java, data, and infrastructure (devOps). Java, Python, PySpark Mechanisms: MongoDB more »

Lead Data Engineer (Java/Python/AWS/Glue/Dremio). Investment Management. £160,000 - £170,000 25% Bonus Target and Benefits. Hybrid 2 Days a week in St Pauls Office. CANNOT SPONSOR CANNOT SPONSOR My client is a top tier multi asset investment management house … based in Central London. The Data Office team is playing a key role in helping build the future of financial services, working together with business partners to create client experiences that are changing the way people invest. You will work with smart, talented people across our business. We … platform s architectural design Development and testing of the integration, modelling, data persistence and querying tools and analytical systems Data pipeline implementation, maintenance and testing Metadata management processes and tools Monitoring the overall performance and stability of the pipelines Implementing data curation, metadata more »

LEAD DATA ENGINEER LONDON BASED £80,000-90,000 PER ANNUM This Real estate company is searching for a new Lead Data Engineer, to Revolutionise their expanding Data and Analytics team, and help build out brand new Data pipelines using Python and … SQL. You will also be responsible for building a new Data platform in GCP. THE COMPANY This growing company are searching for a new Lead Data Engineer, to own the technical direction of their expanding Data and Technology team. They have been investing heavily … in their data team over recent years, and are a very data-focused business. THE ROLE Joining this team as a Lead data engineer, you will be responsible for building the company's next-generation data platform using GCP. This role has more »

Partnered with a global fintech firm, in the prime brokerage space, that are looking for a skilled SWE (data focused) to join their London office as part of their core ledger & populator team, ensuring connectivity is in place to move/ingest large amounts of (market) data and systems in place to move money. Responsibilities: Build connectivity to facilitate the ingestion of market data and trading activities from both traditional and digital exchanges. Develop data pipelines for transferring large volumes of data, enabling real-time analysis. Build systems to … nice to have) Prior experience working with a mid-early stage fintech start up (nice to have) Experience with standard tooling for big data (nice to have) Experience in cryptocurrency (nice to have) This is a great opportunity to join a fast pace growing company with good funding more »

Data Platform Manager Location: dmg::media Headquarters – London Position: Permanent Package Description Our benefits package increases the longer you’ve been with us. Here’s what to expect: 25 days’ holiday (increasing by 1 per year up to a total of 30) Upon joining you will be automatically … many other benefits…. Job Introduction DMG:Media own some of the world’s biggest news brands, and as such processes billions of data points every month in order to provide insights and targeting criteria for teams and stakeholders as part of our ambition to be the world … s leading Publisher when it comes to the sophisticated application of data – to power relentless growth. DMG:Media is seeking an experienced and talented Data Platform Manager to evolve its Big Data Platform and advance its overall data proposition. The successful candidate more »

risk analysis and ongoing risk research while supporting and refining the existing processes. You will collaborate with other risk engineering teams providing risk data, analytics and visualisation primitives.This is a highly collaborative role, requiring linking together data & analytics from varied sources into a uniform, actionable, high … role will be varied, including overseeing the implementation and rollout of new risk models, optimising, and maintaining analysis and reporting from 100s TiB data, architecting and implementing new datamarts and providing thought leadership and strategic direction for the future architecture of the Man Risk Platform. You will be … an emphasis on Python and the Python scientific stack: numpy, scipy, pandas, scikit-learn, etc. We implement the systems that require the highest data throughput in Java. We implement most of our long running services and analytics in C#.We use Airflow for workflow management, Kafka for data more »

days on-site. Key Responsibilities: Design and implement integration solutions using Azure, M365, Active Directory and Intune. Develop and maintain APIs, services, and data pipelines. Collaborate with stakeholders to ensure integration solutions meet business needs. Troubleshoot and resolve integration issues. Perform system monitoring, performance tuning, and optimisation. Document more »

Engineering Lead (Data) - Personalised Communications & Experiences Platform SALARY: £86,964 - £102,310 LOCATION: Bristol (Hybrid, at least two days per week in the Bristol Office) Opportunity Lead multi-functional engineering teams within the Personalised Experiences and Communications Platform, delivering high-quality data capabilities and leveraging cloud … squads for skill development and career growth. Technical Solutions: Troubleshoot, debug, and resolve issues; deliver reusable technical solutions. Patterns & Models: Define integration patterns, data models, and contracts. Automation: Create tools for data reconciliation and curation; drive process automation. Best Practices: Implement TDD, code reviews, and CI … operational stability. Expertise in automation and CI/CD. Proficiency in Python, Java, Rust, JavaScript, React, Angular, TensorFlow, PyTorch. Experience with large-scale data sets, data pipelines, and cloud platforms (AWS, Azure, Google Cloud). Experience in distributed systems and event-driven architecture. Practical experience with more »

I'm looking for a Data Engineer with experience working across SQL Server, Python, and ideally some Azure services to join a London-based events organisation. This is a hybrid role, requiring 2 days in the office and 3 days at home per week. You will be joining … their growing Data and Analytics team, who have been re-building and enhancing their Customer Data Platform in partnership with a well-renowned tech consultancy, to deliver better value and insights to various internal teams such as Commercial, Marketing, Digital etc. To do this, you will … leverage SQL, Python, Azure Cloud Services (e.g. Azure Data Factory) and Snowflake (though please note that prior experience with Snowflake is not a requirement). Their front-end reporting is in Power BI, which you may also have the opportunity to get involved in, though this is not more »

I'm looking for a Data Engineer with experience working across SQL Server, Python, and ideally some Azure services to join a London-based events organisation. This is a hybrid role, requiring 2 days in the office and 3 days at home per week. You will be joining … their growing Data and Analytics team, who have been re-building and enhancing their Customer Data Platform in partnership with a well-renowned tech consultancy, to deliver better value and insights to various internal teams such as Commercial, Marketing, Digital etc. To do this, you will … leverage SQL, Python, Azure Cloud Services (e.g. Azure Data Factory) and Snowflake (though please note that prior experience with Snowflake is not a requirement). Their front-end reporting is in Power BI, which you may also have the opportunity to get involved in, though this is not more »

SQL DBA - Fully Remote - £55K - £60K Overview: An excellent opportunity has arisen with a market leading, data driven Saas company for an experienced, motivated, and autonomous SQL DBA with skills in T-SQL and AWS. Role & Responsibilities: Manage AWS RDS SQL Server databases Monitor system's health and … T-SQL programming 1+ years of performance tuning Index analysis Profiler Performance monitor AWS console: RDS Databases Performance insight Knowledge of Data Pipeline and Glue (Desirable) Data warehousing Package: £50K - £60K basic salary Fully remote 25 days annual leave more »

A fantastic opportunity has arisen for a Data/Software Engineering leader to work within the Personalised Experiences and Communications Platform to join product engineering multi-functional teams . As a Head of data your responsibilities will be delivering the highest quality data capability … the software development lifecycle. Deliver technical solutions that can be utilised across multiple entities across the group. Define consumption and integration patterns and data models and contracts. Create automation tools for data reconciliation and curation. Drive the culture of delivering highly secured and high-quality software. … to make our Group a wonderful place for everyone. Including you. We understand no one can be an expert in every aspect of data/software Engineering . However, if you have coding or scripting knowledge/experience with a data engineering background , we would like more »

Data Engineer - London - Hybrid £60,000 - £70,000 + Package You will be responsible for the engineering of the data platform which is essential to the running and evolution of their mass participation event portfolio. This involves working with databases, coding in Python, integrating data from API's, building automated ingest and curation data pipelines, and using Azure cloud services to hydrate their core data system and downstream systems used for key business functions and analytics. You will be reporting into the Head of Data, working closely … support the thousands of Charities that they work with. Tech Stack: Python and SQL Demonstrable knowledge and skills in Azure Cloud Stack - Azure Data Lake, Azure Data Factory, Azure Logic Apps, Azure Function App and Azure SQL Managed Instance or Microsoft SQL Server. Skilled in T more »

shift, built for a new era of data driven decisions, predictive modelling, modern data framework, so the data pipeline is use in collective efficient decisions making. A team of two, movingly to scale our platform the technology that powers BULLISHIER , growing data … volumes of data, modernising our data platforms and removing bottlenecks to produce better outcomes for analytics and data pipeline workflows. Areas to cover will include: Quantum cloud-native disaggregated architecture (Machine learning and deep learning algorithms, AI , modern analytics, streaming data … and supporting internal and external data pipelines for complex data-based systems to the vast ecosystem of data pipeline tools. Areas to focus attention: The ability to move workloads back and forth, cost performance with simplicity factors appropriate for each application. In the more »

Data Architect We are searching for an experienced Data Architect About RSM Consulting brings together multiple teams across Transactions, Risk & Consulting, ABA, and Outsourcing to provide client-centric solutions for RSM s current and future clients within the middle market. Our Consulting team has a major … empower our people to make an impact and develop their own unique career path. About the team and your role Our Digital and Data Advisory practice, as part of our broader Business Transformation client offering, consists of three teams: Digital Finance team working with clients to understand how … from defining a digital strategy and roadmap, creating a business case, selecting the right software, documenting user stories, to managing and assuring projects. Data analytics and insights team working with clients to help them understand their level of data analytics maturity and to develop a data more »

Job Overview Position: Senior Data Engineer In this role, you will be responsible for storing, processing, modeling, and applying data science to ensure that data and insights are available for data analytics and business intelligence (BI) systems. You must be proficient in … data assessment and profiling, source system analysis, and providing a clear vision for processing and interpreting data to support its end-use. Ideal Candidate Profile The ideal candidate will have a background in Data … Engineering or Data Warehousing (Data Engineer/DW Developer) with robust DW development and ETL/ELT data pipeline experience, along with expert knowledge of SQL. Key Skills - Mandatory Strong data modeling and SQL/database design skills Proficiency in ETL more »

html Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? Position: Lead Data Engineer Lead Data Management and Architecture: Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data storage solutions and manage large-scale processing more »

ValueDriven is a leading Artificial Intelligence and Data Consulting company dedicated to providing credible, reliable, and value-driven outcomes for transformative impact. We specialize in the development of AI and data-driven solutions, helping organizations of all sizes make data-informed decisions and optimize … innovation, collaboration, and the delivery of outstanding results for our clients. We are currently seeking an experienced and highly skilled Senior Consultant in Data Engineering to join our team at ValueDriven. The Senior Consultant will play a crucial role in designing, implementing, and optimizing data solutions … for our clients, ensuring that their data infrastructure meets their unique needs and requirements. The ideal candidate will have extensive knowledge of data engineering best practices, technologies, and tools, as well as a passion for solving complex data challenges. Responsibilities: Collaborate with clients to more »

Learning Engineer (NLP & LLM) Up to £90,000 LONDON Ready to elevate your career to new heights? Join a team of experienced Data Scientists and Data Engineers to create impactful LLM s models and work with cutting-edge NLP technologies. The Role: Work end-to … performance, and make informed solution selections. Conduct comprehensive work spanning the entire data lifecycle, including data exploration, data pipeline development, insights and feature extraction, machine learning model creation, and deployment. Collaborate on consulting-style pilot projects aimed at helping clients gain a deep … understanding of their data and identifying problems that can be addressed through the platform. Collaborate closely with data scientists and data engineers to construct and implement machine learning models. Take an active role in leading team meetings and client workshops. Assist in establishing software more »

aspirations for strategic expansion and growth to be achieved through both organic and acquisitive means. The Role: We are currently hiring for a Data Architect, who will be accountable for architecting, enhancing, managing, and supporting a highly available, distributed, cloud data platform for Sysco International business … units and enterprise data. The Architect will collaborate with the International business units, US architecture team and Sysco 3rd party Vendors to create data solution andl ensure alignment between the international businesses in the deployment of initiatives. The role will also be the point of escalation for the … Expertise in both leading and hands-on experience in data engineering and management activities such as Data Integration, Building Pipelines, Data ingestion, warehousing, modelling and reporting. Hands-on experience in designing and managing large-scale, distributed data platforms, particularly using data more »

partnering with banks, businesses and fintechs all over the world. Partner with Visa: A Network Working for Everyone. What s it all about? Data Engineering Team at VXBS Our small but dynamic data engineering team we play a pivotal role in managing critical aspects of VXBS … s data ecosystem. AWS : Our cloud computing platform. Snowflake : Our trusted data warehouse platform. DBT Core : We leverage DBT for data transformations and modelling. Matillion : An essential tool for orchestrating ETL workflows. Datadog: For monitoring and alerting. Terraform for our infrastructure, including Snowflake. The … Balancing Act: As we embark on a transformation journey toward a more modern data stack, we find ourselves juggling our current workload while keeping an eye on the future. Our mission: seamlessly transition to a stack that empowers us even further. Join us as we navigate this exciting more »

team reflective of society improves outcomes. We provide support for any needs or adjustments during the application process. Role Overview: As a Senior Data Engineer, your responsibilities include: Developing modern data systems and platforms to drive decision-making. Collaborating with clients to shape their data strategy. Enabling public sector organizations to adopt a data-driven approach. Owning and managing cloud infrastructure using a DevOps approach. Building data pipelines, handling various data types, and integrating error handling. Implementing data system architectures like Data Warehouse … Data Lakes, and Data Meshes. Creating reusable libraries for data pipelines and documenting end-to-end systems on a cloud environment. Adopting a robust DevOps approach and utilizing tools for DataOps. Ensuring software resilience through testing strategies and best practices. Applying SOLID, DRY, and more »