51 to 75 of 482 Data Pipeline Jobs in the UK

with various jurisdictions. Implement scalable, supportable models for capital and liquidity management in collaboration with the Engineering team. Develop historical analysis and perform data research to back capital & liquidity models. Ensure balanced focus on internal management and external reporting for both capital and liquidity. Deploy production-grade code … and manage data pipelines as a full-time member of the Engineering team. Work closely with Risk, Structuring, Finance, and Compliance teams for regulatory interpretations, focusing equally on capital and liquidity management. Knowledge, Skills & Abilities Strong data science skills, particularly in Python and SQL, for data more »

Role: Data Engineer (Azure) Location: Bristol (hybrid on-site/remote) Salary: Up to £65,000 + package DOE Requirements: Active and Current UK SC Clearance Skills: Azure stack, Python, Java, ETL Company: Join a dynamic IT transformation consultancy at the forefront of innovation, delivering cutting-edge solutions … re seeking a Data Engineer with a focus on Azure, showcasing proficiency in: Leveraging Azure cloud technologies for tasks such as ETL pipeline development, data warehousing, data lake creation, and data movement. Utilizing Azure data and analytics services, including … but not limited to Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Stream Analytics, and Azure Synapse Analytics. A solid foundation in data engineering, data science, and product development, encompassing experience in both stream and batch processing. Designing and deploying production data more »

Job Title: Data Engineer Job Type: Full Time, Permanent Working location: London, Hybrid Role Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and … for you to solve. Our work is fast-paced although our team remains collaborative and flexible even when delivering fast. Your role as data engineer has a wide remit: working with product owners, subject matter experts and managers across the company, to design and deliver new components of … to the traditional data engineering type! Significant hands-on experience with AWS, especially AWS services focused on data flow, pipelines, data transformation, storage and streaming. Excellent data engineering skills, for example with SQL, Python, DBT and Airflow. Good understanding of service-oriented more »

LEAD DATA ENGINEER LONDON BASED £80,000-90,000 PER ANNUM This media company is searching for a new Lead Data Engineer, to Revolutionise their expanding Data and Analytics team, and help build out brand new Data pipelines using Python and SQL. … You will also be responsible for building a new Data platform in AWS. THE COMPANY This growing company are searching for a new Lead Data Engineer, to own the technical direction of their expanding Data and Technology team. This new AI-powered logistics platform … they formed in 2015 and aims to make world-class logistics available to every company. THE ROLE Joining this team as a Lead data engineer, you will be responsible for building the company's next-generation data platform using AWS. This role has many hands-on more »

in the last Series C funding round in January 2023. Your Team Autonomous vehicles depend on sensor data. The quality and dependability. Sensor data is central to performance and safety. The Sensor & Compute team is therefore at the very heart of Oxa’s software and platform development. This … team makes it possible for vehicles to know where they are and what is around them. Furthermore, by analysing the quality of data at run-time, the products of Sensor & Compute team allows vehicles to gauge and predict their own performance and safety margins. The diversity of sensor … peer coding reviews. Maintaining and updating drivers and diagnostic systems, with a focus on applications on low-power embedded devices Developing data pipeline capabilities to ensure that sensor data is high-fidelity and the autonomy system it serves can leverage the full capability of a more »

be working with a London-based B2B SaaS startup who are looking to expand the engineering team by bringing in a business-savvy data engineer to be the third member of the data team. The role: This role primarily covers building out the core product by … create a scalable SaaS product. Once this has achieved a certain level of product maturity, this role will also involve building their own data platform. We they offer: • Senior mentorship • Competitive compensation package, incl. stock options • Flexible hours and hybrid working, typically 3 days a week in the … a net-zero CO2 company and donate 1% of their revenues to social impact causes. Requirements: • A proven level of commercial experience in data engineering • Experience at seed to series A/B stage companies or an excitement to experience first-hand the joys and pains the come more »

Our client, a leading energy-focused trading house are seeking a commodity pricing focused developer/data engineer for their European Cross Commodities portfolio in London. The role will be focused on using pricing data for the commercial trading teams, and playing a key role between … internal clients and technical staff. A major part of the role will consist of facilitating the creation of large and automated data pipelines for commercial trading stakeholders. The individual hired will be hands-on in helping to collect, cleanse and analyse data from multiple sources, identifying … an individual who can harness cross-functional collaboration, across all technology, infrastructure, devops, and commercial trading stakeholders, ensuring alignment and successful implementation of data projects and initiatives. Requirements: Degree program in Computer Science or related field 2+ years of Python programming experience in financial services or within energy more »

Principal Data Engineer Northwest Up to £76,000 VIQU are seeking Principal Data Engineer to join a leading social enterprise working to transform lives by developing and managing vibrant communities. They excel in placemaking, regeneration, development, investment, property management, and leisure services. With a dedicated team … across numerous specialist subsidiaries, they create thriving environments and reinvest profits to enhance community projects, ensuring sustainable and flourishing communities. The successful Principal Data Engineer will be responsible for ensuring robust and compliant data processes on Google Cloud Platform. The key responsibilities of the role will … include data ingestion, quality, modelling, transformation, security, and monitoring. The Principal Data Engineer will also be involved in automation, process improvement, mentoring engineers, setting coding standards, and optimising queries. The Principal Data Engineer will work in collaboration with the Principal Cloud Engineer and Data more »

two days) at our Leeds, Edinburgh, or Bristol locations. About this opportunity Join IP&I as we embark on a transformative journey in data engineering. We are seeking a dynamic and experienced Senior Manager within Advanced Analytics to lead our team in architecting and implementing cutting-edge data solutions. If you are passionate about driving innovation and possess a proven track record in cloud and legacy data engineering leadership, we want to hear from you! You will be at the forefront of our data engineering initiatives. You'll lead a talented team … in designing, developing, and optimizing scalable data solutions. Your role will involve providing technical leadership, driving innovation, and ensuring the successful delivery of complex data projects. Here's where you'll make the difference: Lead and mentor a team of data engineers Architect and more »

Responsibilities: As a pivotal member of our rapidly expanding business, your role involves: Designing and Maintaining Data Architectures and Pipelines: Create and optimize robust data architectures and pipelines. Assemble intricate data sets that align with both functional and non-functional business requirements. Identify opportunities … to acquire new data from various systems. Enhancing Processes: Develop and refine data set processes for effective data modeling, mining, and production. Build the necessary infrastructure for seamless data extraction, transformation, and loading from diverse data sources. Data Reliability and Efficiency: Recommend and implement strategies to enhance data reliability, efficiency, and quality. Collaborate with Data Scientists and Machine Learning Engineers to construct analytics tools that leverage the data pipeline. These tools provide actionable insights related to asset management, operational efficiency more »

Data Visualisation Engineer Location: UK Salary: £41,000 per annum We are Forward, the social enterprise that empowers people to break the cycle of crime or addiction to move forward with their lives. For more than 25 years we have been working with people to build positive and … make positive changes and build productive lives with a job, family, friends and a sense of community. What you will be doing The Data Visualisation Engineer (DVE) is responsible for designing, developing, and maintaining data visualisation systems and dashboards. Using a variety of tools and technologies … to create interactive and visually appealing representations of data that enable users to quickly and easily understand complex data sets. Services are designed to deliver a range of interventions, programmes, workshops, advice and guidance for our clients. As a member of the Data & Performance more »

experience we're looking for, please go ahead and apply. We'd love to hear from you! About the role As the Senior Data Management Analyst, you will be responsible for continuous evolution of the data quality life cycle practices. This includes defining and proposing data … and their current levels under each dimension. You will take a leading role in supporting the organisation in improving the quality of its data by acquiring data to be brought under governance and perform data profiling analysis to understand it's origination, definition, lineage … into wider data management subjects e.g. Data Quality Insight, Data Quality Management, Data Quality in the pipeline, Master Data Management etc. Design, implement & maintain data quality technology Partner with key project initiatives and act as escalation point more »

secure operation of AI and ML systems. Develop and own robust enterprise security architecture for AI/ML systems, CI/CD and data pipelines. Participate in enterprise-wide architecture initiatives for AI/ML. This is a hybrid role with multiple UK locations Salary up to more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

ML systems, CI/CD and data pipelines. Participate in enterprise-wide architecture initiatives for AI/ML. Understand the workflow and pipeline architectures of ML and deep learning workloads. Conduct security risk assessments and threat modeling for AI/ML and other business projects performed thorough … test specification, development, and execution. Define and enforce robust access controls, authentication mechanisms, and encryption standards to safeguard AI/ML systems, training data, provenance, and any sensitive data. Stay abreast of emerging AI security threats, trends, and technologies, evaluating their impact and proactively adapting security measures to … tools, technologies, and frameworks to enhance the overall security posture of AI/ML and other business systems. Develop and maintain relationships between data scientists, data engineers, developers, operations (DevOps, DataOps, MLOps) and business unit leaders to govern and support scalability of the ML and AI more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

EU. We are looking for a Sr. DSP Insights Manager who is ready to roll up their sleeves and help us develop actionable, data-driven insights, using various quantitative tools and core project management skills. Key job responsibilities Core responsibilities include : Managing our organizations AWS account, including a … data warehouse in Redshift. Manage permissions structure by eg administrating IAM settings. Owning reporting for strategic roadmaps by working with business leaders; driving expansion and optimization of new business units in our DSP program. Developing and maintaining performance dashboards with key metrics for review with senior leadership and … business teams; synthesizing large quantities of data to help create new, step-change initiatives for the organization. Onboarding data to our data warehouse by building automated data pipelines utilizing eg Python in Lambda or Sagemaker to invoke API calls. Using statistical techniques more »

We're now recruiting a Senior BI Developer to work in the D2C business area of AJ Bell. The Data and Insights team is a small but dynamic and experienced group of SMEs who work across Google Analytics, Bloomreach CDP and BI/data engineering. The … drive business decisions by product and marketing teams, as well as the Senior Management Team. Youll be responsible for owning and developing our data model for the Direct-to-consumer business as well as creating custom solutions and fulfilling data analysis requests for key stakeholders at … of the role are as follows: Working closely with engineering and streaming teams across Tech Services and Operations to ensure reliable and accurate data is available for D2C data and insights team to develop solutions from. Be subject matter expert in D2C team for cloud-based more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

hybrid role 80% homeworking available and 20% in Stratford Upon Avon About the role We are seeking a skilled Developer to join our Data Engineering team. In this role, you will play a pivotal part inmaintainingand enhancing our technical service offerings, ensuring operational efficiency and effectiveness. As a … Developer in our Data Engineering team, you willbe responsible formanaging technical service operationssupporting data pipelines throughout the Cloud Data Platform. With a focus on problem-solving, adaptability, and analytical skills. Collaborating closely with stakeholders, you will gain a deep understanding of business imperatives and … Experience in data management, with an understanding of data storage and extraction principles. Azure Data Factory (error handling, pipeline triggers), Azure Storage, Azure Synapse and Azure SQL Server. Previous experience in technical service operations or a related field Strong problem-solving abilities, adaptability more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

work in an environment that's fun, fast paced, customer focused and commercial with fantastic development opportunities? Look no further than the Asda Data team! If you have recently graduated or are due to graduate this year, come and join us on a 2-year Data graduate programme, working in the Geospatial & Market Team. Joining Asda as a Data Graduate, you will be working within the Geospatial & Market team, who provide geospatial and market intelligence and insight across Asda. We work most closely with our property team, supporting location planning and finding … at an exciting time as were looking at opening hundreds of new convenience stores over the next couple of years. Furthermore, we provide data to support our Marketing and Buying teams to deliver targeted marketing campaigns and support operational decision-making. For example, we provide insight and analytics more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

We are partnered with a reputable global consultancy that are recruiting AWS DATA ENGINEERS to work on a very exciting LONG-TERM CONTRACT within the financial service sector. Role: AWS Data Engineer Rate: Up to £425 per day (inside IR35) Location: London Hybrid Duration: 6 months … backlog grooming. Key skills: Extensive experience in developing Bigdata pipelines in cloud using Bigdata technologies such as Apache Spark Expertise in performing complex data transformation using Spark SQL queries Experience in orchestrating data pipelines using Apache Airflow Proficiency in Git based version control tools Proficiency with … Linux commands and Bash Scripting Working experience in AWS Bigdata services (EMR, Glue, Data Pipelines, Athena, S3, Step Functions etc.) & AWS CLI Experience with CI/CD tools such as Jenkins Experience working with relational and/or MPP databases Knowledge of Data warehousing concepts Experience more »

building products that are performant, scalable, and robust to support critical transportation processes. You build web applications that delight our customers, and complex data pipelines to process the data that serves them. We will give you the space to explore new technologies and approaches, and apply … job responsibilities working with AWS technologies such as Lambda, ECS Fargate, API Gateway, RDS, DynamoDB, EMR building customer-facing applications and APIs building data pipelines using Spark + Scala that process Tb of data per day working with customers to understand the business context of new more »