1 to 25 of 308 Data Pipeline Jobs in England

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: Perform system administration on specific cyber defence applications and systems more »

Data Scientist/Data Engineer - Mid Level - Hybrid in Bristol (2-3 days) £50,000 - £60,000 About Based in Bristol, our client is at the forefront of cyber risk assessment and management. Their innovative approach and proprietary models are revolutionising the way cyber risk is … and innovation, making them leaders in their field. Position Overview Our client is on the lookout for a talented and experienced Mid-Level Data Scientist/Data Engineer to play a crucial role in the development and operations of their automated ingestion system. This role is … ideal for someone passionate about crafting sophisticated data ingestion pipelines that incorporate advanced ML components and thrives in a dynamic, agile environment. Key Responsibilities - Automated Ingestion System Development: Be a key player in the design, development, and maintenance of sophisticated data ingestion pipelines, ensuring they are more »

Job Specification: Mid-Level Data Scientist/Data Engineer About Us Based in Bristol, we are at the forefront of cyber risk assessment and management. Our innovative approach and proprietary models are revolutionizing the way cyber risk is understood and mitigated in the reinsurance domain. We … excellence and innovation, making us leaders in our field. Position Overview We are on the lookout for a talented and experienced Mid-Level Data Scientist/Data Engineer to play a crucial role in the development and operations of our automated ingestion system. This role is … ideal for someone passionate about crafting sophisticated data ingestion pipelines that incorporate advanced ML components, and who thrives in a dynamic, agile environment. Key Responsibilities Automated Ingestion System Development: Design, develop, and maintain efficient, scalable, and reliable data ingestion pipelines. ML Component Integration: Integrate advanced machine more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. Implement data management standards, requirements, and specifications. Develop data standards, policies, and procedures. Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: Knowledge of big data more »

a Senior Go Lang Developer. Job Responsibilities/Objectives - Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Extensive ETL and Data warehousing experience - Strong experience in GoLang programming - Google Cloud Services and BigQuery Required Skills/Experience - Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application) - Understand and provide inputs for STT documents. - Design and develop the data pipelines more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. Implement data management standards, requirements, and specifications. Develop data standards, policies, and procedures. Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: Knowledge of big data more »

services from local and federal governments, as well as other companies, thanks to the software they supply. Though they have more experience with data science, you will be joining a team that is less skilled with Microsoft Azure technologies. Along with being a member of a team that … Azure technologies, along with a solid understanding of cloud architecture and networking for customised solutions. Infrastructure experience such as code deployments Knowledge of data pipelines for operations involving machine learning. Familiarity with implementing huge language models and machine learning models. Familiarity in setting up CI/CD processes. more »

Developer - £500 per day - London/remote - 12 months Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Duties Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using CICD … pipelines built. Essential Skills: Extensive ETL and Data warehousing experience Strong experience in GoLang programming Google Cloud Services and BigQuery ... more »

days pweek) iR35 determination - Inisde iR35 Duration - 6 months Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud … Key Responsibilities Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application ) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using … CICD pipelines built. Essential Skills Extensive ETL and Data warehousing experience Strong experience in GoLang programming Google Cloud Services and BigQuery ... more »

will need as much of the following as possible. Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Assess, analyse, data ingestion, transformation & storage layers (RAW, Exploratory, Curated application) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using CICD pipelines more »

Job Title: Cloud Data Engineer Location: Knutsford, UK Hybrid work: 2-3 days a week Capgemini is seeking a Cloud Data Engineer t o join Capgemini Financial Services I&D Practice. This is a permanent, full-time position in Knutsford, UK and this represents a unique … opportunity for someone to enhance their I&D Consulting career. What you’ll do and bring: Proficiency in AWS services relevant to data engineering such as S3, Glue, EMR, Athena, and Lambda. Hands-on experience with Snowflake, Redshift cloud data warehousing solutions. Familiarity with DBT (Data … Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of more »

Job Description: Key Responsibilities • Oversee the evolution of the data platform, encompassing data pipelines, storage, and processing. • Establish and nurture robust relationships with stakeholders to guarantee a positive user experience and to identify any potential opportunities for additional services or products. • Play a key role in … the formulation and execution of the data management strategy for the platform and ensure that the platform aligns with the Bank’s goals and objectives. • Guide the technical trajectory and blueprint of the platform in partnership with the engineering teams, including the adoption of novel technologies and solutions. … record in the development and management of an Azure cloud-based data & analytics platform. • Comprehensive experience in the complete data pipeline development process, encompassing data warehousing, data analytics, and ETL processes. • Profound understanding of best practices in data governance more »

Senior Data Engineer Location: Hybrid, with 2 days per month to be onsite (Abingdon) We have the pleasure to be representing a business that is on a mission to increase the number of people who can live longer, healthier lives and reduce unnecessary medical costs while doing so. … To help with the success of this strategy They are recruiting for a Senior Data Engineer that will take a pivotal role in driving the Data team forward. Responsibilities: Lead Cloud Data Warehouse Project : Take charge of leading the Cloud data warehouse … project through its next phase of development, ensuring its scalability, reliability, and performance meet the highest standards. Ownership of Data Management : Assume ownership of all aspects of data management, serving as the go-to person for any data queries from the internal team. Your more »

days on-site. Key Responsibilities: Design and implement integration solutions using Azure, M365, Active Directory and Intune. Develop and maintain APIs, services, and data pipelines. Collaborate with stakeholders to ensure integration solutions meet business needs. Troubleshoot and resolve integration issues. Perform system monitoring, performance tuning, and optimisation. Document more »

Leading eCommerce client are now searching for a Lead Data Engineer to play a pivotal role in delivering their innovative data strategy. This role will shape our client’s data function, delivering end to end data requirements and designing data pipelines/warehouses to ensure data is streamlined and enhances overall business success. The role: Design, develop and test an array of data migration processes to streamline the conversation from existing system to a new Snowflake infrastructure. Manage the end-to-end data lifecycle, enabling the use of data to influence overall business decisions. Streamline our client’s data processes to ensure that customer journeys are personalised. Ensure the accuracy and reliability of data. Collaborate on business projects, advising on data and data more »

and this role will face off with a range of business leads in order to build the pipelines that will ultimately ingest the data for their new strats! Responsibilities: Design and implement scalable ETL solutions (structured and unstructured; batch and streaming) Building data pipelines, standardizing, and … cleansing data for the investment team Define and set best practice standards surrounding data (i.e., data modelling, database design, ETL design, job scheduling and monitoring, etc) Manage the entire data flow, from acquisition to transformation into actionable insights using SQL and Python …/Java Design and maintain APIs Requirements: Degree in Computer Science. 4+ years of professional software development experience. A passion for data and experience in applying that passion to high quality data products. Strong knowledge of software engineering best practices, object-oriented concepts, and the ins more »

Principal Data Engineer Here at Anson McCade we are partnering with a leading Digital Transformation Consultancy who are in need of an experienced Data Engineer to join a multi-disciplinary team building, supporting and maintaining User-Centred digital solutions that offer real value and work for … is also desirable. Solid experience of at least one Cloud provider such as AWS, Azure or GCP. Strong experience of working with different data formats, e.g., CSV, JSON and XML Building reliable Data Pipelines Broader knowledge of IT — e.g., Security and Networking Working in an Agile more »

Data Engineer - Contract (Outside IR35) Overview: An Exciting opportunity has arisen with one of the UK’s largest life insurance brokers to join as a Data Engineer. Reporting to the Data Platform Manager, you will be responsible for ensuring that all of the raw … data is prepared into clear and coherent presentations for the stakeholders. The successful candidate will be someone with at least 3-5 years’ experience in a similar environment, strong SQL and Python knowledge and with the ability to communicate effectively with senior stakeholders. Role & Responsibilities: Collaborating with stakeholders … to develop and maintain products Working cross-functionally across the business to understand the requirements of the products Designing and implementing performance related data ingestion pipelines from multiple sources using Apache Spark Integrating end-to-end data pipelines ensuring a high level of quality is maintained more »

We are looking for a Data Engineer to join our Y Combinator backed start-up. They are using AI and ML technology to build a SaaS product. You will be working closely with the Data Scientists and wider engineering teams. Responsibilities Maintaining Python/SQL ETL … capabilities Collaborate with several teams to provide technical solutions Requirements Early-stage start-up passion Strong Python knowledge Experience building and optimizing "big data" pipelines more »

Job Title: Cloud Data Engineer Location: Northampton,UK Hybrid work: 2-3 days a week Capgemini is seeking a Cloud Data Engineer t o join Capgemini Financial Services I&D Practice. This is a permanent, full-time position in Northampton, UK and this represents a unique … opportunity for someone to enhance their I&D Consulting career. What you’ll do and bring: Proficiency in AWS services relevant to data engineering such as S3, Glue, EMR, Athena, and Lambda. Hands-on experience with Snowflake, Redshift cloud data warehousing solutions. Familiarity with DBT (Data … Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of more »

Data Engineer - Leading Pharmaceutical Company - Bristol (Tech Stack: Data Engineer, Databricks, Python, Power BI, TSQL, ETL, Agile Methodologies, Refactoring) Are you a talented Data Engineer looking for a new challenge? Our client, a leading pharmaceutical company based in Bristol, is seeking a dedicated and … skilled Data Engineer to join their innovative team. This role offers the flexibility of remote work, with the requirement to be in the office just once a month. Key Responsibilities: Develop and maintain advanced data pipelines using Databricks. Implement and manage rigorous testing processes. Utilise Python … for data manipulation and automation tasks. Create dynamic reports and visualisations with Power BI. Optimise and manage databases using TSQL. Handle complex ETL processes efficiently. Collaborate within Agile frameworks to enhance project delivery. Refactor code to improve performance and maintainability. Essential Skills: Strong proficiency in Databricks. In-depth more »

Data Engineer Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. They offer … a base salary of £50,000 and £60,000 Get in touch if you are looking for your next opportunity! Responsibilities: Data pipeline development in Python Development of the data warehouse stack Integrate additional data sources Experience: SQL Python AWS, Glue and Lambda more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

a critical role in developing and implementing machine learning models to drive the company’s products and services. You will work closely with data engineers and other stakeholders to ensure the scalability and performance of the solutions. This is a hybrid role with some travel to Manchester and … London to meet business requirements. Key Responsibilities: Design, develop, and deploy machine learning models Collaborate with data engineers to manage and optimise data pipelines Implement data governance and data management best practices Work with AWS tools, including Glue and SageMaker, to build … Participate in code reviews and contribute to the improvement of the development processes Key Skills and Qualifications: Proven experience in machine learning and data engineering Proficiency in Python Experience with AWS services, particularly Glue and SageMaker Strong understanding of data governance and data management more »

UK (Hybrid: Flexible working arrangements) Salary: Up to £90,000 per annum Are you a seasoned IT professional with expertise in Cyber Security, Data Compliance (GDPR), Data Engineering, Data Warehousing, and Cloud Engineering (AWS)? We have an exciting opportunity for an IT Director to … of IT initiatives alongside our CTO, aligned with business goals. Cyber Security measures, ensuring the protection of our systems and data. compliance with data regulations, particularly GDPR, and … implement best practices for data governance. data engineering projects, including data modeling, ETL processes, and data pipeline development. data warehousing solutions, optimizing for performance, scalability, and reliability. Cloud Engineering efforts, leveraging AWS services for infrastructure and application needs. and more »