1 to 25 of 185 Data Pipeline Jobs in London

Key Responsibilities As a Data Architect, you will support the clients in their journey to become data-driven, by capturing and understanding their business needs, and defining and implementing Data and Analytics reference architectures, solutions and roadmaps. Furthermore, you will actively help our practice … in its objective to stay a Leader in Data & Analytics. • Define, guide and advise clients in developing data strategies • Engage with the clients to understand their business needs, pain points and realities • Gather, challenge and prioritize clients data requirements • Define and design innovative Data … architectures and solutions that leverage the right trade-off between clients realities, latest and disruptive technologies, current and future business needs, budget constraints, data availability and resource requirements. • Oversee the design of the data architecture and the technical architecture for the old-fashion and modern data more »

Job summary The Senior Data Engineer will have direct oversight and responsibility for the development, implementation, review, and establishment of data and techniques used in the Advanced Data Analytics Portfolio (DAP). As a Senior Data Engineer, you will work with project … managers, data scientists, counter fraud experts, stakeholders, and senior management across the full extent of the NHSCFA and our wider stakeholders in pursuit of the above. Your primary responsibilities will include developing data pipelines, processing data, and preparing data suitable for analysis. … data wrangling methods, models, data structures, and data formats. oJSON, SQL, and XML oWriting robust data pipeline code that can run unattended. oMachine learning for engineering practices, such as meta driven intelligent ETL and pipeline processes. oStrong skills in relevant more »

Job Title - Data Engineer Entity - Aventum Reports to - Head of Data Location - London/Hybrid Who We Are At The Aventum Group, we’re not one for following the usual rules. We work hard at being different and that very much extends to how we look … to advance their careers at Aventum. Role Summary You will work in various settings to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. Your ultimate goal is to make data accessible so … lifecycle design, data ontology alignment, annotated datasets, and elastic search approaches Developing, creating and maintaining a reliable data pipeline and schemas that feed other data processes; includes both technical processes and business logic to transform data from disparate sources more »

Retail and CPG, and Public Services. Consolidated revenues as of $13 billion. Job Description - Job Description- We are seeking a talented and experienced Data Architect to join our team. The ideal candidate will have a strong background in designing and implementing data solutions in an AWS … DevOps microservices environment using Kafka and Snowflake. As a Data Architect, you will play a crucial role in defining and implementing the data architecture, data modeling, and data integration solutions for our organization. Responsibilities : Design and implement scalable and efficient data architecture solutions in an AWS environment, leveraging microservices architecture and DevOps principles. Work closely with development teams to understand data requirements and design appropriate data models and schemas for various applications and services. Implement and maintain data pipelines using Kafka for real more »

Job Title: Data Pipeline Engineer Location: London Hexegic is a leading technical consultancy providing agile multi-disciplinary teams to high performing organisations. The company promises exciting, engaging and rewarding projects for those that are keen to develop and build a successful career. The Role The Data Pipeline Engineers ensure that our customer work with updated and correct data in our data analytics tool. You are the first to respond to data health failures on key pipelines. You will be comfortable reading code to identify fixes and making … that pipelines run effectively. Setting up and maintaining health checks on different data pipelines Respond, Triage, and debug the data pipeline when broken. Reading code and writing code changes and/or modifying the monitoring set-up Knowing and understanding how to navigate the data more »

Azure Synapse Data Engineer - £75K - Remote (London) My client a market leading provider grants clients access to bespoke data insights and analytics specific to their needs, providing research, data and expertise to support clients and all lines of business (Advisory, Digital, Search). They … and loading of data from a wide variety of data sources using Azure Synapse tools and technologies. Data pipeline development/testing. Test the reliability and performance of each part of a system during the development phase. Identify, design, and implement process improvements … automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability and data quality. Track pipeline stability. Monitor the overall performance and stability of the system and ensure automated parts/scheduled tasks are monitored. Manage structured and unstructured data and more »

Data Quality Engineer (CDP Customer Data Platform) – Camden/Hybrid – 6 month contract £600-£650 per day outside IR35 Are you an experienced Data Quality Engineer with proficiency in SQL and data querying? Are you interested in joining a giant in the … retail and fashion sector? As the Data Quality Engineer you will be at the forefront of the team ensuring data quality is maintained in the overall data, data pipelines and data warehouse. KEY RESPONSIBILITIES Designing and executing comprehensive data validation strategies, including data profiling, data cleansing, and data transformation. Conducting data quality assessments and audits to evaluate adherence to data quality standards and identify areas for improvement. Collaborating with data engineering teams to design and more »

brand licensing through IMG; live event experiences and hospitality through On Location; full-service marketing through global cultural marketing agency 160over90; and sports data and technology through IMG ARENA and OpenBet. Endeavor is also the majority owner of TKO Group Holdings (NYSE: TKO), a premium sports and entertainment … multiple greenfield projects supporting internal business units and external clients Ensure the accuracy and reliability of data across the data pipeline Bring positive innovative ideas that lead to innovative solutions rather than a lift-and-shift mentality Constantly stay up to date on new technologies … Ingestion tools such as FiveTran or Stitch Experience using Business Intelligence tools such as PowerBI, Looker or Tableau Experience with data pipeline tools such as DBT, Airflow or Luigi are a plus! Experience using cloud environments e.g. Azure or AWS Understanding of the Agile delivery method more »

Truth Builds Trust If our mission, values, and purpose align with your own, we would love to hear from you! Your opportunity The Data Engineer is a crucial member of the Investments Data, BI & Analytics Technology team, tasked with developing, managing, and optimizing data pipelines within our data ecosystem. Reporting directly to the Head of Investments Data, BI & Analytics, this role is instrumental in implementing the data strategy that supports front office stakeholders, systems, and clients. The Data Engineer will leverage cutting-edge technology … within our subsidized onsite canteen Must have skills Proven experience as a Data Engineer, with a strong background in data pipeline construction, data architecture, and data warehousing. Expertise in Snowflake, Python, SQL, and cloud-native ETL/ELT tools. Familiarity with more »

business in the coming years. We’re looking for an engineer to help us build BPP’s platforms, products and user experiences. The Data Engineering team will enable BPP to leverage data, to inform and optimise how we deliver scalable, secure and performant experiences that delight … and engage learners during their time studying with BPP and beyond, throughout their working lives. As a Data Engineer, you will report to the Data Engineering Manager. Collaborating closely with teams across BPP P&T department (Data, Product Management, Product Design & Engineering), you will … help deliver and evolve BPP Education’s vision to become more customer centred, design and data informed, to build products that meet and exceed our users’ needs across our education ecosystem. Key responsibilities Run data pipelines to extract data from various source systems (log more »

Job Purpose As a Data Engineer, you'll be at the forefront of enhancing our client's data capabilities, ensuring every bit is optimised for success. You will be collaborating closely with their visionary Data Scientists, crafting intricate data models and stunning … visualisations that will drive strategic decision-making to new heights. Your expertise in programming languages and data analysis tools will be the key to unlocking their data's full potential. Your contributions will be pivotal in driving strategic decision-making, fueling various impactful use cases. Harness … your prowess in programming languages and data analysis tools, while showcasing your formidable analytical and communication skills. Responsibilities Support the design and maintenance of data integrity checks across our client's various systems. Collaborate with the Data Scientists to understand data needs more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … science, and engineering discipline. Experience performing data warehouse architecture development and management. Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … best of open-source data engineering and scientific Python toolset. Our tech stack includes Airbyte for data ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data more »

Location : London, Must have SC Clearance Certificate Mode: Hybrid Salary: Competitive Overview: We are seeking a talented Data Engineer with expertise in Azure to join our team. The ideal candidate will be responsible for developing data pipelines, building and maintaining data warehouses and data lakes, implementing data integration and transformation solutions, ensuring data quality and integrity, and optimizing data performance. Responsibilities: Develop data pipelines for ETL/ELT (Extract, Transform, Load) processes using Azure technologies. Build and maintain scalable and efficient data warehouses and data lakes on Azure platforms. Implement data integration and transformation solutions to enable seamless data flow across systems and applications. Ensure data quality and integrity by implementing data validation and cleansing processes. Perform performance tuning and more »

management firm that utilizes long-short equities, long-only, fixed income and commodities futures strategies across its funds seek to add a experienced Data Engineer Risk Developer. They have deep expertise in trading, technology and operations and attribute their success to rigorous scientific research. As a technology and … data - driven firm, they design and build their own cutting - edge systems, from high- performance trading platforms to large- scale data analysis and compute farms. The group manages the lifecycle of data used by investment for trading, backtesting and research. Working with quants and … strong understanding of data engineering principles and best practices) experience in database technologies (PostgreSQL, SQL), and query optimization with data pipeline development and automation. coding skills (mainly with Python) knowledge (understanding of risk concepts such as VaR calculations) and good understanding of statistics with version more »

Introduction Axtria is a global provider of cloud software and data analytics to the Life Sciences industry. We help Life Sciences companies transform the product commercialization journey to drive sales growth and improve healthcare outcomes for patients. We are acutely aware that our work impacts millions of patients … Artificial Intelligence and Machine Learning. Our cloud-based platforms - Axtria DataMax™, Axtria InsightsMax™, Axtria SalesIQ™, and Axtria MarketingIQ™ - enable customers to efficiently manage data, leverage data science to deliver insights for sales and marketing planning and manage end-to-end commercial operations. With customers in over … to be the leaders we produce for the industry. Will you be next? Job Description: We are seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have 5-7 years of recent experience in data engineering and hold a more »

Position: Data Engineer Reporting to: Lead Data Engineer Contract type: Full Time, Permanent, Remote (with Occasional Travel) This is an exciting opportunity to join one of Europe's fastest growing healthtech startups About Cera At Cera Care, our mission is to empower people to live longer … the global leader in revolutionising affordable care for the better, for our service users and society and the perfect coordination across care practice, data, science and technology is how we’ll get there. . About Data at Cera Data has a critical role to … operational workforce, from caregiver recruitment to care delivery to our client base, and provides insights for performance improvement and cost-efficiency. Health-related data offers a comprehensive financial overview that could lead to cost savings and crucially lead to health deterioration prevention. The primary mission of our Data more »

Job Summary: We are looking for a skilled and experienced Data Architect Technical to work as part of our IT team. As a Data Architect Technical, you will be responsible for designing, developing, and maintaining the data architecture for our organization. You will have … a strong understanding of data integration best practices and will play a key role in driving successful data integration and transformation initiatives. Responsibilities: Overall Solution: Over 15 years of IT experience and over 7 years of experience in architecting, designing, developing and maintaining an End to … visualization skills for data models and logical workflows. Continuous Improvement: Analyze performance metrics and propose improvements. Provide resolution to data pipeline challenges. Cloud Data Technologies: Work with Cloud Infrastructure Architects to create data security, backup, disaster recovery, and archiving strategies. Advocate more »

Data Engineer – London - £85,000 Description Are you passionate about data engineering and looking for an exciting opportunity in the retail industry? I am seeking a skilled Data Engineer to join my clients team and play a pivotal role in their data-driven decision-making process. As a Data Engineer, you will work closely with data scientists, analysts, and other stakeholders to design, develop, and maintain data infrastructure. Your expertise will contribute to improving data pipelines, optimizing data flows, and … ensuring data quality and consistency. Responsibilities Design and implement scalable data solutions to collect, process, and analyze large volumes of data Collaborate with cross-functional teams to identify and define data requirements Develop and maintain data pipelines and ETL processes more »

Find your magic with us Connecting people to their family stories is special. We’re proud to deliver these magic moments. Through technology, data and insights, we bring people closer to their past, to benefit their future. We’re a fast-growing team of smart people, on a … the agility of a scale-up with the security of real financial backing, in a place where every story matters. Why join our Data Science and Insights team? We’re driven by the desire to create a fantastic customer experience We love nothing more than taking big data … up to date data We have a modern data stack already in place comprising of a Snowplow data pipeline, a Snowflake data warehouse, dbt as the data transformation tool and our BI tool, Looker. We don’t expect you more »

As the Data Architect, you'll lead data architecture initiatives, establishing standards and practices essential for effective data management. Your focus will be on designing and implementing robust data and information systems, ensuring they meet stringent criteria for modelling, flow, lineage, quality … and security. Working closely with stakeholders and data governance teams, you'll drive adherence to established standards, facilitating data-driven decision-making processes across the organisation. This role encompasses: Developing and maintaining a comprehensive Data Architecture Framework and maturity roadmap. Enforcing best data practices, setting standards, and ensuring compliance. Creating data models and overseeing the implementation of changes to address evolving business needs. Providing leadership in evaluating and implementing master data management and metadata management solutions. Collaborating with IT operations and delivery teams, producing essential data more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. The ideal candidate will have a proven … of large data sets Data quality system and process design and implementation. Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

brand licensing through IMG; live event experiences and hospitality through On Location; full-service marketing through global cultural marketing agency 160over90; and sports data and technology through IMG ARENA and OpenBet. Endeavor is also the majority owner of TKO Group Holdings (NYSE: TKO), a premium sports and entertainment … company comprising UFC and WWE. What You’ll Do: The Lead Data Engineer role sits within the Data Services department, you will work actively to unlock the value of data by building and enabling product teams and data practitioners to use the … parties involved to ensure a smooth process with minimal disruption Ensure the accuracy and reliability of data across the data pipeline Advising on product upgrades and Improvements Responsible for coordinating the software development process Work with the team to ensure that features are estimated correctly more »

Senior Data Analyst/Data Modelling/SQL/Tableau/Python/Databicks Role: Senior Data Analyst Company: Renewable Energy Package: Up to £80,000 Location: London (Hybrid, 1 day a week in office) A world leader in renewable and sustainability energy are … currently in search of a seasoned Senior Data Analyst to join their growing Data team. As the Senior Data Analyst, you will be instrumental in providing comprehensive data analysis, utilizing your expertise in Tableau to unearth actionable insights that inform strategic business … Python. Experience with advanced analytics techniques, including modeling and segmentation. Strong visualization skills, including proficiency with Tableau. Familiarity with cloud-based data pipeline architectures (e.g., Databricks) and platforms (e.g., AWS). Knowledge of Git-based source control methodologies, including branching and pull requests. Proven track record of more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data more »