101 to 125 of 178 Data Pipeline Jobs in London

Portfolio Manager and other investment professionals. Testing and deploying programs and applications. Troubleshooting, debugging, maintaining and improving existing software. Build and maintain robust data pipelines and databases that ingest large amounts of data every day, while developing processes that validate the integrity of the data. Qualifications more »

Lead Data Engineer/Lead Data Developer - Azure, SQL What's it all about? Information, Systems and Change and Data and Digital Solutions. You will be responsible for leading the technical design and implementation of data integration projects using Microsoft Azure and … Integration Services alongside other cloud components. The Lead Data Engineer will also oversee the quality, performance, and security of the data integration solutions. How you'll make a difference By delivering data integration solutions on their data platform, you will contribute to … the continuous improvement of data-driven decisions that improve the services the client provides for their customers. How you'll do it Lead the Engineering Team in the development of data and integration solutions leveraging the latest technologies, tools, processes, and best practice. Create and provide more »

a Senior Go Lang Developer. Job Responsibilities/Objectives - Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Extensive ETL and Data warehousing experience - Strong experience in GoLang programming - Google Cloud Services and BigQuery Required Skills/Experience - Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application) - Understand and provide inputs for STT documents. - Design and develop the data pipelines more »

a Senior Go Lang Developer. Job Responsibilities/Objectives - Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Extensive ETL and Data warehousing experience - Strong experience in GoLang programming - Google Cloud Services and BigQuery Required Skills/Experience - Assess, analyse, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application ) - Understand and provide inputs for STT documents. - Design and develop the data pipelines more »

will need as much of the following as possible. Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Assess, analyse, data ingestion, transformation & storage layers (RAW, Exploratory, Curated application) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using CICD pipelines more »

promises exciting, engaging and rewarding projects for those that are keen to develop and build a successful career. The Role The Data Pipeline Engineers ensure that our customer work with updated and correct data in our data analytics tool. You are the first … that pipelines run effectively. Set up and maintaining health checks on different data pipelines Respond, Triage, and debug the data pipeline when broken. Reading code and writing code changes and/or modifying the monitoring set-up Knowing and understanding how to navigate the data … providers when data is incorrect or not received on time. Communicating outages with the end users of a data pipeline What We Value Comfortable reading and writing code in Python, Pyspark and Java. Basic understanding of Spark and interested in learning the basics of more »

Senior Data Engineer Location: London, UK Duration: 9 months Role/Position Overview As a Senior Data Engineer, you will lead the team’s thinking when it comes to building, deploying, and hosting data engineering pipelines and solutions in production. You will be responsible … for designing and implementing scalable data pipelines, managing data infrastructure, and tooling, and working closely with stakeholders to deliver quality solutions. This role will be a key contributor to the ongoing development of our data architecture and data governance capabilities. We want … to minimize manual processes and strong software engineering proficiency in this role is key. Key Responsibilities: - Design and implement scalable data pipelines that extract, transform and load data from various sources into the data lakehouse. - Help teams push the boundaries of analytical insights, creating more »

Job Title: Junior Data Engineer Location : London (Anywhere in UK) Pay Rate : £150/Day We are looking to hire a Junior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures on AWS. You will work closely … with senior data engineers … data scientists, and other stakeholders to ensure data is accessible, reliable, and ready for analysis. Key Responsibilities: Data Pipeline Development : Design, build, and maintain data pipelines to ingest, transform, and load data from various sources into AWS data more »

Introduction Axtria is a global provider of cloud software and data analytics to the Life Sciences industry. We help Life Sciences companies transform the product commercialization journey to drive sales growth and improve healthcare outcomes for patients. We are acutely aware that our work impacts millions of patients … Artificial Intelligence and Machine Learning. Our cloud-based platforms - Axtria DataMax™, Axtria InsightsMax™, Axtria SalesIQ™, and Axtria MarketingIQ™ - enable customers to efficiently manage data, leverage data science to deliver insights for sales and marketing planning and manage end-to-end commercial operations. With customers in over … to be the leaders we produce for the industry. Will you be next? Job Description: We are seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have 5-7 years of recent experience in data engineering and hold a more »

Position: Data Engineer Reporting to: Lead Data Engineer Contract type: Full Time, Permanent, Remote (with Occasional Travel) This is an exciting opportunity to join one of Europe's fastest growing healthtech startups About Cera At Cera Care, our mission is to empower people to live longer … the global leader in revolutionising affordable care for the better, for our service users and society and the perfect coordination across care practice, data, science and technology is how we’ll get there. . About Data at Cera Data has a critical role to … operational workforce, from caregiver recruitment to care delivery to our client base, and provides insights for performance improvement and cost-efficiency. Health-related data offers a comprehensive financial overview that could lead to cost savings and crucially lead to health deterioration prevention. The primary mission of our Data more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … databases, as well as NoSQL databases Familiarity with cloud services (preferably GCP ) and understanding of its ETL frameworks. Desired : experience with data pipeline and workflow management (Airflow, Composer). Desired : Familiarity with machine learning algorithms and data science principles is a plus. Qualifications Degree in more »

Senior Data Engineer £65,000 - £75,000 + 10% Bonus London based - Hybrid working A consultancy at the heart of ground-breaking innovation and transformative solutions where expertise meets ambition. With a focus on collaborative brilliance shaping industries and propelling client towards their full potential whilst driving meaningful … change is seeking to add a Senior Data Engineer to the team based in London. Requirements of a Senior Data Engineer; Adeptness in utilising AWS and Azure data and analytics services (or their open-source equivalents), including but not limited to EMR, Glue, RedShift … Kinesis, Lambda, Data Factory, and Databricks. Strong experience in data engineering, data science, and product development, encompassing experience in both stream and batch processing. Proficiency in the design and deployment of production data pipelines, involving languages like Java, Python, Scala, Spark, and more »

Principal Data Engineer £80,000 - £103,000 + 15% Bonus London based - Hybrid working A consultancy at the heart of ground-breaking innovation and transformative solutions where expertise meets ambition. With a focus on collaborative brilliance shaping industries and propelling client towards their full potential whilst driving meaningful … change is seeking to add a Principal Data Engineer to the team based in London. Requirements of a Principal Data Engineer; Adeptness in utilising AWS and Azure data and analytics services (or their open-source equivalents), including but not limited to EMR, Glue, RedShift … Kinesis, Lambda, Data Factory, and Databricks. Strong experience in data engineering, data science, and product development, encompassing experience in both stream and batch processing. Proficiency in the design and deployment of production data pipelines, involving languages like Java, Python, Scala, Spark, and more »

Data Engineer Grade: Senior Consultant – Senior Manager Location: London Employment Type: Full Time The Opportunity: A Tier One Technology Consultancy seeks a highly ambitious and technically astute AWS Data Engineer to help design, build and deploy cloud-based data solutions - transforming raw data … verticals: Consumer & Manufacturing, Defence & Security, Energy & Utilities, Financial Services, Government & Public Services, Healthcare, Life Sciences, and Transport. Essential Skills & Experience: • Design and deploy data pipelines using Java, Python, Scala, Spark, and SQL in big data architecture. • Execute tasks involving scripting, API data extraction, and … SQL queries. • Proficient in data cleaning, wrangling, visualization, and reporting. • Specialised in AWS cloud technologies for ETL, data warehouse, and data lake design. • Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. • Capable of processing large volumes of structured and more »

knowledge on which to develop their careers and progress in the direction they choose. The Role: We are looking for an experienced Senior Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with a … fantastic team of technology experts. You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability. … Self motivated, enthusiastic and committed The ideal candidates will share our values and be aligned with our ways of working Key Responsibilities: Develop Data Engineering solutions for our clients/projects Design and build data models, schemas to support business requirements Develop and maintain data more »

Data Engineer Paddington, London (Hybrid) Overview Data and Analytics is foundational to our Petcare OGSM and will drive our transformation to a business that is powered by data. To deliver on this ambition set by this OGSM we require the very highest level of technical/… engineering expertise within Global Petcare Data & Analytics. There is a need to expand this high performing team of creative, skilled individuals – to help build out new capabilities and bring fresh ideas to the table. Key Responsibilities Engineer and orchestrate data flows & pipelines in a cloud environment … in a cloud environment (Azure, AWS, GCP) Experience in at least one of: Python (or similar), SQL, PySpark Experience in building data pipeline/ETL/ELT solutions Ability and strong desire to research and learn new technologies and languages Interest in the contemporary data more »

and Oaktree Capital. Find out more about us by visiting our website at: http://www.hartreepartners.com/JOB OVERVIEW: As a Senior Data Scientist at Hartree, you will play a pivotal role in shaping our data strategy and driving the development of advanced machine learning … time series forecasting solutions, specifically focused on the European Power Markets. You will collaborate with cross-functional teams to design, implement, and optimize data pipelines and predictive models that inform trading decisions and enhance operational efficiency. JOB RESPONSIBILITIES: Market Data Analysis: Conduct comprehensive analysis of data … Python and its data science libraries (e.g., Pandas, NumPy, Scikit-Learn, TensorFlow, PyTorch). Solid understanding of ML and data pipeline architectures and best practices. Experience with big data technologies and distributed computing (e.g., Spark, Hadoop) is a plus. Proficient in SQL and more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

Vertex Search is representing a fast-growing financial services firm that is now looking to make a Senior Data Engineering hire, the first role of its kind into the business with huge scope for impact and influence. The nascent data engineering team will collaborate with internal … teams around the business with a focus on data engineering, pipelines, tooling and services. Key Responsibilities: Data Engineering: Design and develop data services across various business functions Data Pipelines: Design and develop data pipelines for extracting, transforming, and loading data into appropriate storage systems. Data Tools and Services: Develop data tools and services to discover, understand, and utilize data effectively. Best Practice: Advise on and contribute to the development of data governance and management practices. Collaborate across teams to understand more »

Senior Data Engineer (Inside IR35) - £700 to 825 per day Location: Remote/Hybrid (London) Rate: £825 per day Contract: Inside IR35 Are you an experienced Data Engineer with a strong background in banking? Our client are seeking Senior Data Engineers to contribute to … innovative data projects within a leading financial institution. Key Responsibilities: Data Engineering: Design and implement data models, ETL processes, and data warehousing solutions. Programming: Utilize Python, Java, Scala, or GoLang to build and optimize data pipelines. Distributed Processing: Work with … Hadoop, Spark, and other platforms for large-scale data processing. Real-Time Data Streaming: Develop and manage pipelines using CDC, Kafka, and Apache Spark. Database Management: Handle SQL databases like Oracle, MySQL, or PostgreSQL. Data Governance: Ensure data quality, security, and compliance more »

About Tredence: - Tredence focuses on last mile delivery of insights into actions by uniting its strengths in business analytics, data science, and software engineering. The largest companies across industries are engaging with Tredence and deploying its prediction and optimization solutions at scale –empowering end users to improve decision … strategically and execute methodically. Develop and manage capacity and growth projection forecasts of the environment within budgets. Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional/non-functional business requirements. Drive optimization, testing and tooling to improve quality … years of relevant experience At least two years of experience building and leading highly complex, technical engineering teams. Strong hands-on experience in Data bricks Experience managing distributed teams preferred. Strong technical experience in large distributed systems, Data Warehousing, Data Lake at scale Project more »

Senior Data Engineer Python, SQL, Big Data Frameworks London - 4x a week £80-90,000 + bonuses Seeking a skilled Data Engineer to join a company specialising in quantitative trading and investment management. This role is pivotal in building and maintaining the infrastructure that … supports their cutting-edge trading strategies. Key Responsibilities: Design and manage scalable data pipelines. Integrate data from various sources and develop ETL processes. Optimise database systems for high performance. Work with data scientists and analysts to meet data needs. Automate data processing tasks to enhance efficiency. Implement data security measures and ensure compliance with industry standards. Requirements: Master’s degree in Computer Science, Engineering, Mathematics, or related field from a top uni. Proven experience as a Data Engineer, preferably in the financial services or hedge more »

Data Engineer Opportunity – AI for Finance Leader in London - £75,000-£85,000 Harrington Starr are currently partnered with an AI/Data leader within the Financial Services space in London. Their ground-breaking software is helping investors to achieve superhuman performance. In order to support … further growth, we are looking for a Data Engineer skilled in the following. What will you be doing? Work on a web scraping platform for one of the firm’s largest clients Develop solutions to parse and process tabular data from PDF and HTML documents Maintain … support and expand existing data pipelines using DBT, Snowflake and S3 Implement standardised data ingress/egress pipelines Onboard new, disparate data sets, sourced from many and varied data vendors, covering all asset types and frequencies from daily to real-time Liaise more »

of industries. Cerberus Technology Solutions (CTS) is a subsidiary of CCM that specializes in information organization, storage and analysis. The CTS teams include Data Science, Data Management and Client Engagement, which work closely together with clients to identify business opportunities and create new business value through … improved data handling and analysis. Responsibilities: Build predictive models using machine-learning techniques that generate data-driven insights on modern data platforms (Spark, Hadoop and other map-reduce tools); Develop and productionalize containerized algos for deployment in hybrid cloud environments (GCP, Azure) Connect and … blend data from various data sources within enterprise tools (python, pandas, or SQL) to enable application of Data Science methods Create metrics and analytical reports to ensure data quality and business value. Clean, structure and normalize data to eliminate redundant more »