1 to 25 of 373 Data Pipeline Jobs in the UK

business critical reporting systems across the Protein Works including marketing, trading, operations and finance. You will have a strong understanding and appreciation of data engineering concepts and how to apply them using the Google Cloud Platform. You will possess an unwavering passion for data and how … Engineer Immediate Responsibilities Managing and delivering end to end BI reporting projects from initial brief/requirement gathering to data pipeline development (APIs/ETL/ELT/SQL) to visualization Undertaking ad hoc BI tasks based on business requirements and priorities As part of … Coding experience with Python/Javascript/C# Familiarity with statistical/machine learning/AI concepts and techniques Understanding of data pipeline/orchestration tools e.g. dbt, dataform Appreciation of GCP’s serverless technologies e.g. Cloud Run/Workflows Understanding of Google’s marketing stack, Google more »

Contract Data Analyst Length: 6 Months (Possible Extention) Rate: Up to £400 per day (Outside IR35) Location: Central London (Hybrid 2/3 Days) I am seeking a skilled Data Analyst to join my clients Digital & Tech team for a contract role. You will aid in … supporting a system implementation and data migration project, collaborating closely with project members, including the Senior Data Analyst, to navigate data challenges and suggest optimal approaches. Ideal candidates have expertise in data collation, profiling, scripting, and report writing, with agility in deploying … Dynamic Tables, and Tasks (in SQL and/or JavaScript) Understand the purpose, functionality and deployment of unit tests in a data pipeline Python skills; be able to write scripts for data processing, cleaning, transformation and analysis Knowledge of various python packages; pandas, SQLite Power more »

Portfolio Manager and other investment professionals. Testing and deploying programs and applications. Troubleshooting, debugging, maintaining and improving existing software. Build and maintain robust data pipelines and databases that ingest large amounts of data every day, while developing processes that validate the integrity of the data. Qualifications more »

SENIOR DATA SCIENTIST (NLP & LLM) Up to £85,000 LONDON Ready to elevate your career to new heights? Join a team of experienced Data Scientists and Data Engineers to create impactful LLM's models and work with cutting-edge NLP technologies. The Role: Work … performance, and make informed solution selections. Conduct comprehensive work spanning the entire data lifecycle, including data exploration, data pipeline development, insights and feature extraction, machine learning model creation, and deployment. Collaborate on consulting-style pilot projects aimed at helping clients gain a deep … understanding of their data and identifying problems that can be addressed through the platform. Collaborate closely with data scientists and data engineers to construct and implement machine learning models. Take an active role in leading team meetings and client workshops. Assist in establishing software more »

Cloud Architect/Data Architect/Cloud Data Architect ⚙️ Google Cloud/GCP, Data Engineering Knowledge/Background, Kubernetes, Clusters, Spark, Hadoop, ETL Tools etc 🏡Remote Based, Work From Home/WFH 💷£90k-£120k basic salary + bonus Benefits: 25 days holiday + bank … Gym Membership (Nuffield + Hussle) Life Insurance, Income Protection, Employee Assistance Program Pension Scheme (max 10%) We are working with an AI/Data Consultancy with core clients within the Retail, Financial Services, Gaming, SaaS sectors. The successful Cloud Architect/Cloud Data Architect will work … devising innovative solutions to address their needs, defining project parameters, and supervising the execution of these projects. Role of the Cloud Architect/Data Architect: Utilising advanced data processing and cloud technologies effectively to stimulate innovation. Engaging with clients to create top-notch solutions tailored to more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Reading. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP, AWS, and/or Azure services Cloud Database Management: BigQuery, Snowflake, Databricks, and Redshift Architect data … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

and isn't afraid to disrupt the status quo, then this is the journey for you. You'll be the architect of seamless data integrations, weaving together sensors, systems, databases, and microservices. Dive deep into the realm of AI-driven workflows, sculpting systems that streamline operations and enhance … strategies. Distributed Messaging and Streaming: Hands-on experience with Apache Kafka or similar distributed messaging systems for building event-driven and real-time data pipelines. Database Technologies: Experience in working with different types of databases, including Relational Databases (RDBMS like PostgreSQL, MySQL), Time-Series Databases (TSDB like InfluxDB more »

Data Engineer – Salary National up to £35,000/London up to £45,000 Do you like working with the latest technology and are interested in enhancing your tech abilities? We have an exciting opportunity to join our highly skilled Data Engineer community. Within one of … our Data Engineering teams, we are looking for an individual who is passionate about data to work within our multi cloud environment to h elp shape the future of insurance through cutting-edge predictive analytics and Data Science. A bit about the job: Data … maintainable, and robust code in SQL or Python. Scala, Java or similar languages would be advantageous but not required. Experience in data pipeline engineering and best practice with CI/CD setup. Working with Oracle and Snowflake are desirable. Knowledge and expertise of data engineering more »

Role: Senior Data Engineer Location: UK (Remote-first with monthly travel to London required) Recently listed as a "RegTech Top Performer" in Market Fintech's RegTech Supplier Performance Report, CUBE is pioneering the development of machine automated compliance. We are a global RegTech business defining and implementing the … change for the financial services industry. We deliver our services through a SaaS platform, powered by an innovative combination of AI and proprietary data ontology, to simplify the complex and everchanging world of compliance for our clients. At CUBE, we are creating the future and are a company … detection, classification, entity extraction, summarisation, and recommendations. Available to customers and partners as APIs and via a UI. Role mission: Establish a strong data architecture in GCP for the backend of AI/ML/DS Services supporting a SaaS product. Build cost-efficient, low latency data more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

The ideal candidate will use their passion for big data and analytics to provide insights to the business covering a range of topics. They will be responsible for conducting both recurring and ad hoc analysis for business users. Work Location: London, UK We are seeking a skilled Data Analyst to help drive business insights and strategy through analysis of large data sets. The Data Analyst will work cross-functionally to collect, structure, analyze, interpret, and clearly present data findings that inform business decisions across the organization. Responsibilities: Consult with key … business stakeholders to understand questions and problems Data can help solve Design and implement processes for collecting structured and unstructured data from a variety of internal systems and external sources Organize, cleanse, and validate raw data to prepare for analysis Perform exploratory analysis to more »

Lead Data Engineer with Palantir Foundry Remote Contract NOTE:- The Consultant Needs to work In Europe Time Zone Description: Total experience of 7+ years in Data Engineering and must have … at least 3 years of experience in Palantir Foundry. Strong experience with Palantir Data Engineering features such as, Code Repo, Code Workbook, Pipeline Build, migration techniques, Data Connection and Security setup. Design, develop Data Pipelines, and have excellent skills in PySpark and Spark more »

for the pricing of multiple motor and home products, including the following core activities: Maintaining and improving behavioural models, prioritising a range of data science techniques. Advance the adoption of data science & statistical techniques. Develop reporting structures to monitor pricing performance in an automated fashion. Develop … Responsibilities: Maintain and improve the pricing process to meet business requirements, including; Development of market leading behavioural modelling using a combination of traditional & data science techniques Improve sophistication within the optimisation of direct brands Support delivery of pricing & underwriting business plan across Motor & Home portfolios Support development of … new product offerings Advance the adoption of the latest data science & statistical techniques across pricing & underwriting Research and leverage new and existing data sources; capturing and explaining trends with innovative data features Clearly communicate results and proposed actions to key decision makers across the more »

The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. … About you We know that many Data Engineer's will come from a variety of backgrounds, and we encourage and value people who bring unique perspectives and knowledge to our team. Your previous experience will include working daily with disparate data sources, ingesting, transforming and processing … data streams into actionable insights for direct use or for onward processing. You will have a proven track record of delivery and a passion for adherence to best practices in the data engineering discipline. You will be passionate about data in all its forms more »

developer 6 months contract – Competitive market rates – Inside IR35 - London – 3 days on-site Key Skills: Python, Power BI, SQL, Experience with Big Data pipelines, DAX, Excel, Financial services background The Role: A global trading organisation is looking a Python/Power BI to join their Data … be working with the business helping to build Apps using Python and Power BI for the output dashboards. Key Responsibilities: Develop and construct data pipelines to facilitate efficient extraction, transformation, and loading of data from diverse sources. Create and implement Business Information Access portals, comprising dashboards … continued deployment and enhancements. Innovate existing solutions and forecast future demands to meet evolving business needs. Develop analytics tools leveraging the data pipeline to offer actionable insights across various business functions, thereby enhancing key performance metrics. Emphasize automation and adopt an early feedback approach to streamline processes. more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure … to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data lineage. The successful candidate will have: A strong background in data engineering using Python Strong understanding of Test Driven Development Experience building Star Schemer Databases more »

dedicated to leadership and 30% to individual contribution. Your responsibilities include: - Driving the technical vision and ensuring alignment with business goals. - Scaling workflows, data pipelines, and integrations core to "her" functionality. - Orchestrating the building of new workflows and contributing to maintenance. - Identifying innovative ways to establish our architecture more »

Lead Data Engineer – Blockchain Remote across Europe, hybrid options – London office Up to £120k Lead the charge as a Data Engineer in the Blockchain realm! This role offers remote opportunities across Europe, with hybrid options available in our London office. Earn up to £120k while making … a meaningful impact in the Web3 infrastructure. Our mission? To introduce unprecedented transparency and depth to Web3 infrastructure data, reshaping interactions with blockchain technologies worldwide. As a key player in our team, you'll craft and maintain robust data pipelines and APIs, facilitating seamless connections across … our systems. Your mastery of Python and SQL will be pivotal in handling and analyzing vast data volumes. Requirements: Proficient Python skills, ideally showcased through an active GitHub profile. Previous experience in Web3 is essential, you have to have worked in a Web3 company before to be considered. more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Newcastle. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP services such as BigQuery, Dataflow, and Cloud Storage. If you have experience across AWS and Azure, we can … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … best of open-source data engineering and scientific Python toolset. Our tech stack includes Airbyte for data ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data more »

Data Quality Engineer (CDP Customer Data Platform) – Camden/Hybrid – 6 month contract £600-£650 per day outside IR35 Are you an experienced Data Quality Engineer with proficiency in SQL and data querying? Are you interested in joining a giant in the … retail and fashion sector? As the Data Quality Engineer you will be at the forefront of the team ensuring data quality is maintained in the overall data, data pipelines and data warehouse. KEY RESPONSIBILITIES Designing and executing comprehensive data validation strategies, including data profiling, data cleansing, and data transformation. Conducting data quality assessments and audits to evaluate adherence to data quality standards and identify areas for improvement. Collaborating with data engineering teams to design and more »

Data Engineer Embark on a Rewarding Journey with MIKI Travel: Where Your Career Takes Flight. At MIKI Travel, our 55+ years of expertise make us a trailblazing wholesale travel provider, curating unforgettable global adventures. As a premier private travel company with a presence in 36 offices, we invite … your career with opportunities for growth, powered by values of quality, innovation, and commitment that shape the Miki way. Job Purpose As a Data Engineer you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to enable efficient data ingestion, processing … storage, and analysis. The ideal candidate will have a strong background in data engineering, including expertise in building and optimizing ETL processes, data warehousing, and working with big data technologies. Main Responsibilities ● Collaborate with business stakeholders to gather and analyse business requirements, translating them more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Milton Keynes. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP services such as BigQuery, Dataflow, and Cloud Storage. If you have experience across AWS and Azure, we can … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

Lead Data Engineer Want to help drive the Data function with one of the most reputable employers in Belfast? This company is known for it’s innovation, delivery, and culture. Their Belfast hub is at the centre of the tech community, promoting learning and diversity at … all levels. They have a new opportunity for a Lead Data Engineer – this function is still relatively young in this region, and they’re looking for a thought leader in the industry to come in to this blank page and help write a new story. The person Experienced … in designing and deploying production data pipelines within a big data architecture Strong technical skills in Python, SQL and AWS (though they’re very open to any cloud experience) Experience processing large volumes of structured and unstructured data Experience of leading teams, mentoring junior more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … science, and engineering discipline. Experience performing data warehouse architecture development and management. Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality more »