101 to 125 of 372 Data Pipeline Jobs in the UK

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

Data Analyst/Engineer - £50,000 - £60,000/Carmarthenshire (South Wales) + Hybrid 2 days on-site Are you looking to join an exciting business as they continue their scale-up plans in 2024? This is a fantastic opportunity with a business proud to be expanding in … Wales. This is a great opportunity for a Data Engineer or Data Analyst/BI dev who wants to develop their skills in Data Engineering. If you have a background in creating dashboards in Power BI feel free to apply or reach out for … ll be doing: You’ll be the brain behind developing and putting into action strategies to gather, process, and dissect large sets of data, all aimed at giving clients actionable insights. You’ll focus on crafting and maintaining data pathways to ensure data accuracy more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure … to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data lineage. The successful candidate will have: A strong background in data engineering using Python Strong understanding of Test Driven Development Experience building Star Schemer Databases more »

Senior Data Engineer- Fast growing SaaS Scale up 🚀 West Midlands- hybrid working Up to £86K base with bonus and benefits The client SaaS Software business platform which is revolutionising financial management for their customers. Responsibility: Working to ensure a high-quality data platform. Design, develop, test … and deploy data pipelines and integrations using IPaaS technology Collaborate on engineering platform ingestion, orchestration, data warehouse/data lake and API strategies for the data management ecosystem Willingness and an enthusiastic attitude to work within existing processes/methodologies. Collaborate with … the DevOps team to provide high availability data solutions that scale geographically. Big data processing - Expertise in processing enterprise-scale volumes of data, ideally with proficiency in Snowflake. Experience Data Stores - Technical excellence in SQL, NoSQL, Blob,Delta Lake, and other enterprise more »

Senior Data Engineer Luton/Hybrid Job Purpose With a big investment into Databricks, and with a large amount of interesting data, this is the chance for you to come and be part of an exciting transformation in the way we store, analyse and use data in a fast paced organisation. You will join as a Senior Data Engineer providing technical leadership to the Data Engineering team. You will play a major role in adhering to big data best practices whilst building and maintaining reliable data pipelines and products across a range of exciting projects. Bringing a wealth of technical experience in big data platforms combined with excellent skills in development. This role will support the business on the journey from a greenfield big data estate to a mature data more »

Data Engineer Location: London – Hybrid Employment: Full Time All applicants must hold a DV (Developed Vetting) clearance to apply. The Opportunity: For the Ministry of Defence (MOD), maintaining National Security is a 24-hour-a-day, 365-day-a-year mission. For this reason, it partners with this … Big 4 Firm to support its multidisciplinary AI & Cloud Data Engineering Capabilities. The Role: As an AI & Cloud Data Engineering practitioner, you’ll build large-scale data pipelines and infrastructure, utilising data integration tools to optimise complex data solutions. Collaborating … with high-performing Data Science and ML Engineering teams to drive innovation and insight through AI and Data across the Defence & Security sector. Required Skills/Experience: British Passport Holder with active DV (Developed Vetting Clearance) End-to-end development with data pipelines, ETL more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing Real Time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support Real Time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

About Us Airtime Rewards is delivering the leading data-led rewards technology globally. We believe every shopper should feel great about their daily spend. Our app is loved by our members (>3.5m!) and we're continually building new features that drive towards our vision that sees hundreds of … and insight that transforms their understanding of their customers. We do all this with our cutting-edge proprietary technology, utilising real-time purchase data and AI learning to delight our members and maintain relationships with >200 retailers (including Boots, Aldi, Tesco, Greggs, Selfridges, and Halfords to name a … ourselves on culture, are solving new problems, and need great people to join us on our international journey as we deliver the leading data-led rewards technology globally. The Opportunity We are looking to recruit a Data Engineer to join and strengthen our existing team and more »

Data Engineer with Azure & Power BI Permanent £65-70k + Great Bens Hybrid Flexible Working - London area Talented and experienced Data Engineer/Power BI Developer, strong in both interpersonal communication skills, as well as key technical experience including; strong SQL, Azure Synapse Analytics and … Power BI (DAX & PowerQuery). This is a key technical role to join a small dynamic and friendly team, responsible for the data that enables our client to deliver premium entertainment to a Global network of over a billion airline passengers. You will … play a crucial part in the continued development and maintenance of the existing data warehouse solution, including data analysis, integration pipeline creation, SQL JSON querying and reporting using Power BI. The cloud based systems leverage data to support operational activities and inform business more »

Data Scientist - Hybrid/Bristol Job Title : Mid-Level Data Scientist Location : Hybrid/Bristol Remuneration : £50,000 - £60,000 per annum Contract Type : Permanent Responsibilities : Play a pivotal role in crafting, enhancing, and managing intricate data ingestion pipelines, ensuring their efficiency, scalability, and … dependability. Integrate advanced machine learning elements into data pipelines to elevate data processing and analytical capabilities. Devise and implement data quality metrics and dashboards for monitoring, ensuring the fidelity and precision of ingested data. Collaborate closely with data scientists, engineers, and cross … functional team members in an agile development setting to meet project objectives and deadlines. Harness modern NLP techniques to enrich data comprehension and processing. Stay abreast of the latest advancements in data science and engineering to continuously refine our systems and methodologies. About Us : Situated in more »

Job Role: Azure Data Engineer Location: Central London - Hybrid/3 Days in Office Salary: £60,000 - £70,000 + 12.5% Bonus Chapman Tate is collaborating with a leading property management company that is revolutionizing the industry through innovative data-driven solutions. As they continue to … expand their operations, they are seeking a talented Azure Data Engineer with strong SQL expertise to join their dynamic team. Are you passionate about leveraging data to drive business insights and enhance operational efficiency? Do you have experience working with Azure data technologies and … so, this is an exciting opportunity to contribute to the success of a forward-thinking property management company. Responsibilities: Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory. Optimize data storage and retrieval processes to ensure scalability, reliability, and performance. more »

I'm looking for a Data Engineer with skills in Databricks, Azure Data Factory and SQL Server, for a Fixed Term Contract role until August 2025. It's with an international organisation headquartered in London, and this role will require you to be in their offices … near Victoria station 3 days per week. You will be joining a small cross-functional data team, where you will focus on the development of their newly-created Azure Data Platform - this will involve designing and developing ETL pipelines using Azure Data Factory, and … utilising Databricks to support the ongoing development of the cutting-edge Data Platform. You'll seek opportunities for improvement where possible, and will collaborate with both business and technology stakeholders to ensure that the solution is fit for purpose. You will also design and develop data more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »

Senior Data Engineer Location: London (1 x a week in London) £55,000 - £65,000 per annum La Fosse is elated to be partnering with a boutique consultancy, who are seeking a Senior Data Engineering role. Join this team and work closely with clients to deliver … innovative data solutions. This consultancy has a mature data team and function, this role has amazing progression for those that have aspirations for a Lead Data Engineering role later on in their career! Responsibilities: Collaborate with clients to develop data engineering solutions … and performance. Utilize extensive SQL skills to create and optimize data extraction, transformation, and loading processes. Troubleshoot and resolve data pipeline issues promptly. Stay up-to-date with the latest data engineering technologies, suggesting process improvements. Requirements: Bachelor's or Master's degree more »

Staff Data Engineer Tech - Python, NoSql, Spark, AWS/GCP Salary – Up to £115,000 Location – Hybrid (twice a week on-site) A rapidly expanding AI, data, and research business is looking for a highly skilled Data Engineer to join their growing team focused … on delivering high quality data to their recovery system, LLM’s, and end users. As a Data Engineer, you will be responsible for the design, optimization, and maintenance of robust data pipelines ensuring secure data handling via end-to-end encryption and … data isolation. You will oversee cloud orchestration and data infrastructure using the most relevant and modern data processing tools. Skills required as a Data Engineer: Python NoSql Spark AWS/GCP This a challenging and rewarding position for a self-motivated Data more »

Contract Data Analyst Length: 6 Months (Possible Extention) Rate: Up to £400 per day (Outside IR35) Location: Central London (Hybrid 2/3 Days) I am seeking a skilled Data Analyst to join my clients Digital & Tech team for a contract role. You will aid in … supporting a system implementation and data migration project, collaborating closely with project members, including the Senior Data Analyst, to navigate data challenges and suggest optimal approaches. Ideal candidates have expertise in data collation, profiling, scripting, and report writing, with agility in deploying … Dynamic Tables, and Tasks (in SQL and/or JavaScript) Understand the purpose, functionality and deployment of unit tests in a data pipeline Python skills; be able to write scripts for data processing, cleaning, transformation and analysis Knowledge of various python packages; pandas, SQLite Power more »

We are currently seeking a Snr Data Engineer who has experience with the Azure Data stack to join the Business Intelligence team. The team is part of a wider team that includes the MS D365 implementation team (on-going project), Oracle ERP team and Digital Applications … to deliver MI reporting to our Finance stakeholders. Main responsibilities Collaborate with business stakeholders, process owners and BAs to understand requirements. To build data pipelines, reading data from various on-premise or cloud services to the Azure datalake. To design and build data models … suitable for reporting. To design and build Reports. Implementation of Data Quality, Data Governance, Data Security processes as part of the solution design About the Candidate Essential experience Experience with the following technologies Azure Synapse, Data Factory, Databricks, SQL Db, Datalake, Key more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »

change the status quo, we’ll help you make your moment. Join Team Purple. Join IFS. Job Description As a Principal/Senior Data Engineer, you will spearhead our data initiatives, transforming raw data into strategic assets that propel our business forward. This role … who excel in a dynamic environment, offering the freedom to innovate and the responsibility to deliver. Core Responsibilities: Identify and harness high-value data opportunities within our product suite, converting raw data into impactful features and reusable assets. Act as the authoritative voice on data … Strong programming skills in Python, Scala, C#, or Java, and adeptness with cloud SDKs and APIs. Deep understanding of AI/ML to enhance pipeline efficiency, with experience in TensorFlow, PyTorch, AutoML, Python/R, and MLOps platforms like MLflow and Kubeflow. Solid background in DevOps , including CI/ more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »

have the financial capital and expertise to grow their capabilities and reach, with a real emphasis on building out a truly world class data function. The role As a Senior Analytics Engineer, you contribute to the development, enhancement, and maintenance of scalable and resilient data pipelines … reports, and dashboards, as part of the businesses ongoing analytics evolution. You will also play an essential role in executing the data strategy and roadmap, leading the delivery of one or more key analytics projects, and working closely with and mentoring junior team members. What you'll be … enhancement, and maintenance of data models and ELT processes within a data mesh architecture using Python for data pipeline development and automation, following best practices with dbt and Snowflake. You will play a pivotal role in the delivery of key analytics projects, ensuring more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »