1 to 25 of 29 Contract Data Pipeline Jobs in the UK

Senior Data Engineer (Databricks) – Remote – Leeds - 6 Months Contract (Initial) – Outside IR35 We are seeking 4 Senior Data engineers (Databricks) on a contract basis to assist our client as they migrate over from legacy to AWS. The main focus will be to centralise the organisations data warehouses and consolidating/centralising a number of BI tools. Senior Data Engineers should have extensive experience working with Databricks on recent projects and be exposed to the latest features and toolsets of Databricks Role Responsibilities: Design and develop scalable data pipelines using Databricks. … Implement data processing workflows in Python and SQL. Manage and optimise data storage solutions on AWS. Ensure data quality and integrity across all stages of data processing. Develop and maintain documentation for data engineering processes. Utilise Databricks features, including Delta more »

contract basis to assist our client as they migrate over from legacy to AWS. The main focus will be to centralise the organisations data warehouses and consolidating/centralising a number of BI tools. Databricks Engineers should have extensive experience working with Databricks on recent projects and be … exposed to the latest features and toolsets of Databricks Role Responsibilities: Design and develop scalable data pipelines using Databricks. Implement data processing workflows in Python and SQL. Manage and optimise data storage solutions on AWS. Ensure data quality and integrity across all … stages of data processing. Develop and maintain documentation for data engineering processes. Utilise Databricks features, including Delta Live Tables (DLT) to simplify streaming and ensure data quality. Work with asset bundles in Databricks for streamlined data operations. Key Skills: Proven experience as more »

Data Engineer 12 Month FTC £60,000 - £65,000 Nottingham - Hybrid A Data Engineeris needed at an established company in Nottingham as it continues a period of growth and expansion. The individual will come in at the start of an exciting project and will have the … ability to really make their mark on it. This position is to develop and the firms Central Data Platform. The candidate will have the skills needed to develop solutions using on-premise Microsoft BI and cloud-based Azure technologies and will have a proven experience in applying best … rapidly changing market as well as keeping it up to date with continuous improvement. What does the role actually involve? Design and develop data pipelines and data solutions on the Azure platform. Build and maintain data warehousing solutions on Azure. Develop and maintain data more »

the forefront of innovation to address the entire breadth of opportunities in the evolving world of cloud, digital and platforms. Context program: The Data & Analytics team's vision is to: Help answer complex questions quickly but bringing the right data and insights to the people that … need them Build a culture of using data with speed and agility to drive decisions. The goal of our 2024 program is to improve Global Procurement's data analytics capability and maximise value creation from HSBC's supply chain data. The program is a success when … Modernising Procurement's data & analytics platforms: extending our Data Management Platform (Google Cloud) and the associated data products Deliver Next Generation Insights: enabling data-driven decision making through both brand new and improved insights across Global Procurement domains Enable Supplier Mastering & Normalisation more »

Senior Developer (Data & Analytics) £450-£550 Inside IR35 12 months Hybrid Working - 2 full days a week working in London Talent are seeking a highly skilled Senior Developer with extensive experience in ETL/ELT processes, data warehousing/business intelligence methodologies, and best practices. The … ideal candidate will have a strong background in handling big data, cloud technologies, and unstructured data, along with a deep understanding of star schema structure and hybrid data warehouse design. As a Developer your key responsibilities will be: Design, develop, and maintain robust ETL …/ELT processes and data pipelines. Implement data warehousing solutions using Kimball and Inmon methodologies. Develop and manage cloud-based data solutions using Azure services such as Data Factory, Event Hubs, Data Lake, Synapse, and Azure SQL Server. Create and more »

Job summary We're on the lookout for an experienced Data Engineer to join our vibrant Performance, Information & BI team. In this role, your responsibilities will include designing, implementing & optimizing data pipelines for loading & maintaining the data warehouse to support the Trust's data & analytics needs. Join us at this exciting time as we delve into new data driven projects & work towards modernizing our data warehouse. The perfect fit for this role would have technical expertise in data engineering tools, strong data analysis skills … excellent problem-solving abilities & strong communication skills. A successful candidate will have: Extensive experience in data engineering, with expertise in designing, building & maintaining data pipelines, data warehouses, and/or data lakes. Significant experience in using ETL tools such as SQL Server more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … our impact using your knowledge, experience, and passion for your chosen field. The Data Scientist role requires expertise in machine learning, statistical analysis, anomaly detection, and strong communication and collaboration skills. Key responsibilities include developing innovative models, designing, applying, and optimising models in a dynamic environment. The role more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … our impact using your knowledge, experience, and passion for your chosen field. The Data Scientist role requires expertise in machine learning, statistical analysis, anomaly detection, and strong communication and collaboration skills. Key responsibilities include developing innovative models, designing, applying, and optimising models in a dynamic environment. The role more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … our impact using your knowledge, experience, and passion for your chosen field. The Data Scientist role requires expertise in machine learning, statistical analysis, anomaly detection, and strong communication and collaboration skills. Key responsibilities include developing innovative models, designing, applying, and optimising models in a dynamic environment. The role more »

appoint a talented Automation Test Engineer on a Contract basis. The role will work on creating test automation frameworks, developing test pipelines for data pipelines, and contributing to both manual testing and test management activities. About the role: Based in Greater London (Hybrid): Create test automation frameworks. Develop … test pipelines for data pipelines. Contribute to manual testing efforts. Assist in requirements and scenario definition and elicitation. Participate in test management and test planning activities. Create and manage CI/CD pipelines. About you: You will have the following experiences: Proficiency in test automation and knowledge of … test architecture. Experience with data and database testing. Strong test management and planning skills. Ability to create use cases, requirements, and scenarios. Local Authority/Public Sector experience is highly desired. What's on offer: Salary: £450+ per day (Inside IR35) *negotiable based on experience *please submit your more »

and reliability of their products through innovative testing practices. Key Responsibilities: Create and maintain test automation frameworks. Develop and manage test pipelines for data pipelines. Contribute to manual testing efforts and overall test management activities. Assist in requirements and scenario definition and elicitation. Participate in test planning and … and manage Continuous Integration/Continuous Deployment (CI/CD) pipelines. Skills and Qualifications: Proficiency in test automation and related tools. Experience with data and database testing. Strong skills in test management and planning. Ability to create use cases, requirements, and scenarios. Knowledge of test architecture … and best practices. About You: You are a detail-oriented and innovative professional with a strong background in test automation and data pipeline testing. Your ability to create robust test frameworks and manage test activities will contribute significantly to our product's success. Your experience with CI more »

office on a hybrid basis. As a Contract Senior Golang Developer, you will play a crucial role in assessing, analysing, and enhancing our data ingestion, transformation, and storage layers. Your primary focus will be on Go Programming and DBT, with secondary skills in Google Cloud Services. The successful … candidate will bring 5 to 10 years of experience in developing robust data pipelines, conducting unit testing, and providing support for production deployments. Key Responsibilities: Assess and analyse data ingestion, transformation, and storage layers, including RAW, Exploratory, Curated, and application layers. Understand existing codebase and make … alterations or fixes as per design requirements. Develop and optimize data ingestion pipelines using Go Programming and DBT. Conduct unit testing and validation of the developed code to ensure reliability and performance. Collaborate with cross-functional teams to support production deployments and resolve any issues that may arise. more »

Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise … Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages … Proficiency in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable. Data engineering concepts: Knowledge of data warehousing, data pipelines, data modeling more »

already hold active DV or SC clearance to be considered for this position. Role Outline: Assist with the continued development and maintenance of data pipelines using NiFi and signature updates using Elastic/Kibana . System administration on specific cyber defence applications and systems to include installation, configuration more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. Implement data management standards, requirements, and specifications. Develop data standards, policies, and procedures. Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team Knowledge: Knowledge of big data more »

Data Analyst - Spotfire, PowerBI, Mendix, RShiny, Java, SQL, Python - 100% Remote A large global organisation are in need of Data Analysts to join the team to assist with a backlog of digital solutions which need to be built. These include dashboards, Low-code applications and simple … data manipulation pipelines. This will be an initial 6 months contract with the opportunity for extensions. The position can be worked 100% remotely. Key responsibilities: Ability to scope, assess and develop dashboards Proven experience with Low-code applications Data Manipulation Qualification of Data Pipelines … Exposure to High-code applications and industrialised solutions Technical Skills required: Spotfire PowerBI Mendix RShiny Java SQL Python R Data Analyst - Spotfire, PowerBI, Mendix, RShiny, Java, SQL, Python - 100% Remote more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. Implement data management standards, requirements, and specifications. Develop data standards, policies, and procedures. Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: Knowledge of big data more »

a Senior Go Lang Developer. Job Responsibilities/Objectives - Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Extensive ETL and Data warehousing experience - Strong experience in GoLang programming - Google Cloud Services and BigQuery Required Skills/Experience - Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application) - Understand and provide inputs for STT documents. - Design and develop the data pipelines more »

a Senior Go Lang Developer. Job Responsibilities/Objectives - Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Extensive ETL and Data warehousing experience - Strong experience in GoLang programming - Google Cloud Services and BigQuery Required Skills/Experience - Assess, analyse, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application ) - Understand and provide inputs for STT documents. - Design and develop the data pipelines more »

will need as much of the following as possible. Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Assess, analyse, data ingestion, transformation & storage layers (RAW, Exploratory, Curated application) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using CICD pipelines more »

days pweek) iR35 determination - Inisde iR35 Duration - 6 months Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud … Key Responsibilities Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application ) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using … CICD pipelines built. Essential Skills Extensive ETL and Data warehousing experience Strong experience in GoLang programming Google Cloud Services and BigQuery more »

Developer - £500 per day - London/remote - 12 months Strong programming skills on GoLang with experience on building APIs and having knowledge on Data warehousing/ETL who can design and develop the data pipelines for Group data consolidation platform built on Google Cloud. … Duties Assess, analyze, data ingestion, transformation & storage layers (RAW, Exploratory, Curated .application) Understand and provide inputs for STT documents. Design and develop the data pipelines using GoLang programming. Able to build APIs to integrate different sources Perform UT and validate the transformations. Deploy code using CICD … pipelines built. Essential Skills: Extensive ETL and Data warehousing experience Strong experience in GoLang programming Google Cloud Services and BigQuery more »

Position: Software Engineer - Data & AWS We are seeking a Software Engineer to build and optimize data platforms leveraging AWS and key technologies like CDAP, Snowflake, and Databricks. You will design and implement robust and scalable data pipelines, ETL, and analytics systems in the cloud. … Responsibilities: Develop and enhance data pipelines, ETL processes using CDAP on AWS infrastructure. Build data integration flows to migrate large datasets into Snowflake data warehouse. Implement AWS infrastructure-as-code solutions for deployment automation. Instrument data pipelines and leverage monitoring for performance … tuning and reliability. Work with data scientists to optimize data workflows and models on Databricks. Follow security best practices for access control, encryption, and auditing across data platforms. Participate in architecture reviews and technology selections. Continuously monitor and improve data platforms for more »

Dive into a pivotal role leading data management and analytics transformation. We're on the hunt for an SC Cleared Azure Data Lead with a knack for innovation and a comprehensive skill set: MUST HAVE ACTIVE SC CLEARANCE Key Responsibilities Oversee the evolution of the data platform, encompassing data pipelines, storage, and processing. Establish and nurture robust relationships with stakeholders to guarantee a positive user experience and to identify any potential opportunities for additional services or products. Play a key role in the formulation and execution of the data management … record in the development and management of an Azure cloud-based data & analytics platform. Comprehensive experience in the complete data pipeline development process, encompassing data warehousing, data analytics, and ETL processes. Profound understanding of best practices in data governance more »

Principal Data Engineer Job Type: Full-Time Sector: PropTech Location: Preston, Lancashire - Predominantly Remote!! (average expectation of one site visit per month) Salary: £85,000.00+ Standard benefits + Annual Bonus My well established PropTech client is looking for an experienced Principal Data Engineer to join their … team and be instrumental in transforming the way that they consume data and transitioning from On Premise to Google Cloud. This is an exciting time to join a growing business function and gain hands on experience working with market leading technology such as BIG Query, Looker, Data Flow, and more. The Data Team: The Data and Platform Engineering team are the foundation for the Data Office function. Responsible for designing, building, and maintaining their data platform, they extract data from source, transform it into a more »