1 to 25 of 29 Contract Data Pipeline Jobs

building products that are performant, scalable, and robust to support critical transportation processes. You build web applications that delight our customers, and complex data pipelines to process the data that serves them. We will give you the space to explore new technologies and approaches, and apply … job responsibilities working with AWS technologies such as Lambda, ECS Fargate, API Gateway, RDS, DynamoDB, EMR building customer-facing applications and APIs building data pipelines using Spark + Scala that process Tb of data per day working with customers to understand the business context of new more »

ML systems, CI/CD and data pipelines. Participate in enterprise-wide architecture initiatives for AI/ML. Understand the workflow and pipeline architectures of ML and deep learning workloads. Conduct security risk assessments and threat modeling for AI/ML and other business projects performed thorough … test specification, development, and execution. Define and enforce robust access controls, authentication mechanisms, and encryption standards to safeguard AI/ML systems, training data, provenance, and any sensitive data. Stay abreast of emerging AI security threats, trends, and technologies, evaluating their impact and proactively adapting security measures to … tools, technologies, and frameworks to enhance the overall security posture of AI/ML and other business systems. Develop and maintain relationships between data scientists, data engineers, developers, operations (DevOps, DataOps, MLOps) and business unit leaders to govern and support scalability of the ML and AI more »

consultancy that are recruiting SECURITY ARCHITECTS to work on a very exciting LONG-TERM CONTRACT within the financial service sector. Role: Security Architect (Data Solutions) Rate: Up to £500 per day (inside IR35) Location: Swindon/London Hybrid (2 days onsite) Duration: 6 months (initially view to extend … integrations team, you will play a crucial role in ensuring the security and integrity of systems related to service mesh, container platforms, and data solutions. Your responsibilities will involve designing, implementing, and maintaining robust security measures to protect sensitive data, prevent unauthorized access, and mitigate risks. … Key responsibilities: Collaborate with development and operations teams to design secure and scalable architectures for containerized applications, data mesh, and Kafka. Assess and implement security controls for container platforms (e.g., Kubernetes, Docker). Define and enforce container image security practices, including vulnerability scanning, image signing, and runtime security. more »

office on a hybrid basis. As a Contract Senior Golang Developer, you will play a crucial role in assessing, analysing, and enhancing our data ingestion, transformation, and storage layers. Your primary focus will be on Go Programming and DBT, with secondary skills in Google Cloud Services. The successful … candidate will bring 5 to 10 years of experience in developing robust data pipelines, conducting unit testing, and providing support for production deployments. Key Responsibilities: Assess and analyse data ingestion, transformation, and storage layers, including RAW, Exploratory, Curated, and application layers. Understand existing codebase and make … alterations or fixes as per design requirements. Develop and optimize data ingestion pipelines using Go Programming and DBT. Conduct unit testing and validation of the developed code to ensure reliability and performance. Collaborate with cross-functional teams to support production deployments and resolve any issues that may arise. more »

with company policies and standards Developing custom logic to prioritize vulnerabilities and compliance issues Integrating compliance tool output into the company's big data platform for compliance purposes Executing queries to extract data for reporting and alerting Performing application patching for the company's vulnerability and …/knowledge: Experience with InsightVM and/or Azure Defender suite Proficient in Python with experience in PowerShell, integration work, and processing structured data such as JSON and CSV for data importing and exporting Strong background in cybersecurity, with a focus on vulnerability and compliance management … Experienced with data pipelines, workflows, and modern development practices including version control and CI/CD pipelines Ability to understand user requirements and implement them effectively within an existing tech stack Knowledge of integration and processing of structured data for seamless data platform operations more »

of our Kafka ecosystem, ensuring its scalability, reliability, and performance to meet the evolving needs of our organization. You will collaborate closely with data engineers, software developers, and other stakeholders to architect robust solutions and streamline data pipelines. Your key responsibilities will include: Installing, configuring, and … performance metrics, and throughput, and proactively identifying and addressing potential bottlenecks or issues. Implementing security measures, access controls, and encryption protocols to safeguard data privacy and integrity within the Kafka ecosystem. Managing Kafka topics, partitions, replication, and consumer groups, and optimizing configurations for efficient resource utilization and high … availability. Collaborating with cross-functional teams to design and implement data ingestion pipelines, Real Time processing workflows, and event-driven architectures. Performing capacity planning, scaling, and disaster recovery planning to ensure scalability, fault tolerance, and business continuity. Automating routine tasks, such as cluster provisioning, deployment, monitoring, and alerting more »

a crucial role in managing and maintaining our Hadoop ecosystem to ensure optimal performance, reliability, and security. You will collaborate closely with our data engineers, analysts, and other stakeholders to understand their requirements and provide efficient solutions. Your responsibilities will include but are not limited to: Installing, configuring … health and performance, diagnosing and troubleshooting issues, and implementing solutions to minimize downtime. Capacity planning and scaling the Hadoop infrastructure to accommodate growing data volumes and user demands. Implementing and enforcing security policies, access controls, and data governance measures to protect sensitive information. Performing regular backups … disaster recovery planning, and data retention policies to ensure data integrity and availability. Collaborating with cross-functional teams to design and implement data pipelines, ETL processes, and data workflows. Evaluating new technologies and upgrades, conducting performance tuning, and optimizing resource utilization for more »

Azure Data Engineer - Contract - Inside IR35 This is an Azure Data Engineer position for a financial services end client helping them on delivering and implementing new data pipelines. You will need prior financial services experience as well as having strong Azure Data Factory and Databricks experience. Work Involved: Implementing and delivering data pipelines Providing mentorship to junior members of the team Working with the Platform team defining capabilities for the lake … framework Developing business conceptual data models Producing new frameworks using design patterns and data ingestion Requirements: Strong Data Pipeline experience Databricks Azure Data Factory Spark Azure cloud Financial Services If you're interested get in touch ASAP more »

complex containerised AWS/Azure architecture. Develop and support Infrastructure as Code using Terraform and develop patterns for wider use. Develop and support data pipelines from AWS to Azure, and work on improvements to existing service. Develop, support, and improve Azure build pipelines. Expert in building and running … containerised microservices (e.g Docker) Experience in non-relational databases (e.g., MongoDB) Operationally support our agile product development and data analytic teams Maintain secure and stable application & data platforms Build strong relationships with team members. Work to meet project deadlines and manage time effectively. Work independently to … more challenging issues. Competency with the following technologies: Azure, AWS, Terraform, Ansible, Java, Node.js, MongoDB, Docker, JSON, JavaScript, SQL, GIT - branching strategy, and release pipeline creation/maintenance. Behaviours: Collaborates with others when necessary to review specifications and uses these agreed standards and tools to design, code, test, correct more »

Tasks - This Cyber Platform project aims at creating a centralized platform for risk assessment and premium calculation. - The platform will utilize pre-calculated data features and criticality indicators to provide more accurate and efficient risk assessments, reducing response times and improving the overall customer experience. - Implement a kind … CDC or an easy-to-implement alternative to prepare incremental updates to a) a SSAS model and b) a DWH model - Support regarding data pipelines - Create data migration model Interested? We look forward to receiving your application documents in electronic form. more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: * Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. * Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. * Implement data management standards, requirements, and specifications. * Develop data standards, policies, and procedures. * Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. * To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: * Knowledge of big data more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: * Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. * Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. * Implement data management standards, requirements, and specifications. * Develop data standards, policies, and procedures. * Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. * To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: * Knowledge of big data more »

Our FTSE100 client needs a Data Engineer with SQL , SSIS and Snowflake experience to join the team to assist in the design and implemententation of scalable data pipelines . You will have expertise in database management , data modelling and ETL including Microsoft SQL and more »

Azure Data Engineer- Birmingham - £350pd Outside I'm currently looking for a Data Engineer to join a company that is based in Birmingham. They need someone to start in the next few weeks. Location: Birmingham (on site 2 times a week) Length: 6 months Day Rate … per day. IR35 Status: Outside of IR35 Required experience will include: Expertise in designing and implementing data pipelines using Azure services such as Azure Data Factory and Azure Databricks. Hands-on experience with SQL database design Experience of product lifecycle management principles and tools (e.g. DevOps … terraform ) and relational database manipulation and interrogation. If you are interested in this Data Engineer role please apply with your most recent CV. Alternatively email me on George . batson @ Randstad . co . uk Azure Data Engineer- Birmingham - £350pd Outside Randstad Technologies is acting as more »

and Compliance Engineer who will assist the SEA team with improving and maintaining the Bank's vulnerability and compliance tooling. Transforming and presenting data within the Bank's big data platform to the Governance and Assurance Team (and others). Developing custom compliance checks where they … in both Python and PowerShell, understands modern development practices such as version control and CI/CD pipelines, and has a background in data flows (eg ETL). They will be passionate about security vulnerability management and compliance scanning, and passionate about turning data into actionable … custom logic to prioritise vulnerabilities and compliance issues. Integrate tool output (eg from a compliance tool or system) into the Bank's big data platform, used for compliance. Implement queries (SQL or similar) to extract relevant data for reporting and alerting. Perform application patching of the more »

We are partnered with a reputable global consultancy that are recruiting AWS DATA ENGINEERS to work on a very exciting LONG-TERM CONTRACT within the financial service sector. Role: AWS Data Engineer Rate: Up to £425 per day (inside IR35) Location: London Hybrid Duration: 6 months … backlog grooming. Key skills: Extensive experience in developing Bigdata pipelines in cloud using Bigdata technologies such as Apache Spark Expertise in performing complex data transformation using Spark SQL queries Experience in orchestrating data pipelines using Apache Airflow Proficiency in Git based version control tools Proficiency with … Linux commands and Bash Scripting Working experience in AWS Bigdata services (EMR, Glue, Data Pipelines, Athena, S3, Step Functions etc.) & AWS CLI Experience with CI/CD tools such as Jenkins Experience working with relational and/or MPP databases Knowledge of Data warehousing concepts Experience more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … Utilise expertise in data management, analytics and stakeholder engagement and management to facilitate and secure datasets and/or data pipeline partnerships across the NHS. Actively work with project managers, functions and workstreams to define detailed and problem centric data requirements for delivery more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … Utilise expertise in data management, analytics and stakeholder engagement and management to facilitate and secure datasets and/or data pipeline partnerships across the NHS. Actively work with project managers, functions and workstreams to define detailed and problem centric data requirements for delivery more »

About the Role: I am looking for a talented Data Engineer to join our clients dynamic team on a 6-month contract basis. This is an exciting opportunity for a mid-level professional with 3-5 years of experience to make a significant impact on our data …/hybrid working options and a competitive day rate of £250-£400, falling inside IR35 regulations. Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes using AWS, Databricks, Python, Spark, and SQL. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions. Optimize and troubleshoot data workflows to ensure efficiency and reliability. Implement data quality and validation checks to ensure data integrity. Contribute to the development and maintenance of our data lake and more »

Required: Data Architect Location: Remote Duration: 6 months + IR35 Status: Inside IR35 As a minimum you will have existing or prior experience in the following areas: * A strong background in Data Engineering (perhaps as a prior DBA or Data Engineer); * A high level … of exposure and hands-on familiarity with database technology; * A solid understanding of Cloud Infrastructure; * Experience of Linux platforms; * Knowledge and experience of Data Warehousing; * Strong Data Analysis skills; * Data Platform Modernisation; * Data Migration; * Data Security (at-rest/in … transit); * Architecting Data Pipelines; * Data Modelling and Database Design; * Metadata Management; * Stakeholder Management & Data Governance; * Excellent communication skills; * Willingness to learn and expand your Data and architectural skills. * Pre-sales content contribution and pricing for new business pursuits. Highly Desirable skills: * Cloud more »

with company policies and standards Developing custom logic to prioritize vulnerabilities and compliance issues Integrating compliance tool output into the company's big data platform for compliance purposes Executing queries to extract data for reporting and alerting Performing application patching for the company's vulnerability and … compliance applications Your experience/knowledge: Proficient in Python with experience in PowerShell, integration work, and processing structured data such as JSON and CSV for data importing and exporting Strong background in cybersecurity, with a focus on vulnerability and compliance management Experienced with data … CD pipelines Ability to understand user requirements and implement them effectively within an existing tech stack Knowledge of integration and processing of structured data for seamless data platform operations Familiarity with tools such as Dataiku DSS, Tableau, and the Azure security ecosystem, especially Defender suite, is more »

and annual plan for delivering this is available on our website. Our new team are embarking on a piece of work to monitor data to identify and respond to patterns indicative of potential fraud. This will support our current work that reduces the likelihood of fraud occurring. We … will bring in data science capabilities to be deployed in counter fraud activity and work closely with partners across health and government to further maximise the preventative impact of proactive counter fraud analysis. We will combine this with our range of counter fraud and domain expertise to maximise … Utilise expertise in data management, analytics and stakeholder engagement and management to facilitate and secure datasets and/or data pipeline partnerships across the NHS. Actively work with project managers, functions and workstreams to define detailed and problem centric data requirements for delivery more »

Data Scientist - Lead Generation & Prospecting Contract Role - Inside IR35 via umbrella Our client, a London based Financial Services/Payements Solutions provider is seeking a Data Scientist to join the team on contract with experience within new business/prospecting - Please only aply if you have … demonstrable experience in these areas. Responsibilities: Utilise advanced data analytics and machine learning techniques to identify high-value prospects and generate leads in the payments industry. Design and implement propensity models to predict potential client engagement and conversion rates, optimising our prospecting strategies. Analyse transactional and financial data … or Master's degree in Statistics, Mathematics, Computer Science, Finance, or a related field. Experience working with Salesforce from a data/pipeline management perspective along with Snowflake Data Connectors a plus. If you are a results-oriented Data Scientist with a passion more »

Job Title: Senior Data Engineer (SAS) Responsibilities: Rewrite existing data pipelines (SAS, Info, pearl, shell scripts) to Python and DBT transformation-based solutions. Convert SAS-based modules to Python-based pipelines. Load transformed data to Snowflake on Azure. Requirements: Strong experience with SAS programming … primary skill). Proficiency in pearl and shell scripting (secondary skills). Expertise in Python and DBT for data transformation. Experience in converting SAS-based modules to Python-based solutions. Familiarity with Snowflake for data management. Experience with Airflow or similar technologies is a plus. Desired more »

ML systems, CI/CD and data pipelines. Participate in enterprise-wide architecture initiatives for AI/ML. Understand the workflow and pipeline architectures of ML and deep learning workloads. Conduct security risk assessments and threat modeling for AI/ML and other business projects performed thorough … models, LLM and development of trustworthiness, safety, fairness, bias and inappropriateness detection, as well as adversarial test specification, development, and execution. Collaborate with data scientists, engineers, and other stakeholders to embed security controls and measures into AI/ML applications, models, algorithms, and data pipelines. Define … and enforce robust access controls, authentication mechanisms, and encryption standards to safeguard AI/ML systems, training data, provenance, and any sensitive data. Stay abreast of emerging AI security threats, trends, and technologies, evaluating their impact and proactively adapting security measures to mitigate potential risks. Provide technical guidance more »