16 of 16 Contract Apache Spark Jobs in the UK excluding London

prem data platform aligned to the metals strategy while developing and maintaining scalable data pipelines and analytics infrastructure. Using Hadoop, Big Data, and Spark technologies, you will ensure data quality through automated validation, monitoring, and testing. You will also enable seamless integration across data warehouses and data lakes, contributing … detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such ...

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g. ...

requirements Excellent communication, problem-solving and documentation skills Desirable: Experience with cloud data platforms (AWS, Azure or GCP) Exposure to big data technologies (Hadoop, Spark, Kafka) Familiarity with ETL/orchestration tools (e.g. Informatica, Talend, Airflow) Experience with API integration and microservices data flows Familiarity with AI/ ...

DataOps practices, including CI/CD, containerisation, and workflow orchestration Hands-on experience with ETL/ELT frameworks and big data tools (e.g. Spark, Airflow, Hive) Proficiency in programming languages such as Python, Java, and SQL Experience with SQL and NoSQL database design and optimisation Strong understanding of batch ...

/or Computer Vision. Strong knowledge of Machine Learning algorithms and statistics. Experience with SQL/NoSQL databases and big data tools (Spark, Hadoop preferred). Experience with MLOps tools such as Docker, Kubernetes, CI/CD pipelines. Preferred Skills Experience with LLMs/Generative AI (OpenAI, Hugging Face ...

frameworks o CI/CD pipelines, DevOps, Git-based workflows o Data lineage, metadata and data governance tooling o Big data ecosystems (Hadoop/Spark) ? Understanding of how data flows across customer onboarding, KYC, payments, lending, and fraud platforms. Domain/Banking Knowledge ? Experience delivering within a Customer Data ...

ServiceNow Automation, Automation Anywhere, Camunda, or similar). Proven experience in AI/ML, data modeling, data lakes, event streaming, and analytics (e.g., Kafka, Spark, Python ML stack). Deep exposure to telecom protocols and domains Experience with cloud-native architectures (Azure/AWS/GCP), containerization, and DevSecOps ...

PostgreSQL, MySQL) Strong knowledge of data warehouse, data lake, and lakehouse architectures Experience with ETL/ELT frameworks and orchestration tools (e.g. Informatica, Talend, Apache Airflow) Proven experience establishing and maintaining data governance frameworks Strong understanding of data security, access controls, and compliance requirements Excellent problem-solving, communication …/ML data pipelines and analytics platforms Preferred Qualifications Experience with cloud data platforms (AWS, Azure, GCP) Experience with big data technologies (Hadoop, Spark, Kafka) Familiarity with UML/SysML Strong understanding of API integration and microservices data flows ...

Oracle, SQL Server, PostgreSQL, MySQL). Knowledge of data warehouse and lakehouse architectures. Familiarity with ETL/ELT and orchestration tools (e.g., Informatica, Talend, Apache Airflow). Experience establishing and maintaining data governance frameworks. Experience complying with data security requirements. Experience designing data models. Excellent problem-solving, communication … pipelines and analytics platforms. Preferred Qualifications: Experience with cloud data platforms (e.g., AWS, Azure, Google Cloud). Experience with big data technologies (e.g., Hadoop, Spark, Kafka). Experience with UML/SYSML. Strong understanding of API integration and microservices data flow. If you have a strong background in data ...

architectures using Delta Lake principles Ingest, transform, and curate data from multiple sources (APIs, databases, files, streaming) Develop scalable data transformations using PySpark and Spark SQL Implement data models optimized for analytics and reporting (e.g. star schemas) Monitor, troubleshoot, and optimize performance and cost of data workloads Apply data … version control (Git) and collaborative development workflows Nice to Have Experience with Power BI and semantic models Exposure to streaming technologies (Kafka, Event Hubs, Spark Structured Streaming) Infrastructure-as-code experience (Bicep, Terraform) CI/CD tooling (Azure DevOps, GitHub Actions) Familiarity with data governance and cataloging tools Experience ...

Oracle, SQL Server, PostgreSQL, MySQL Solid understanding of data warehouse and lakehouse architectures Experience with ETL/ELT and orchestration tools (eg Informatica, Talend, Apache Airflow) Proven experience designing data models Strong understanding of data governance and data security requirements Excellent communication, documentation, and problem-solving skills Exposure …/ML data pipelines and analytics platforms Desirable Experience Cloud data platforms ( AWS, Azure, Google Cloud ) Big data technologies ( Hadoop, Spark, Kafka ) UML/SysML API integration and microservices data flows ...

Requirements: Technical expertise in designing, building, and maintaining data pipelines and warehouses Experience with ETL/ELT frameworks and Big Data Processing Tools (e.g., Spark, Airflow, Hive) Knowledge of programming languages (e.g., Python, SQL) Hands-on experience with SQL/NoSQL database design Degree in a STEM field ...

Requirements: Technical expertise in designing, building, and maintaining data pipelines and warehouses Experience with ETL/ELT frameworks and Big Data Processing Tools (e.g., Spark, Airflow, Hive) Knowledge of programming languages (e.g., Python, SQL) Hands-on experience with SQL/NoSQL database design Degree in a STEM field ...

Requirements: Technical expertise in designing, building, and maintaining data pipelines and warehouses Experience with ETL/ELT frameworks and Big Data Processing Tools (e.g., Spark, Airflow, Hive) Knowledge of programming languages (e.g., Python, SQL) Hands-on experience with SQL/NoSQL database design Degree in a STEM field ...

Join our client in embarking on an ambitious data transformation journey using Databricks, guided by best practice data governance and architectural principles. To support this, we are recruiting for talented data engineers. As a major ...

scale Hadoop clusters across on-prem environments Supporting and optimising our HDFS-based Data Lake and adjacent platform components Working with technologies including Hadoop, Spark, HDFS, YARN, Hive, Oozie , and related tooling Partnering with engineering teams to deliver scalable, high-quality data solutions Contributing to our cloud migration strategy … objectives Ensuring stability, performance, and security of critical data services What We're Looking For Strong hands-on experience with Hadoop ecosystem tools (HDFS, Spark, Hive, Oozie, YARN, etc.) Experience operating and supporting large, complex, distributed clusters Understanding of data engineering principles and data platform design Experience with ...