1 to 25 of 77 Apache Spark Jobs in London

Data Platform Engineer - ADF, Terraform, Synapse, Spark £Market Rate (Inside IR35) London/Hybrid 6 months We are currently working with a client who urgently require a Data Platform Engineer with hands on expertise in Synapse Spark Architecture, ADF and Terraform (IaC), to assist in designing and delivering … Data Platform Engineer Advanced knowledge of Azure Data Factory (ADF), including network configurations and integration runtime options. Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts. Experience in efficient handling of large-scale data Deep understanding of Azure DevOps pipelines and deploying Terraform (Infrastructure ...

Data Platform Engineer - ADF, Terraform, Synapse, Spark £Market Rate (Inside IR35) London/Hybrid 6 months We are currently working with a client who urgently require a Data Platform Engineer with hands on expertise in Synapse Spark Architecture, ADF and Terraform (IaC), to assist in designing and delivering … Data Platform Engineer Advanced knowledge of Azure Data Factory (ADF), including network configurations and integration runtime options. Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts. Experience in efficient handling of large-scale data Deep understanding of Azure DevOps pipelines and deploying Terraform (Infrastructure ...

LLMs), and specialized models in Computer Vision and NLP. Big Data Engineering: Design, build, and optimize high-throughput, distributed data pipelines and features using Apache Spark (Scala) and Hive on massive datasets to support model training and inference. Cross-Cloud Execution: Lead the design and deployment … environment. Required Skills and Expertise (10+ Years) 1. Big Data and Cloud Mastery Programming & Big Data: 10+ years of extensive, hands-on experience with Apache Spark, with strong preference for production development using Scala. Deep expertise with Apache Hive for data querying and management. Cloud Proficiency: Demonstrated ...

LLMs), and specialized models in Computer Vision and NLP. Big Data Engineering: Design, build, and optimize high-throughput, distributed data pipelines and features using Apache Spark (Scala) and Hive on massive datasets to support model training and inference. Cross-Cloud Execution: Lead the design and deployment … environment. Required Skills and Expertise (10+ Years) 1. Big Data and Cloud Mastery Programming & Big Data: 10+ years of extensive, hands-on experience with Apache Spark, with strong preference for production development using Scala. Deep expertise with Apache Hive for data querying and management. Cloud Proficiency: Demonstrated ...

warehousing, and data governance frameworks. Define and enforce best practices, security standards, and performance optimization across Databricks workloads. Design scalable big data architectures leveraging Spark, Delta Live Tables, Unity Catalog , and MLflow. Lead cloud architecture design on Azure/AWS/GCP (based on organization need) integrated with Databricks. … years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

warehousing, and data governance frameworks. Define and enforce best practices, security standards, and performance optimization across Databricks workloads. Design scalable big data architectures leveraging Spark, Delta Live Tables, Unity Catalog , and MLflow. Lead cloud architecture design on Azure/AWS/GCP (based on organization need) integrated with Databricks. … years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Excellent communication and documentation skills. · Ability to work independently and lead cross-functional teams. · Adaptability to rapidly evolving technological environments PREFERRED SKILLS · Airflow, dbt, Spark, Kafka, RabbitMQ, Redis. · Git, CI/CD pipelines. · Experience with data warehousing solutions such as Snowflake, Redshift, BigQuery, or Synapse. · Exposure to AI/ ...

Java Commercial experience in client-facing projects is a plus, especially within multi-disciplinary teams Deep knowledge of database technologies: Distributed systems (e.g., Spark, Hadoop, EMR) RDBMS (e.g., SQL Server, Oracle, PostgreSQL, MySQL) NoSQL (e.g., MongoDB, Cassandra, DynamoDB, Neo4j) Solid understanding of software engineering best practices - code reviews, testing ...

queries, data modeling, and transformation logic Strong experience with data pipelines and orchestration tools Hands-on experience with big data ecosystems (e.g., Snowflake, Databricks, Spark, AWS/GCP/Azure data services) Advanced skills with Tableau, Power BI, or Looker, including data source optimization and dashboard automation Proficiency ...

queries, data modeling, and transformation logic Strong experience with data pipelines and orchestration tools Hands-on experience with big data ecosystems (e.g., Snowflake, Databricks, Spark, AWS/GCP/Azure data services) Advanced skills with Tableau, Power BI, or Looker, including data source optimization and dashboard automation Proficiency ...

cloud environments (AWS, Azure, GCP) including services like S3, BigQuery, Azure ML, etc. Solid understanding of data engineering principles , SQL, and distributed computing frameworks (Spark, Hadoop). Experience in consulting or client-facing roles , with ability to manage stakeholders and deliver strategic insights. Strong knowledge of machine learning algorithms ...

cloud environments (AWS, Azure, GCP) including services like S3, BigQuery, Azure ML, etc. Solid understanding of data engineering principles , SQL, and distributed computing frameworks (Spark, Hadoop). Experience in consulting or client-facing roles , with ability to manage stakeholders and deliver strategic insights. Strong knowledge of machine learning algorithms ...

proficiency in programming languages such as Python, Java, or Scala. Experience with cloud platforms like Azure. Knowledge of big data technologies such as Hadoop, Spark, or Kafka. Proficiency in SQL and database management systems. Familiarity with data warehousing concepts and tools. Ability to work collaboratively with cross-functional teams. ...

Quantexa delivery experience Strong knowledge of Entity Resolution, Scoring Framework, Network Generation, and UI configuration 5 -10 years' experience with Big Data technologies (Scala, Spark, Hadoop, Elasticsearch) Solid data modelling and analytics platform design experience Excellent stakeholder communication skills Experience with cloud platforms (AWS, Azure, or GCP) Desirable Experience ...

proficiency in programming languages such as Python, Java, or Scala. Experience with cloud platforms like Azure. Knowledge of big data technologies such as Hadoop, Spark, or Kafka. Proficiency in SQL and database management systems. Familiarity with data warehousing concepts and tools. Ability to work collaboratively with cross-functional teams. ...

evaluation techniques. Cloud & MLOps: Experience with AWS, GCP, or Azure; Docker/Kubernetes; MLflow or similar tools. Data Handling: Familiarity with big data frameworks (Spark, Hadoop) and data versioning tools (DVC). Soft Skills: Strong problem-solving, communication, and ability to work in agile teams. Preferred Qualifications Experience with ...

evaluation techniques. Cloud & MLOps: Experience with AWS, GCP, or Azure; Docker/Kubernetes; MLflow or similar tools. Data Handling: Familiarity with big data frameworks (Spark, Hadoop) and data versioning tools (DVC). Soft Skills: Strong problem-solving, communication, and ability to work in agile teams. Preferred Qualifications Experience with ...

Solid understanding of data warehousing, lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment ...

lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Excellent Python and SQL skills Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment ...

commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

technology (AWS or Azure). Strong experience with creation of data ingestion pipeline and ET process. Strong knowledge of big data tool such as Spark, Databricks and Python. Strong understanding of common machine learning techniques and frameworks (e.g. mlflow). Strong knowledge of Natural language processing (NPL) concepts. Strong ...

10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. ·Good proficiency in Python and Spark (PySpark) or Scala. ·Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. ·Extensive experience with Azure data services, including Azure ...