1 to 25 of 132 Apache Spark Jobs in London

Data Platform Engineer - ADF, Terraform, Synapse, Spark £Market Rate (Inside IR35) London/Hybrid 6 months We are currently working with a client who urgently require a Data Platform Engineer with hands on expertise in Synapse Spark Architecture, ADF and Terraform (IaC), to assist in designing and delivering … Data Platform Engineer Advanced knowledge of Azure Data Factory (ADF), including network configurations and integration runtime options. Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts. Experience in efficient handling of large-scale data Deep understanding of Azure DevOps pipelines and deploying Terraform (Infrastructure ...

warehousing, and data governance frameworks. Define and enforce best practices, security standards, and performance optimization across Databricks workloads. Design scalable big data architectures leveraging Spark, Delta Live Tables, Unity Catalog, and MLflow. Lead cloud architecture design on Azure/AWS/GCP (based on organization need) integrated with Databricks. … Qualifications 10–15 years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks. Expert in Apache Spark, PySpark, Databricks SQL, Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture, BI ecosystems, and modern data platforms. Cloud expertise ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Excellent communication and documentation skills. · Ability to work independently and lead cross-functional teams. · Adaptability to rapidly evolving technological environments PREFERRED SKILLS · Airflow, dbt, Spark, Kafka, RabbitMQ, Redis. · Git, CI/CD pipelines. · Experience with data warehousing solutions such as Snowflake, Redshift, BigQuery, or Synapse. · Exposure to AI/ ...

Java Commercial experience in client-facing projects is a plus, especially within multi-disciplinary teams Deep knowledge of database technologies: Distributed systems (e.g., Spark, Hadoop, EMR) RDBMS (e.g., SQL Server, Oracle, PostgreSQL, MySQL) NoSQL (e.g., MongoDB, Cassandra, DynamoDB, Neo4j) Solid understanding of software engineering best practices - code reviews, testing ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

queries, data modeling, and transformation logic Strong experience with data pipelines and orchestration tools Hands-on experience with big data ecosystems (e.g., Snowflake, Databricks, Spark, AWS/GCP/Azure data services) Advanced skills with Tableau, Power BI, or Looker, including data source optimization and dashboard automation Proficiency ...

queries, data modeling, and transformation logic Strong experience with data pipelines and orchestration tools Hands-on experience with big data ecosystems (e.g., Snowflake, Databricks, Spark, AWS/GCP/Azure data services) Advanced skills with Tableau, Power BI, or Looker, including data source optimization and dashboard automation Proficiency ...

cloud environments (AWS, Azure, GCP) including services like S3, BigQuery, Azure ML, etc. Solid understanding of data engineering principles, SQL, and distributed computing frameworks (Spark, Hadoop). Experience in consulting or client-facing roles, with ability to manage stakeholders and deliver strategic insights. Strong knowledge of machine learning algorithms ...

cloud platforms (Azure, AWS, GCP) and data migration tools (Azure Data Factory, Informatica, Talend, etc.). Proficiency in SQL , Python , and big data technologies (Spark, Hadoop). Knowledge of data governance , security , and compliance frameworks . Excellent communication and stakeholder management skills. Preferred Qualifications Experience with modern data platforms ...

cloud platforms (Azure, AWS, GCP) and data migration tools (Azure Data Factory, Informatica, Talend, etc.). Proficiency in SQL , Python , and big data technologies (Spark, Hadoop). Knowledge of data governance , security , and compliance frameworks . Excellent communication and stakeholder management skills. Preferred Qualifications Experience with modern data platforms ...

Data Software Engineer/Developer (Python Spark Azure) London/WFH to £100k Are you a data centric Software Engineer with strong Python coding skills? You could be progressing your career in a senior, hands-on Data Software Engineer role at a scaling, global technical services company as they … collaboratively problem solving as part of an Agile development using a range of technology to create data pipelines with a focus on Python and Spark; you'll be working with Azure, ETL pipelines and CI/CD, ingesting and analysing terabytes of data with varying structures from a range ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create ...

Knowledge of machine learning frameworks (e.g., PySpark, PyTorch ) and model deployment tools (e.g., MLflow, TensorFlow Serving). Strong experience with data processing frameworks (e.g., Apache Spark, Flink). Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). Hands-on experience with cloud platforms (e.g. ...

Knowledge of machine learning frameworks (e.g., PySpark, PyTorch ) and model deployment tools (e.g., MLflow, TensorFlow Serving). Strong experience with data processing frameworks (e.g., Apache Spark, Flink). Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). Hands-on experience with cloud platforms (e.g. ...

proficiency in programming languages such as Python, Java, or Scala. Experience with cloud platforms like Azure. Knowledge of big data technologies such as Hadoop, Spark, or Kafka. Proficiency in SQL and database management systems. Familiarity with data warehousing concepts and tools. Ability to work collaboratively with cross-functional teams. ...

Quantexa delivery experience Strong knowledge of Entity Resolution, Scoring Framework, Network Generation, and UI configuration 5 -10 years' experience with Big Data technologies (Scala, Spark, Hadoop, Elasticsearch) Solid data modelling and analytics platform design experience Excellent stakeholder communication skills Experience with cloud platforms (AWS, Azure, or GCP) Desirable Experience ...

evaluation techniques. Cloud & MLOps: Experience with AWS, GCP, or Azure; Docker/Kubernetes; MLflow or similar tools. Data Handling: Familiarity with big data frameworks (Spark, Hadoop) and data versioning tools (DVC). Soft Skills: Strong problem-solving, communication, and ability to work in agile teams. Preferred Qualifications Experience with ...

lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Excellent Python and SQL skills Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment ...

commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

technology (AWS or Azure). Strong experience with creation of data ingestion pipeline and ET process. Strong knowledge of big data tool such as Spark, Databricks and Python. Strong understanding of common machine learning techniques and frameworks (e.g. mlflow). Strong knowledge of Natural language processing (NPL) concepts. Strong ...