1 to 25 of 41 Apache Jobs in London

however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

Solid understanding of data warehousing, lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Excellent Python and SQL skills Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

Azure, or GCP). Proficiency in Python and SQL for data manipulation and transformation. Experience with ETL/ELT development and orchestration tools (e.g., Apache Airflow, dbt, Prefect). Knowledge of data modelling, data warehousing, and lakehouse architectures. Familiarity with DevOps practices, CI/CD pipelines, and infrastructure ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

Code tools Experience with NoSQL databases (e.g., MongoDB) or SQL (e.g., Postgres) Understanding of web standards, accessibility and development best practice Experience with Apache NiFi Bonus Skills Microservices in C# Integrating LLMs (e.g., LangChain, Vercel AI) Experience using the GOV.UK Design System and Nunjucks What You'll Love ...

data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems and distributed systems Hands-on with Lakehouse technologies (Apache Iceberg, S3 Tables, StarRocks) Strong grasp of data governance, schema design, and quality frameworks Comfortable leading infrastructure decisions and collaborating across distributed teams This ...

technologies (NoSQL, time-series databases) and deep competence in data modeling principles. Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes). The Environment We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving in a fast ...

Code tools Experience with NoSQL databases (e.g., MongoDB) or SQL (e.g., Postgres) Understanding of web standards, accessibility and development best practice Experience with Apache NiFi Bonus Skills Microservices in C# Integrating LLMs (e.g., LangChain, Vercel AI) Experience using the GOV.UK Design System and Nunjucks What You'll Love ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data ...

TensorFlow, PyTorch, or Scikit-learn for ML, and Pandas, PySpark, or similar for data processing. Experience designing and orchestrating data pipelines with tools like Apache Airflow, Spark, or Kafka. Strong understanding of SQL, NoSQL, and data modeling. Familiarity with cloud platforms (AWS, Azure, GCP) for deploying ML and data ...

using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort keys ...

applications and high proficiency in SQL (query tuning, complex joins) Hands-on experience designing and deploying ETL/ELT pipelines using Google Cloud Dataflow (Apache Beam) or similar tools Proficiency in data architecture, data modeling, and scalable storage design Solid engineering practices: Git and CI/CD for data ...

years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

NICE FOR THE SENIOR SOFTWARE ENGINEER TO HAVE. Cloud based experience Microservice architecture or server-less architecture Big Data/Messaging technologies such as Apache Nifi/MiNiFi/Kafka TO BE CONSIDERED. Please either apply by clicking online or emailing me directly to For further information please call ...

Technical experience will be with Python, dealing with huge volumes of data at scale across both bathc and stream processes. Orchestration tools such as Apache Airflow, Datawarehouse is Snowflake whilst having microservices experience within a cloud environment, ideally AWS. A solid all rounder who can still roll their sleeves ...

Spring Boot, Hibernate, JEE, MVC, Swing, JSP, Webservices (REST/SOAP), Angular/AngularJS, jQuery, JSON, HTML, CSS, XSLT Expertise with: Tomcat, JBoss EAP, Apache, SQL, Oracle, PL/SQL, XML, Maven, Ant, CI/CD (Azure Pipeline), Jira, Subversion Experience generating reports in PDF, MS Word, Excel . ...

cloud technologies Experience with Docker and agile delivery Desirable Skills Terraform or other IaC tools NoSQL (ideally MongoDB) or relational databases (e.g., Postgres) Apache NiFi Web standards, accessibility and development best practice Bonus Experience Microservices in C#/.NET Integrating LLMs with tools like LangChain or Vercel AI What ...

cloud data warehouses (e.g., Snowflake, Redshift). Familiarity with data orchestration tools (e.g., Airflow). Experience with data visualisation platforms (such as Preset.io/Apache Superset or other). Exposure to CI/CD pipelines, ideally using GitLab CI. Background working with media, marketing, or advertising data. The Opportunity ...

Optimise data workflows for performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence to data governance, security ...

real-world innovation powered by data and technology. Tech Stack & Skills We're Looking For: Palantir Azure Databricks Microsoft Azure Python Docker & Kubernetes Linux Apache Tools Data Pipelines IoT (Internet of Things) Scrum/Agile Methodologies Ideal Candidate: Already DV Cleared or at least SC Strong communication skills – comfortable ...