1 to 25 of 29 Apache Jobs in London

working in a cloud environment such as Google Cloud Platform or AWS Hands on programming experience of the following (or similar) technologies: Kubernetes, Docker Apache Beam, Apache Flink, Apache Spark Google BigQuery, Snowflake Google BigTable Google Pub/Sub, Kafka Apache Airflow Experience implementing observability around ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI ...

/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA, MES, and Historian systems. Information Modeling ...

strong plus Technical Skills: • Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL) • Familiarity with data pipeline and workflow management tools (e.g., Apache Airflow) • Experience with programming languages such as Python, Java, or Scala. Python is highly preferred • Basic understanding of cloud platforms and services (e.g. ...

management principles, including modelling, integration, and governance. Familiarity with modern data architectures (cloud-based, distributed systems). Proficiency in SQL, Python, and pipeline tools (Apache Airflow, Spark). Experience with cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness and evidence of data analytics and visualisation. Essentials Sole ...

NICE FOR THE SENIOR SOFTWARE ENGINEER TO HAVE. Cloud based experience Microservice architecture or server-less architecture Big Data/Messaging technologies such as Apache Nifi/MiNiFi/Kafka TO BE CONSIDERED. Please either apply by clicking online or emailing me directly to For further information please call ...

using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort keys ...

Certified Professional. Working knowledge of MLOps. Extensive experience in a consulting environment i.e. strong stakeholder engagement capability. Strong programming proficiency in Python, SQL, and Apache Spark, with a proven ability to design, optimise, and scale high-performance data pipelines for AI and analytics applications. Deep understanding of cloud-native ...

Code tools Experience with NoSQL databases (e.g., MongoDB) or SQL (e.g., Postgres) Understanding of web standards, accessibility and development best practice Experience with Apache NiFi Bonus Skills Microservices in C# Integrating LLMs (e.g., LangChain, Vercel AI) Experience using the GOV.UK Design System and Nunjucks What You'll Love ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across security ...

maintain highly reliable and well-tested data and ML pipelines to extract, transform, and structure large datasets for ML applications. Expertise in using Apache Airflow (or similar orchestration tools like Prefect/Dagster) to define, schedule, and monitor complex data and ML workflows (DAGs). Implement comprehensive software quality ...

maintain highly reliable and well-tested data and ML pipelines to extract, transform, and structure large datasets for ML applications. Expertise in using Apache Airflow (or similar orchestration tools like Prefect/Dagster) to define, schedule, and monitor complex data and ML workflows (DAGs). Implement comprehensive software quality ...

industry. Extensive, demonstrable experience reading and writing complex SQL queries for data collection and analysis including using visualization platforms such as Tableau, Power BI, Apache Superset or similar (essential). Scripting experience using Python (web scrapping), dbt and bash (essential). Experience using or ingesting data from Splunk, Snowflake ...

cloud data warehouses (e.g., Snowflake, Redshift). Familiarity with data orchestration tools (e.g., Airflow). Experience with data visualisation platforms (such as Preset.io/Apache Superset or other). Exposure to CI/CD pipelines, ideally using GitLab CI. Background working with media, marketing, or advertising data. The Opportunity ...

cloud data warehouses (e.g., Snowflake, Redshift). ·Familiarity with data orchestration tools (e.g., Airflow). ·Experience with data visualisation platforms (such as Preset.io/Apache Superset or other). ·Exposure to CI/CD pipelines, ideally using GitLab CI. ·Background working with media, marketing, or advertising data. The Opportunity ...

Optimise data workflows for performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence to data governance, security ...

cloud technologies Experience with Docker and agile delivery Desirable Skills Terraform or other IaC tools NoSQL (ideally MongoDB) or relational databases (e.g., Postgres) Apache NiFi Web standards, accessibility and development best practice Bonus Experience Microservices in C#/.NET Integrating LLMs with tools like LangChain or Vercel AI Additional ...

projects and delivering secure, scalable cloud solutions. Nice to have: Immediate availability Understanding of Synapse Spark Architecture Familiarity with a variety of technologies, including Apache Spark, Spark ML, Serverless Spark Pools, TSQL, ETL Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment business ...

high-performance, low-latency Java-based systems for front office trading or pricing platforms. Build reactive systems using Kafka Streams , Akka , Eclipse Vert.x , or Apache Flink . Utilize multithreading , concurrency models , and Executor Services to optimize system performance and throughput. Write clean, efficient, and maintainable code using functional programming ...

such as React, Redux, TypeScript, Material UI, Storybook, Jest, and Scala.js Proven working knowledge of Backend & Databases such as Scala (v2), Play Framework, ScalaTest, Apache Pekko (including testkits), Mockito, PostgreSQL, and MongoDB. Familiarity with DevOps & Cloud tools like Docker, Kubernetes, Sysdig, OpenSearch, and AWS. Excellent communication and stakeholder management ...

such as React, Redux, TypeScript, Material UI, Storybook, Jest, and Scala.js Proven working knowledge of Backend & Databases such as Scala (v2), Play Framework, ScalaTest, Apache Pekko (including testkits), Mockito, PostgreSQL, and MongoDB. Familiarity with DevOps & Cloud tools like Docker, Kubernetes, Sysdig, OpenSearch, and AWS. Excellent communication and stakeholder management ...

company's requirements. Design and build data storage solutions to address different use cases. Be technology agnostic and favour open standards (e.g., Parquet, Apache Iceberg). Find the right balance between cost and performance of the proposed solutions. Create scalable ingestion and transformation workflows, covering aspects such as observability … Standards: Hands-on experience with a data lakehouse implementation (either vendor based or open source) which is based open standards such as Parquet and Apache Iceberg. Experience in building data models, partition strategies and performance optimization (e.g. data compaction). Data Transformation & Lineage: Experience in performing data transformation ...

Career Growth: This position offers a growth path to a Sales Manager or Partner Account Manager. As you gain industry expertise and knowledge in Apache CloudStack and open-source and demonstrate strong performance in lead generation and presales activities, you’ll have the opportunity to take on more responsibility ...