1 to 25 of 38 Apache Jobs in London

experience in Provide file-based storage: Provide NFS exports and SMB file shares to specific systems and users. Hands-on experience in Apache HTTPD web service: Configure Apache HTTPD to provide Transport Layer Security (TLS)-enabled websites and virtual hosts. Hands on experience in Execute shell scripting ...

warehousing concepts Familiarity with version control systems, particularly Git Desirable Skills: Experience with infrastructure as code tools such as Terraform or CloudFormation Exposure to Apache Spark for distributed data processing Familiarity with workflow orchestration tools such as Airflow or AWS Step Functions Understanding of containerisation using Docker Experience with ...

documented data products * Nice to Have * Financial markets data - market data feeds (Bloomberg, Refinitiv), tick data, trade history, reference data, or instrument master management * Apache Spark or Flink for large-scale stream and batch processing beyond the Databricks ecosystem * dbt or equivalent SQL transformation layer; experience building and maintaining ...

years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

support Workflow orchestration (Airflow or Dagster) AWS data services (Athena, S3, MWAA, Iceberg) Data quality monitoring & incident management Nice to Have: Airbyte DBT Apache Spark Grafana FTP/data distribution platforms Financial data experience ...

years experience across BI, data warehousing, data management, ETL and database technologies Strong hands-on experience building and optimising data pipelines using Python, SQL, Apache Spark, and Microsoft SSIS Proven ability to design, deliver and maintain scalable data solutions across data warehouses and Big Data Lakes, supporting both ...

years’ experience across BI, data warehousing, data management, ETL and database technologies Strong hands-on experience building and optimising data pipelines using Python, SQL, Apache Spark, and Microsoft SSIS Proven ability to design, deliver and maintain scalable data solutions across data warehouses and Big Data Lakes, supporting both ...

explain technical concepts to varied audiences. Nice to Have Cloud experience: AWS/Azure/GCP . Containerisation: Docker, Kubernetes . Distributed processing: Apache Spark/Flink . Please apply within for further details - Matthew Holmes, Harvey Nash Always use these settings ...

. Hands-on experience with ETL pipeline design and optimization for complex data sets is a strong advantage. Deep familiarity with technologies such as Apache Beam, Pub/Sub, Redis , and other large-scale data processing frameworks. Expertise in backend development with Python and Scala ; knowledge of Node.js ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

will have strong experience in data engineering, with advanced proficiency in SQL and Python, alongside exposure to large-scale data processing frameworks such as Apache Spark. You should have a solid understanding of data architecture, data wrangling and data quality practices, as well as experience working within financial markets ...

security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement Why Apply ...

Kubernetes Strong Git-based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities You'll be hands ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

mindset Support identity, RBAC and governance controls across Azure services Tech stack Azure Azure Databricks Terraform Azure DevOps/CI/CD Python SQL Apache Spark/PySpark Entra ID/RBAC/governance Networking/platform security/automation What were looking for Strong experience in Azure platform ...

preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

general security protocols within a UNIX environment to ensure compliant and hardened architectures. Performance Optimization: Implement and tune high-performance web stacks using Apache, Varnish for caching, and HAProxy for load balancing. Team Leadership: Manage and technically govern a team of 5+ infrastructure engineers, providing guidance on root-cause ...

Kubernetes Strong Git-based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities You'll be hands ...

Work with solution architects to design migration-day data processes for partner onboarding. Build and maintain ETL pipelines and workflow automation using Snowflake and Apache Airflow. Document processes, learnings, and development work using version control best practices. Implement robust data quality checks and reconciliation processes throughout pipelines. Collaborate with … functionally. Strong communication skills, with the ability to translate technical concepts to non-technical audiences. Hands-on experience building and managing production workflows in Apache Airflow. Experience delivering production-grade transformations in dbt, including testing and documentation. Advanced SQL skills across multiple dialects. Strong understanding of data modelling ...

Work with solution architects to design migration-day data processes for partner onboarding. Build and maintain ETL pipelines and workflow automation using Snowflake and Apache Airflow. Document processes, learnings, and development work using version control best practices. Implement robust data quality checks and reconciliation processes throughout pipelines. Collaborate with … functionally. Strong communication skills, with the ability to translate technical concepts to non-technical audiences. Hands-on experience building and managing production workflows in Apache Airflow. Experience delivering production-grade transformations in dbt, including testing and documentation. Advanced SQL skills across multiple dialects. Strong understanding of data modelling ...

Desirable Skills and Experience Experience working with cloud platforms, ideally AWS. A strong Linux background. Experience with data integration and messaging technologies such as Apache NiFi, Apache Kafka, RabbitMQ, or similar tools. Experience using Infrastructure as Code tools such as Terraform or CloudFormation. Previous experience working ...

analysis on complex system-wide issues. Linux Engineering: Manage and optimise a large-scale estate across RedHat, Ubuntu, and CentOS, ensuring peak performance of Apache, Nginx, and Tomcat stacks. Automation & IaC: Drive efficiency through the development of custom Ansible, Puppet, and Terraform scripts to automate deployments and server hardening. ...

Platforms Itential Automation Platform Cisco NSO – Service development and lifecycle Apache Airflow Hasi Corp Vault Openshift Automation development Python Fast API Rest API Netconf Yang modelling XML/Json Jinja/TextFSM Pytest Unit testing Database MongoDB DevOps Docker Git CI/CD Kubernetes Networking Ciena 517x/ ...

with EKS) Proficiency in Git-based development workflows Experience working independently and communicating effectively in distributed teams Desirable : Familiarity with (or happy to learn): Apache Spark (EMR) and Delta Lake PostgreSQL (Aurora) CI/CD (Github Actions) and IaC (Helm, CloudFormation) Monitoring (Sentry) Knowledge/interest in research ...