1 to 25 of 53 Apache Jobs in the UK excluding London

strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. Essential qualifications • Extensive experience with Python and PySpark/Apache Spark. • Proven track record designing and delivering scalable data pipelines and data products. • Strong SQL and data modelling skills. • Experience with modern data architectures ...

analytics and reporting, and contributing to the adoption of AI-powered workflows and automation. Key Responsibilities & Skills: Build and maintain scalable data pipelines using Apache Airflow or similar orchestration tools. Develop integrations with internal and third-party APIs, ensuring reliable and efficient data flows. Design and optimise SQL-based ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable working ...

computer science, IT or a related field, or in bioinformatics with a demonstrated IT expertise Experience in big data technologies and frameworks, such as Apache Spark, Hadoop or similar platforms Hands-on experience with CI/CD (GitLab CI/GitHub Actions) Familiarity with Java Familiarity with Google Cloud ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

optimising ETL/ELT pipelines. Strong understanding of Lakehouse, Data Warehouse, and Data Lake architectures. Experience with Azure Data Factory, Azure Synapse, and Apache Spark. Knowledge of data modelling, semantic models, and enterprise reporting solutions. Experience implementing data governance, security, and quality frameworks. Strong understanding of modern data engineering ...

range of projects that include internal and external suppliers. Essential Skills Proven experience as a Data Engineer Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience of using programming languages such as Python, Scala and SQL. Demonstrable knowledge of data modelling ...

ecosystem •Infrastructure as Code using Terraform, Ansible, or CloudFormation •GitOps tooling such as Argo CD •Scripting (Python, Shell) •Container orchestration tools (e.g., Docker Swarm, Apache Mesos) •Service Mesh technologies (e.g., Istio, Linkerd) •Monitoring and observability tools (Prometheus, Grafana, ELK) •Applying infrastructure security, compliance, and governance practices •Working within agile ...

ensuring data integrity across every layer Lead the transformation of legacy reporting (Python, SQL, Excel) into a secure,scalableand fully observable platform powered by Apache Airflow (WMAA). Define and publish data quality metrics that drive engineering decisions anddemonstrateplatform health to stakeholders Champion AI and modern data tooling ...

data modelling and processing but focuses on practical ability to implement solutions using a range of technologies and languages including SQL, Python, Databricks, Apache Spark, Lakehouse and various RDBMS. This is a senior, hands-on role. You will develop and support more junior team members whilst working ...

TypeScript), Bash Frontend: Vue CI/CD & Automation: Jenkins, GitLab CI, Ansible, Cucumber Cloud & Platforms: AWS, OpenShift, Linux Infrastructure as Code: Terraform Data & Streaming: Apache NiFi Monitoring & Logging: Grafana, ELK Stack, SonarQube Version Control: GitLab Automation & Infrastructure as Code (IaC) Automate infrastructure provisioning, configuration, and CI/CD pipelines ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

modern data engineering best practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence ...

development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security or Security Analytics experience CI/CD and automated deployment practices Strong stakeholder engagement and communication skills You will play ...

development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security or Security Analytics experience CI/CD and automated deployment practices Strong stakeholder engagement and communication skills You will play ...

Cloud: Hands-on experience with Kubernetes , Docker, Terraform, and CI/CD tools like Jenkins or GitHub Actions. Data & Messaging: Expert knowledge of Kafka , Apache Flink, PostgreSQL, Oracle, and Elasticsearch. Security: Deep understanding of OAuth 2.0, secure web services, and API gateway management. Candidate Profile Enterprise Experience: Proven track ...

Ability to build and deploy scalable applications or libraries Strong debugging, problem-solving, and performance optimisation skills Understanding of open-source licensing (MIT, GPL, Apache) Experience with cloud platforms such as AWS or Azure Ability to create clear, comprehensive technical documentation Experience working within development teams on both minor ...

with experience building scalable, reusable frameworks Deep understanding of lakehouse and medallion architectures Experience with data pipeline tools (e.g. DataFactory, Dataflow/Apache Beam) and workflow orchestration A track record of delivering cloud cost optimisation (FinOps) and managing large-scale data environments Strong knowledge of data governance, data lineage ...

scaling data platforms, with a strong focus on Azure Cloud technologies (e.g., Azure Data Factory, Azure Synapse, Azure Data Lake, Azure SQL Database, Databricks, Apache Spark). Proven experience leading the implementation of data platform strategies and managing large‐scale data infrastructure initiatives using Azure. Demonstrated experience implementing modern ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability – Experience ...

professionals with strong Python development expertise, cloud engineering experience, and exposure to modern DevOps and data engineering practices. Must Have: AWS services; Terraform; Apache Spark; Python; Apache Airflow; Docker; GitLab; Security scanning agent; Jupyter; Notebooks; Key Responsibilities Design, develop, and maintain scalable Python applications and services Build … optimize cloud-native data processing pipelines Develop and manage workflows using orchestration tools such as Airflow Work with large-scale data processing frameworks including Apache Spark Deploy and maintain containerized applications using Docker Implement infrastructure automation using Terraform Integrate solutions with enterprise data and secure file transfer systems Collaborate ...

combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud JBLE1 ...

/CD •Terraform Cloud & Infrastructure: •AWS (or equivalent cloud platforms) Testing & Quality: •Cucumber •SonarQube Monitoring & Logging: •ELK Stack (Elasticsearch, Logstash, Kibana) •Grafana Dataflow & Integration: •Apache NiFi We don't expect candidates to know everything on the list. Experience across multiple areas is desirable, but most important is a strong ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...

NICE FOR THE SENIOR SOFTWARE ENGINEER TO HAVE.... Cloud based experience Microservice architecture or server-less architecture Big Data/Messaging technologies such as Apache Nifi/MiNiFi/Kafka TO BE CONSIDERED.... Please either apply by clicking online or emailing me directly to For further information please call ...