1 to 25 of 90 Apache Jobs in the UK

native solutions that support analytics, automation, and digital transformation. Data Engineering, Full Stack & Platform Development Design and implement scalable data pipelines using tools like Apache Spark, Airflow, or dbt.Build and maintain data lakes, warehouses, and real-time streaming solutions. Develop APIs and microservices to expose data securely and efficiently. ...

skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab and modern software development practices. API integration ...

Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/GCP) Experience ...

strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. Essential qualifications • Extensive experience with Python and PySpark/Apache Spark. • Proven track record designing and delivering scalable data pipelines and data products. • Strong SQL and data modelling skills. • Experience with modern data architectures ...

analytics and reporting, and contributing to the adoption of AI-powered workflows and automation. Key Responsibilities & Skills: Build and maintain scalable data pipelines using Apache Airflow or similar orchestration tools. Develop integrations with internal and third-party APIs, ensuring reliable and efficient data flows. Design and optimise SQL-based ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable working ...

Extensive experience delivering large-scale enterprise data and analytics solutions Strong track record leading architecture across complex transformation programmes Expert knowledge of Databricks and Apache Spark ecosystems Deep expertise in either Microsoft Azure Data Platform Services or AWS Data Platform Services Working knowledge of the alternative cloud platform Experience ...

computer science, IT or a related field, or in bioinformatics with a demonstrated IT expertise Experience in big data technologies and frameworks, such as Apache Spark, Hadoop or similar platforms Hands-on experience with CI/CD (GitLab CI/GitHub Actions) Familiarity with Java Familiarity with Google Cloud ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

Labelled property graphs Hypergraph structures Entity resolution Graph schema design Data Engineering & Integration Pipeline Development Data Integration Languages & Tools ETL/ELT pipeline design Apache Kafka Apache Spark AWS Glue dbt Apache Airflow Entity matching and resolution Data lineage tracking Schema mapping Semantic data integration Metadata management ...

optimising ETL/ELT pipelines. Strong understanding of Lakehouse, Data Warehouse, and Data Lake architectures. Experience with Azure Data Factory, Azure Synapse, and Apache Spark. Knowledge of data modelling, semantic models, and enterprise reporting solutions. Experience implementing data governance, security, and quality frameworks. Strong understanding of modern data engineering ...

Core Java Spring Boot frameworks Proven experience building distributed backend systems Hands-on experience with Kafka (or similar streaming platforms) Experience with Scala and Apache Spark for data processing Integration withbackend services such as: PostgreSQL Redis S3 (or equivalent object storage) Expertise in modern engineering practices, including: Continuous delivery ...

range of projects that include internal and external suppliers. Essential Skills Proven experience as a Data Engineer Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience of using programming languages such as Python, Scala and SQL. Demonstrable knowledge of data modelling ...

with monitoring/logging tools (Prometheus, Grafana, ELK) and root cause analysis Knowledge of storage (NFS/EFS), databases (PostgreSQL), and web servers (Nginx, Apache, Tomcat) Experience with SSO and security (SAML, OIDC, Azure AD, Okta, Keycloak) Scripting skills in Python and/or Bash Profile: Client-focused with ...

ecosystem •Infrastructure as Code using Terraform, Ansible, or CloudFormation •GitOps tooling such as Argo CD •Scripting (Python, Shell) •Container orchestration tools (e.g., Docker Swarm, Apache Mesos) •Service Mesh technologies (e.g., Istio, Linkerd) •Monitoring and observability tools (Prometheus, Grafana, ELK) •Applying infrastructure security, compliance, and governance practices •Working within agile ...

tools such as Terraform Familiarity building and integrating with NoSQL database (familiarity with MongoDB is preferred) or SQL databases such as Postgres Familiarity. With Apache NiFI Familiarity with web standards, accessibility, and development best practice Bonus: Building microservices in C# using .Net Experience building solutions that integrate with LLMs ...

ensuring data integrity across every layer Lead the transformation of legacy reporting (Python, SQL, Excel) into a secure,scalableand fully observable platform powered by Apache Airflow (WMAA). Define and publish data quality metrics that drive engineering decisions anddemonstrateplatform health to stakeholders Champion AI and modern data tooling ...

data modelling and processing but focuses on practical ability to implement solutions using a range of technologies and languages including SQL, Python, Databricks, Apache Spark, Lakehouse and various RDBMS. This is a senior, hands-on role. You will develop and support more junior team members whilst working ...

TypeScript), Bash Frontend: Vue CI/CD & Automation: Jenkins, GitLab CI, Ansible, Cucumber Cloud & Platforms: AWS, OpenShift, Linux Infrastructure as Code: Terraform Data & Streaming: Apache NiFi Monitoring & Logging: Grafana, ELK Stack, SonarQube Version Control: GitLab Automation & Infrastructure as Code (IaC) Automate infrastructure provisioning, configuration, and CI/CD pipelines ...

Infra security, compliance and governance • Familiarity with CNCF ecosystem and building platforms on top of Kubernetes • Container orchestration tools, e.g. Docker Swarm/Apache Mesos • Knowledge of Service Mesh technologies, e.g. Istio/Linkerd • Experience of monitoring tools, e.g. Prometheus, Grafana, ELK • Rounded problem-solving skills - troubleshoot and resolve ...

production, providing subject matter expertise on the .Net stack and contributing to technical design discussions. You'll use a range of technology including Apache Flink with Java for large scale data processing and will be able to assess and recommend new and emerging technologies, using the best tool ...

with geospatial datasets and technologies, including: Shapefiles GeoParquet GeoPackage GeoTIFF Spatial data processing and transformation techniques Geospatial analytics and data engineering workflows Experience with Apache Sedona or similar geospatial processing frameworks is highly desirable. Role and Responsibilities: Collaborate with stakeholders, product owners and technical teams to understand data requirements ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

Jenkins or GitLab CI/CD Infrastructure as Code (e.g. Terraform) Data & Integration: Working with large datasets and data pipelines Tools such as Apache NiFi or similar integration frameworks Scripting & Automation: Bash, Ansible Monitoring & Logging: ELK Stack (Elasticsearch, Logstash, Kibana), Grafana Testing & Quality: Automated testing frameworks (e.g. PyTest, Cucumber ...

Jenkins or GitLab CI/CD Infrastructure as Code (e.g. Terraform) Data & Integration: Working with large datasets and data pipelines Tools such as Apache NiFi or similar integration frameworks Scripting & Automation: Bash, Ansible Monitoring & Logging: ELK Stack (Elasticsearch, Logstash, Kibana), Grafana Testing & Quality: Automated testing frameworks (e.g. PyTest, Cucumber ...