26 to 50 of 86 Apache Jobs in the UK

Expopsure to data governance frameworks, metadata management, and regulatory complianc It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders ...

Snowpark Iceberg Tables Secure Data Sharing Snowflake Cortex Data Integration Fivetran SnapLogic Enterprise SaaS integration patterns Data Transformation & Orchestration dbt Cloud/dbt Core Apache Airflow (MWAA) CI/CD pipelines Event-driven orchestration Data Governance & Observability Collibra Data Catalogue Data Lineage Data Governance Frameworks Acceldata Data Observability Data ...

build elegant solutions Understanding of system administration tasks on some of the following; Operating systems (e.g. Linux, CoreOS and Windows) Application stacks (e.g. nginx, Apache, Tomcat, JBoss, Websphere, Hybris, IIS, .Net, Node.js, Solr) RDBMS (e.g. PostgreSQL, MySQL, Oracle, MS SQL) NoSQL (e.g. MongoDB, Cassandra, Riak) Containerisation and microservices (e.g. ...

technical teams in data engineering, data platforms, software engineering, or AI/ML.Technical Depth: Strong technical background capable of evaluating architectures spanning distributed computing (Apache Spark), Python/Scala, modern cloud ecosystems (AWS, Azure, GCP), CI/CD, and MLOps/GenAI frameworks. Leadership Skills: Proven ability to manage ...

migration strategies as well as with machine learning, LLMs, and agentic AI workflows. Proficient in programming, debugging, and problem-solving using SQL, Python, Apache Spark, cloud platforms, and Lakehouse architectures and with AI tools. Experience collaborating with Global System Integrators (GSIs) and third-party consulting organizations to drive customer ...

focus on production reliability. Essential skills and experience Strong hands-on experience developing data engineering solutions using Python and PySpark. Practical experience with Apache Kafka and Apache NiFi for streaming, ingestion and orchestration. Experience working with Hadoop, Hive or comparable enterprise data-lake technologies. Solid knowledge … software development tools. Exposure to agentic AI, workflow orchestration or retrieval-augmented generation. Key responsibilities Design and develop scalable data pipelines using Python, PySpark, Apache Kafka and Apache NiFi. Integrate vendor platforms and enterprise systems through REST APIs and secure authentication mechanisms. Build data processing and storage solutions ...

focus on production reliability. Essential skills and experience Strong hands-on experience developing data engineering solutions using Python and PySpark. Practical experience with Apache Kafka and Apache NiFi for streaming, ingestion and orchestration. Experience working with Hadoop, Hive or comparable enterprise data-lake technologies. Solid knowledge … software development tools. Exposure to agentic AI, workflow orchestration or retrieval-augmented generation. Key responsibilities Design and develop scalable data pipelines using Python, PySpark, Apache Kafka and Apache NiFi. Integrate vendor platforms and enterprise systems through REST APIs and secure authentication mechanisms. Build data processing and storage solutions ...

role). Cloud Infrastructure & Data Pipelines: Build with AWS and Azure services to construct ETL workflows and dynamic systems using orchestration platforms like Apache Airflow. Production Operations: Support production applications, maintain SLAs, and create continuous evaluation metrics for ML solution performance. Qualifications & Skills 2-5 years' hands-on software ...

knowledge sharing and mentoring across the team. What we are looking for Practical experience managing Databricks in a production environment. A strong understanding of Apache Spark, Delta Lake and Unity Catalog. Experience working with one or more major cloud providers, including storage, networking and identity management. Confidence using Terraform ...

knowledge of AI infrastructure. Good proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

with structured streaming, Delta Lake and Delta Live TablesProven experience in Lakehouse migration and modernization using open table formats such as Delta Lake or Apache IcebergProficiency with cloud-native services on Azure and knowledge of multi-cloud environments (AWS or GCP)Hands-on experience with CI/… governance, security practices and enterprise integration controlsBackground in financial trading data or related domainsOfficial Databricks certifications such as Certified Data Engineer Professional or Apache Spark DeveloperWe offerEPAM Employee Stock Purchase Plan (ESPP)Protection benefits including life assurance, income protection and critical illness cover Private medical insurance and dental care ...

/ML for resolving automation problems is highly advantageousFamiliarity with performance testing, security testing, and other non-functional testing approaches using JMeter, Apache BenchmarkCertification in QA or Test Automation (e.g., ISTQB Foundation, Advanced/or Specialist certification)Soft Skills: Excellent problem-solving abilities and attention to detail. Strong communication ...

experience with cloud platforms (AWS, Azure, GCP)Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow)Practical experience with NLP and/or Search techniquesPrompt engineering, and LLMs for enterprise-scale applications. You have team lead experienceStrong collaboration ...

throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experienceFor more information about ...

with external clients across a variety of industry markets. Minimum of 5+ years of customer-facing experience would be preferred[Preferred] Experience working with Apache Spark to process large-scale distributed datasetsCommunication: Proven ability to communicate and teach complex technical concepts to both technical and non-technical audiences. Core ...

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great ...

quality Self-driven, motivated work ethic Energetic and enthusiastic personality What makes you stand out: IMC or another investment management qualification Experience with Apache Airflow Experience with Databricks Experience with asset management data Passion for design and architecture Why join our team: We help you be your best through ...

bandwidth. The goal: no software rewrites, predictable time to production, early integration, and fast iteration. You'll work with tools such as DBT, Python, Apache Airflow, Spark, Iceberg; AWS services like Lambda, Bedrock, and SageMaker, containers, and whatever adds the most value. You'll design tools that are simple ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Other helpful information Hybrid Working: At EMBL-EBI we are pleased to offer hybrid working options for all our employees. A dedicated desk ...

modern AI systems. Strong programming experience in Python. Data & Infrastructure Experience with some of the following technologies is highly desirable: Frameworks/Infrastructure Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch Databases PostgreSQL MongoDB Vector databases such as Qdrant, Milvus, or pgvector Python Libraries Pandas Polars Engineering Skills ...

understanding of the role of testing, DevOps etc. in the development cycle. An understanding of distributed messaging technologies, ideally including experience of JMS, Apache Pulsar or SolaceExperience with relational databases, such as MS SQL Server. Knowledge of frameworks such as Spring, Spring Boot, and Project Reactor/Spring Webflux ...

concisely. Good to HaveKnowledge of the adTech space and particularly the identifiers used in the field. Experience with large-scale distributed data processing (e.g., Apache Spark) and cloud data infrastructure across GCP and AWS (e.g., Dataproc, Dataflow, GKE, EMR).Experience with TemporalWe use automated decision systems (ADS) as part ...

automated deployments, infrastructure management, monitoring, and scaling Coaching and mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark and Apache Airflow for task orchestration Use of Python, SQL, and familiarity with PySpark Understanding of EMR and reviewing output logs ...

explore new technologies, share knowledge, and contribute to a culture of excellence and innovation. Design & Build: Develop and optimise data pipelines using Databricks, Apache Spark, and Python. Develop & Deliver: Create scalable data solutions on AWS leveraging S3, Glue, Lambda, and related services. Integrate & Automate: Implement ETL processes and data … Hands-on commercial expertise with Databricks. You should have: Proven experience as a Data Engineer working with large, complex datasets. Hands-on expertise with Apache Spark, and SQL. Strong proficiency in Python (PySpark preferred). Experience with AWS cloud services including S3, Glue, Lambda, IAM. Familiarity with ETL design ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full time on site in Gloucester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...