1 to 25 of 65 Permanent Apache Jobs in the UK excluding London

stack evolves. Nice to Have Experience with stream processing frameworks such as Flink or Spark Streaming. Experience with task orchestration tools such as Apache Airflow or similar systems. Experience working with and designing infrastructures for large‐scale data processing (Hive, Snowflake, NoSQL databases). Experience with data governance practices ...

automation capabilities, to improve the value realized by automation Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT ...

understanding of Quantexa components such as: Entity Resolution (ER), Scoring Framework, Contextual Network Generation, UI Configuration. Strong background in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch. Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern data ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Python skills * Hands-on experience designing and developing ETL pipelines and data workflows * Experience designing and maintaining data pipelines using technologies such as Apache Airflow or Spark * Experience working with cloud platforms including AWS, Azure or GCP * Strong understanding of data modelling, integration and governance principles * Experience working with ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with AI evaluation and observability practices ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

ownership Technical Application ownership Qualifications Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , Harness Strong experience of developing on Apache Spark/Hadoop Strong experience with AWS cloud services including AWS Glue, S3, Step Strong experience building data pipelines and data lake Experience ...

reviews and continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices ...

innovation-led working environment Key Experience Proven experience as a Data Architect, Databricks Architect, Lead Data Engineer or Solutions Architect Strong expertise in Databricks, Apache Spark, PySpark and SQL Experience designing and implementing modern data lake or lakehouse architectures Hands-on experience with Kafka Strong understanding of AWS data ...

Codebase ownership Technical Application ownership Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins, Harness Strong experience of developing on Apache Spark/Hadoop Strong experience with AWS cloud services including AWS Glue, S3, Step Strong experience building data pipelines and data lake Experience ...

Understanding of Azure data services and API-based integration. Next.js 14, FastAPI Python, Pipeline orchestration, Data transformation, Data quality, Casbin RBAC, Kuzu knowledge graph, Apache Iceberg, MCP AI server integration and ODCS v2 data contracts layer. Experience in establishing the infrastructure, pipelines, processes, and governance required for production environment ...

configuration management and an interest in migrating to Ansible and cloud native tooling. Experience managing Kubernetes platforms and deploying containerised applications. Knowledge of GitLab, Apache NiFi, HashiCorp Vault, Trend Micro Deep Security Manager, Ivanti device control, and Ansible Tower/AWX. Windows Infrastructure Experience with Windows Server administration, deployment ...

configuration management and an interest in migrating to Ansible and cloud native tooling. Experience managing Kubernetes platforms and deploying containerised applications. Knowledge of GitLab, Apache NiFi, HashiCorp Vault, Trend Micro Deep Security Manager, Ivanti device control, and Ansible Tower/AWX. Windows Infrastructure Experience with Windows Server administration, deployment ...

technical trade‐offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with or strong advocacy for AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. Proficiency ...

Snowflake Hands-on expertise with Spark/PySpark and big data processing technologies Knowledge of open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints Familiarity with AI evaluation and observability practices for LLM workloads Understanding ...

Data Engineer to design, build, and maintain scalable data pipelines powering Moody’s next-generation digital content platform. You will work with DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS, driving platform optimizations and cross-functional collaboration. In this role, you’ll collaborate with product owners ...

structures, partitioning strategies, consistency models, and performance optimisation, as well as OLTP databases such as PostgreSQL and MySQL. Distributed compute frameworks – strong experience with Apache Spark and distributed data processing, including optimisation, performance tuning, and large‐scale batch/streaming workloads. Cloud‐native data platforms – demonstrated experience with ...

Security Groups, CloudFormation, RDS, DynamoDB and Redshift, including secure and well-governed implementations. Data engineering and pipelines, building efficient batch and streaming solutions using Apache Spark and AWS services, and working with large-scale datasets in enterprise environments. Demonstrated technical problem-solving capability, with the ability to rapidly understand ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL, Team Leadership Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical … Data Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Gloucester when required. Required experience in the following: Apache Kafka Apache NiFI SQL and NoSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java Understand and interpret technical ...

databases utilised in modern AI applications. Strong Python development skills. Data & Infrastructure: Experience with some of the following technologies is highly desirable: Frameworks & Infrastructure: Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch Databases: PostgreSQL MongoDB Vector databases such as Qdrant, Milvus, or pgvector Python Ecosystem: Pandas Polars Engineering ...

Experience using Git/version control Strong communication skills with both technical and non-technical stakeholders Desirable Skills: Microsoft Fabric Azure Synapse Analytics Databricks Apache Spark/PySpark dbt Apache Airflow Power BI Azure DevOps CI/CD pipelines Terraform What's On Offer: £65k+ salary package Hybrid ...

continuous improvement and automation. Desirable Experience Experience within Financial Services or other highly regulated environments. VMware virtualisation platforms. F5 Load Balancers. Infoblox DNS. Apache Airflow. Prometheus, Grafana and Loki. Infrastructure-as-Code methodologies. Enterprise patching programmes and operating system lifecycle management. Platform modernisation or automation initiatives. What ...