1 to 25 of 130 Apache Jobs in the UK excluding London

Contract Software Data Engineer - Chester - Inside IR35 - Investment Banking

Location
Chester, England, United Kingdom
replay, backfill, recoverability and schema evolution Define and enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using:Apache Kafka Apache NiFi Apache Spark/PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment … interested in engineers who can demonstrate strong expertise across several of the following areas: Core Technical Skills Python development in enterprise environments Apache Spark/PySpark Apache Kafka Apache NiFi Hadoop and Hive Advanced SQL REST API integration and development Data modelling and data lineage Distributed systems ...

Data Engineer (SC Eligible/ SC Cleared)

Location
Newcastle upon Tyne, England, United Kingdom
solving and communication skills. Able to work independently and remotely, and to travel to Newcastle once a week. Desirable skills and experience Experience with Apache Spark, Databricks or Kafka. Familiarity with orchestration tools such as Airflow. Experience with CI/CD for data pipelines and Infrastructure as Code. Knowledge ...

Data Engineer

Location
Glasgow, Scotland, United Kingdom
vector databases Streaming technologies (Kafka, Kinesis, Flink, etc.) Containers and related services (Docker, Kubernetes, container Registries, etc) Data modelling approaches (Kimball, Imnon) Orchestration tools - Apache Airflow, Prefect or cloud-native tools Backend software development (Java, APIs, Scalability, Logging and Monitoring etc.) MLFlow and other MLOps/Machine Learning Engineering ...

Data Engineer - Security & Intelligence

Location
Cheltenham, England, United Kingdom
skills, with experience of Java or Scala beneficial Experience building large-scale ETL/ELT and data processing pipelines Hands‐on experience with Apache Spark and distributed data processing technologies Experience designing and operating cloud-native data platforms, particularly within AWS Familiarity with Docker, Kubernetes and CI/ ...

AWS Data Engineer – Power BI

Location
Lancashire, England, United Kingdom
Automated deployments across environments. Monitoring and logging. Release and configuration management. Production support and incident troubleshooting. Python or another scripting language. AWS certification. Terraform. Apache Airflow/MWAA. Databricks or Spark. Kafka/Kinesis. dbt. Experience working in Agile/Scrum environments. Experience 4+ years of experience in Data ...

Senior Data Engineer - Security Cleared

Hiring Organisation
Hackajob Ltd
Location
South West London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

Secure Data Engineer

Location
Newcastle upon Tyne, England, United Kingdom
will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We are looking ...

Data Platform Specialist

Hiring Organisation
Anson Mccade
Location
South East, United Kingdom
Employment Type
Permanent Contract, On-site
technical leadership of data engineering teams. Experience with AWS, Azure or GCP. Knowledge of Databricks, Snowflake or modern cloud data platforms. Strong experience with Apache Spark, SQL, Kafka and ETL/ELT pipelines. Understanding of Lakehouse, Medallion and Data Mesh architectures. Experience with Terraform, CI/CD, data quality ...

Data Engineer / AI Engineer

Location
Bracknell, England, United Kingdom
builder and owner before IT infrastructure is fully in place — a 'zero-to-one' mindset. Nice to Have Familiarity with orchestration tools such as Apache Airflow, dbt, or Azure Data Factory. Exposure to LLM frameworks (LangChain, OpenAI API, Azure OpenAI).Experience with version control (Git/GitHub ...

Senior Data Engineer

Hiring Organisation
We Are Dcoded Limited
Location
South East, United Kingdom
Employment Type
Permanent Contract, Hybrid
Salary
£75,000
production pipelines Strong SQL and Python, plus hands-on Git Production experience with at least one of Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow or PySpark Cloud-native orchestration and distributed data processing Infrastructure as code (e.g. Terraform) CI/CD and DevOps or DevSecOps practice ...

Lead Data Engineer

Location
Manchester, England, United Kingdom
possess hands‐on experience with one or more of the following technological areas (or their equivalent cloud‐native alternatives): Analytics & Platform Frameworks like Apache Spark or similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON ...

Lead Data Engineer

Location
Swansea, Wales, United Kingdom
possess hands‐on experience with one or more of the following technological areas (or their equivalent cloud‐native alternatives): Analytics & Platform Frameworks like Apache Spark or similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON ...

Data Engineer

Location
Milton Keynes, England, United Kingdom
schemas for both structured and unstructured data with a snowflake environment Enhance cloud data capabilities, working with AWS S3, Azure Data Lake, and Apache Spark Collaborate with cross-functional teams to ensure data availability for advanced analytics and reporting Automate workflows, implement CI/CD pipelines, and maintain data ...

Lead Data Platform Engineer - AWS, Spark

Location
Nottingham, England, United Kingdom
SQS. Strong experience with DevOps and Infrastructure as Code tooling, including Terraform, CI/CD pipelines, Git and Jenkins. Strong experience developing on Apache Spark or Hadoop. Significant programming experience using Scala or Python. Experience with Configuration as Code principles and API integration. Strong understanding of cloud security, data ...

Data Engineer

Location
Blandford Forum, England, United Kingdom
Snowflake and Databricks, Python and SQL, ETL development and data modelling. Experience with cloud-based data platforms and data governance. Nice to have: Apache Airflow, Spark, AWS/Azure, data visualisation and experience delivering solutions within Defence or secure Government environments. Clearance: Due to the sensitive nature ...

Principal Data Engineer

Location
Manchester, England, United Kingdom
analytics and AI/ML at enterprise scale.* Develop and optimise batch and streaming ETL/ELT pipelines using Azure Data Factory, PySpark, SQL, Apache Spark and Airflow, integrated with modern tooling like dbt and Git-based CI/CD.* Implement data cataloguing, lineage tracking and governance frameworks through ...

Data Engineer (early career)

Location
City of Edinburgh, Scotland, United Kingdom
cloud platforms such as AWS, Azure or GCP is desirable. Basic understanding of relational databases and SQL queries. Awareness of tools such as Airflow, Apache NiFi or Talend is helpful. Foundations in data warehousing, modelling and schema design, with an interest in building dashboards using Power BI, QlikView ...

Data Engineer

Location
Newcastle upon Tyne, England, United Kingdom
United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

Microsoft Fabric Data Architect

Hiring Organisation
Dns Info Ltd
Location
West Midlands, United Kingdom
Employment Type
Contract
Contract Rate
GBP 35 Annual
Warehouse Fabric Data Factory Pipelines, Dataflows Gen2 Azure Azure Data Lake, Azure SQL, ADF SQL Advanced SQL, query optimisation Python PySpark, data processing Spark Apache Spark, Spark SQL Data Architecture Enterprise data architecture Data Modelling Star schema, dimensional modelling Power BI Semantic models, DAX Data Governance Security, lineage, data ...

Data Engineer - Databricks

Location
Bristol, England, United Kingdom
engineering or business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source systems. Knowledge ...

Software Engineer III - Cloud Data Platform (AWS/Databricks, Terraform)

Hiring Organisation
Hackajob Ltd
Location
Glasgow, Scotland, United Kingdom
Employment Type
Permanent Contract, On-site
inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworks Familiarity with AWS-native data services such as Glue, Lake Formation, Redshift, or Athena Exposure to data governance practices ...

Principal DevOps Engineer

Location
Bristol, England, United Kingdom
with monitoring/logging tools (ELK stack, Prometheus, Splunk) Familiarity with service mesh and modern networking (Istio, Envoy) Exposure to data engineering and integration (Apache ecosystem, SQL, Python) Recognised technical certifications (AWS/Azure Architect, Kubernetes CKA/CKAD, Terraform) Experience in influencing technical standards, best practices, or contributing ...

Principal DevOps Engineer

Hiring Organisation
Hackajob Ltd
Location
Gloucester, South West, United Kingdom
Employment Type
Permanent Contract, Hybrid
with monitoring/logging tools (ELK stack, Prometheus, Splunk) Familiarity with service mesh and modern networking (Istio, Envoy) Exposure to data engineering and integration (Apache ecosystem, SQL, Python) Recognised technical certifications (AWS/Azure Architect, Kubernetes CKA/CKAD, Terraform) Experience in influencing technical standards, best practices, or contributing ...

Senior Data Engineer

Location
Birmingham, England, United Kingdom
dimensional modelling, star/snowflake schemas, data vault, etc.). Experience designing and operating ETL/ELT pipelines and modern workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, Azure Functions). Professional experience in delivering cloud-based data engineering solutions at scale, with strong client-facing, stakeholder management ...

Principal DevSECOps Engineer

Location
Gloucester, England, United Kingdom
creative mind. It would be nice if you had: It would be nice if you had: Data Engineering – experience of one or more: Apache ecosystem, SQL, Python. Web - HTML, CSS, JavaScript, XML, SOAP. Experience with Secure DevSecOps within an Agile/SAFe environment. Containerisation & Orchestration - Docker, Podman, Kubernetes, Rancher ...