1 to 25 of 76 Apache Jobs in the UK excluding London

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence ...

Java or Scala, and SQL. \n Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). \n Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. \n Experience working with cloud‐native ...

Python, Java or Scala, and SQL. Hands on experience building and operating large scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms ...

Python, Java or Scala, and SQL.\ NHands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.G. Apache Spark, Hadoop).\ NExperience designingand maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.\ NExperience working with cloud‐native data platforms, with ...

Python, Java or Scala, and SQL.\ NHands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.G. Apache Spark, Hadoop).\ NExperience designingand maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.\ NExperience working with cloud‐native data platforms, with ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data ...

Engineer, Principal Data Engineer or Technical Lead within a large-scale, data-driven organisation Strong hands-on software engineering expertise with Python, PySpark and Apache Airflow Deep experience designing and building distributed data processing systems and large-scale data platforms on AWS Strong knowledge of modern AWS data technologies ...

years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools ...

able to guide how one could deploy infrastructure into different environments.\ NKnowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop.\ NGood understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes)\ NAbility to create data ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

record with data governance, quality, lineage, and security practices. \n \n Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). \n \n Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). ...

Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). Deep knowledge ...

Lead Backend Software Engineer – Data Services Federated Data Services | Distributed Systems | Apache Spark | DataFusion | Python | Kubernetes Location: London (2 days per week onsite) Rate: £700–£800 per day (Outside IR35) Duration: Initial 6-month contract with likely extension Overview We're looking for an experienced Lead Backend Software Engineer … data services capable of securely querying and processing data across multiple environments. Design high-performance data processing and streaming workflows using technologies such as Apache Spark, Apache DataFusion, or similar distributed query frameworks. Design and implement Kubernetes-based services and containerised workloads. Build secure APIs and backend services ...

developing others. Desirable to have skills or experience in: - SAS Viya for data analytics. - Job Scheduling tooling (e.g. Berlin/JS7 Job Scheduler, TWS, Apache Airflow). - Denodo for data virtualisation. - Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects). - ALM Tooling (Jira, Confluence). ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly to stakeholders ...

data into something they can actually use and act on. What you'll be doing: Designing and building performant data ingestion pipelines using Apache Spark, ideally within Azure Databricks Contributing to the platform's data modelling and architecture Building dashboards and semantic models, and presenting these directly to stakeholders ...

alerting. Minimum requirements: 3+ years of experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of Python, SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code ...

ensuring data integrity across every layer Lead the transformation of legacy reporting (Python, SQL, Excel) into a secure,scalableand fully observable platform powered by Apache Airflow (WMAA). Define and publish data quality metrics that drive engineering decisions anddemonstrateplatform health to stakeholders Champion AI and modern data tooling ...

Python, React, Go. Experience with deploying, configuring, and managing cloud architecture and technologies in AWS environments. Experience with web application services such as NGINX, Apache, JBoss. Knowledge of OpenShift Containerisation, RHEL 6,7,8, Docker and Kubernetes. Experience with monitoring systems e.g., ELK, Nagios, New Relic, DataDog, Splunk etc. ...

Python, React, Go.\ NExperience with deploying, configuring, and managing cloud architecture and technologies in AWS environments.\ NExperience with web application services such as NGINX, Apache, JBoss.\ NKnowledge of OpenShift Containerisation, RHEL 6,7,8, Docker and Kubernetes.\ NExperience with monitoring systems e.G., ELK, Nagios, New Relic, DataDog, Splunk etc. ...

Python, React, Go. Experience with deploying, configuring, and managing cloud architecture and technologies in AWS environments. Experience with web application services such as NGINX, Apache, JBoss. Knowledge of OpenShift Containerisation, RHEL 6,7,8, Docker and Kubernetes. Experience with monitoring systems e.g., ELK, Nagios, New Relic, DataDog, Splunk etc. ...

modern data engineering best practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence ...