1 to 25 of 36 Apache Jobs in the South West

Python, PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence ...

Java or Scala, and SQL. \n Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). \n Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. \n Experience working with cloud‐native ...

Python, Java or Scala, and SQL. Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop).Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.Experience working with cloud‐native data platforms, with strong ...

Python, Java or Scala, and SQL. Hands on experience building and operating large scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms ...

Python, Java or Scala, and SQL.\ NHands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.G. Apache Spark, Hadoop).\ NExperience designingand maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.\ NExperience working with cloud‐native data platforms, with ...

Python, Java or Scala, and SQL.\ NHands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.G. Apache Spark, Hadoop).\ NExperience designingand maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools.\ NExperience working with cloud‐native data platforms, with ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). Deep knowledge ...

ensuring data integrity across every layer Lead the transformation of legacy reporting (Python, SQL, Excel) into a secure,scalableand fully observable platform powered by Apache Airflow (WMAA). Define and publish data quality metrics that drive engineering decisions anddemonstrateplatform health to stakeholders Champion AI and modern data tooling ...

Python, React, Go. Experience with deploying, configuring, and managing cloud architecture and technologies in AWS environments. Experience with web application services such as NGINX, Apache, JBoss. Knowledge of OpenShift Containerisation, RHEL 6,7,8, Docker and Kubernetes. Experience with monitoring systems e.g., ELK, Nagios, New Relic, DataDog, Splunk etc. ...

Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability ...

digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite … compressed hours. The role will include: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security ...

digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite … hours.\ N \nThe role will include:\ N\nDesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.\ NImplement data ingestion, transformation, and integration processes, ensuring data quality and security.\ NCollaborate with data architects and security teams to ensure compliance with ...

Build and operate batch and streaming data pipelines with strong scalability, performance, and fault toleranceDevelop and manage workflow orchestration using tools such as Apache Airflow to support reliable, observable, and well-scheduled data movement and transformationsImplement and optimize data models and warehouse structures to support analytics and business intelligence … Flink, Beam, or similar)Experience automating deployment, releases, and testing in continuous integration and continuous delivery pipelinesExperience with lakehouse patterns and table formats (e.g., Apache Iceberg)Experience with federated query engines such as TrinoExperience designing automated tests (unit, component, integration, and end-to-end), including use of mocking frameworksExperience ...

Build and operate batch and streaming data pipelines with strong scalability, performance, and fault tolerance Develop and manage workflow orchestration using tools such as Apache Airflow to support reliable, observable, and well-scheduled data movement and transformations Implement and optimize data models and warehouse structures to support analytics … Beam, or similar) Experience automating deployment, releases, and testing in continuous integration and continuous delivery pipelines Experience with lakehouse patterns and table formats (e.g., Apache Iceberg) Experience with federated query engines such as Trino Experience designing automated tests (unit, component, integration, and end-to-end), including use of mocking ...

databases Experience with one of Azure/AWS/GCP ETL tools like Informatica, Alteryx and Business Glossary tools like Collibra, Azure Purview or Apache Atlas. TDD/BDD, Experience working in an Agile development environment Identity management and synchronisation will be an add on for the resources Experience ...

Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability ...

Swindon (we could consider twice per fortnight etc). Required Skills & Experience Strong experience testing data pipelines/ETL processes Hands-on experience with Apache Spark Experience working with Databricks and/or AWS Glue Proven ability to build automation or testing frameworks (Python preferred) Experience with large-scale ...

databasesExperience with one of Azure/AWS/GCPETL tools like Informatica, Alteryx and Business Glossary tools like Collibra, Azure Purview or Apache Atlas.TDD/BDD, Experience working in an Agile development environmentIdentity management and synchronisation will be an add on for the resourcesExperience with Data Dictionaries, Business glossaries ...

languages: Python, React, Go. Familiarity with deploying applications to cloud architecture and technologies in AWS environments. Experience with web application services such as NGINX, Apache, JBoss. Experience of Test Driven or Behaviour Driven development. API development and integration (preferably using Go but not essential). Experience with monitoring systems ...

Familiarity with deploying applications to cloud architecture and technologies in AWS environments. \n \n Experience with web application services such as NGINX, Apache, JBoss. \n \n Experience of Test Driven or Behaviour Driven development. \n \n API development and integration (preferably using Go but not essential). ...

languages: Python, React, Go. Familiarity with deploying applications to cloud architecture and technologies in AWS environments. Experience with web application services such as NGINX, Apache, JBoss. Experience of Test Driven or Behaviour Driven development. API development and integration (preferably using Go but not essential). Experience with monitoring systems ...