1 to 25 of 36 Apache Jobs in the South West

skills, with experience of Java or Scala beneficial Experience building large-scale ETL/ELT and data processing pipelines Hands‐on experience with Apache Spark and distributed data processing technologies Experience designing and operating cloud-native data platforms, particularly within AWS Familiarity with Docker, Kubernetes and CI/ ...

Python, Java or Scala, and SQL. Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms ...

Python, Java or Scala, and SQL. Hands-on experience building and operating large-scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud-native data platforms ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools ...

work effectively with technical teams including Data Scientists and MLOps Engineers. Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/vector ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

file processing , data transformation, and batch processing Strong SQL and database skills with focus on: Query optimization Performance tuning Data modeling Experience with Apache Flink , ETL frameworks, or equivalent data processing technologies Exposure to REST APIs, messaging systems, and event‐driven architectures Understanding of CI/CD pipelines ...

possible, on-site in Gloucester when required. You will design and manage data pipelines, ETL processes, and data integration for National Security clients using Apache tools including NiFi, Kafka, and SQL/NoSQL databases like MongoDB. Proficiency in Groovy, Python or Java for ETL, data modeling, and data governance ...

Jenkins, Bamboo, Concourse etc. Monitoring utilising products such as: Prometheus, Grafana, ELK, filebeat etc. Observability - SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Edge technologies e.g. NGINX, HAProxy etc. Excellent knowledge of YAML or similar languages It is essential for this role ...

implement highly scalable and efficient data pipelines using advanced Python (Pandas, web scraping and machine learning libraries), SQL (including advanced optimisation techniques) and Apache Airflow. Lead the extraction, transformation and loading (ETL) of large and complex datasets, including time series data, from various sources (web APIs, databases, real-time … data modelling and design skills. Extensive experience designing and building large-scale, efficient data pipelines for complex and time series data. Proven experience with Apache Airflow for data orchestration and scheduling at scale. Expertise in AWS cloud services (S3, Redshift, EC2, Lambda, etc.) is a must. Definition and design ...

when required Must hold active Enhanced DV Clearance (West) Circa £700 p/d inside IR35 Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Gloucester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java Desirable skills: Java Docker Kubernetes ...

skills Hands-on experience with Databricks for scalable data engineering, analytics, and collaborative development, as well as proficiency in managing large datasets using Apache Iceberg for efficient, reliable table management and data lake optimization Experience architecting and implementing enterprise-scale data warehouses on cloud platforms (e.g., AWS, Azure … data pipeline deployment and monitoring using CI/CD tools and infrastructure-as-code practices Hands-on expertise with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and real-time data streaming technologies (e.g., Kafka, Kinesis, Spark) Track record of collaborating with cross-functional teams (engineering, analytics, business stakeholders ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full time on site in Gloucester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

role. Managing multi‐site Digital Signage platforms (AMP, Scala, Broadsign, Bightsign) is preferred. Server Technology Proficiency in configuring and managing web server software like Apache and Nginx. Databases & Caching Expertise in setting up and maintaining scalable MySQL databases and Redis cache clusters. Platforms & OS Experience managing multi‐site digital ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that ...

PostgreSQL, and a working proficiency of SQL. Practical application of Data Access Management and Governance in tooling such as Immuta (strongly preferred), Microsoft Purview, Apache Ranger, Privacera, Collibra Protect or GCP Cloud DLP, with experience of native access and masking controls in Snowflake or Databricks. Ability to credibly collaborate ...

Engineer with the following skills: The Fundamentals: Modern JavaScript GitLab CI/CD Observability Technologies Java GitLab Version Control Database Technologies Web API Integration Apache NiFi Message Broker Technologies OpenShift Web Auth Distributed Architecture Kubernetes SecDevOps Cloud Native Technologies Required Clearance: DV or recently active DV Key Benefits ...

strong skills in several of the following Modern JavaScript GitLab CI/CD Observability Technologies Web API Integration GitLab Version Control Database Technologies OpenShift Apache NiFi Message Broker Technologies Java Web Auth Distributed Architecture Kubernetes SecDevOps Cloud Native Technologies The Interview Process We believe in keeping the interview process ...

JUnit and Cucumber Experience working with XML/JSON Experience working with large databases Familiarity with tools such as SVN, Git, Ant, Maven, Tomcat, Apache, and Jenkins Why Edenred PayTech? We are a subsidiary of the Edenred group and leaders in prepaid solutions including banking, virtual cards, debit, credit ...

Role As a Senior Data Engineer, you'll take ownership of the design, development and optimisation of production-grade data solutions using Databricks, Apache Spark and Delta Lake. Your responsibilities will include: Designing, building and optimising scalable ETL/ELT data pipelines using Databricks, PySpark and Spark SQL Implementing … engineering skills with the ability to take technical ownership of complex data solutions. You'll ideally have: Strong commercial experience with Databricks Excellent PySpark, Apache Spark and Spark SQL skills Experience building production-grade ETL/ELT pipelines Strong knowledge of Delta Lake and Lakehouse architecture Experience implementing Medallion ...

Beneficial experience: AWS services, JUnit, Cucumber, XML/JSON, working with large databases, and tools such as SVN/Git, Ant/Maven, Tomcat, Apache, Jenkins. Behavioural skills Decision maker: able to weigh complex options and reach sound decisions. Influencer: able to articulate and challenge the status quo. Well ...

Beneficial experience: AWS services, JUnit, Cucumber, XML/JSON, working with large databases, and tools such as SVN/Git, Ant/Maven, Tomcat, Apache, Jenkins. Behavioural skills Decision maker: able to weigh complex options and reach sound decisions. Influencer: able to articulate and challenge the status quo. Well ...

data processing solutions using BigQuery, Pub/Sub, Dataflow and Dataproc Experience developing end‐to‐end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam. Expert understanding of when to use a range of data storage technologies including relational/non‐relational, document ...