19 of 19 Apache Jobs in the South West

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence ...

skills, with experience of Java or Scala beneficial Experience building large-scale ETL/ELT and data processing pipelines Hands-on experience with Apache Spark and distributed data processing technologies Experience designing and operating cloud-native data platforms, particularly within AWS Familiarity with Docker, Kubernetes and CI/ ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

with monitoring/logging tools (ELK stack, Prometheus, Splunk) Familiarity with service mesh and modern networking (Istio, Envoy) Exposure to data engineering and integration (Apache ecosystem, SQL, Python) Recognised technical certifications (AWS/Azure Architect, Kubernetes CKA/CKAD, Terraform) Experience in influencing technical standards, best practices, or contributing ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

containerisation and orchestration technologies including Kubernetes, Docker, ECS, EKS, and AKS . Experience working with distributed event-streaming technologies such as Kafka and Apache Flink . Strong troubleshooting skills with the ability to resolve complex technical and operational issues. Proven ability to lead, mentor, and develop engineers within Agile ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

build elegant solutions Understanding of system administration tasks on some of the following; Operating systems (e.g. Linux, CoreOS and Windows) Application stacks (e.g. nginx, Apache, Tomcat, JBoss, Websphere, Hybris, IIS, .Net, Node.js, Solr) RDBMS (e.g. PostgreSQL, MySQL, Oracle, MS SQL) NoSQL (e.g. MongoDB, Cassandra, Riak) Containerisation and microservices (e.g. ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead experience ...

with cloud platforms (AWS, Azure, GCP) Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead experience ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external client ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full time on site in Gloucester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that ...

secure, scalable data solutions across the full lifecycle, including acquisition, integration, storage, processing and analysis. Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL. Design data models, schemas and data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate data … take ownership of customer-facing deliverables. Strong Python and SQL skills, with experience building, testing and optimising data pipelines. Hands-on experience with Apache NiFi for data ingestion, routing, transformation and orchestration. Experience of data modelling and schema design, including defining data contracts or Interface Control Documents (ICDs). ...

Management Stakeholder Collaboration Leadership & Mentorship Technology Stack The successful candidate will have experience working with some or all of the following technologies: Data Processing: Apache Spark/PySpark Workflow Orchestration: Apache Airflow Containerisation: Kubernetes Cloud Platform: Microsoft Azure Data APIs & Semantic Layer: CubeJS Programming Languages: Python, SQL Essential ...

Bristol , with salary dependent on experience. As a developer at Shiptheory, youll work across our technology stack, including our core PHP/Linux/Apache application, MySQL databases, mission-critical Python messaging systems, and cloud infrastructure. Your work will range from fixing bugs and building new features to designing ...

robust data models, ETL pipelines, metadata frameworks, and modern data warehousing solutions. Design and optimise large-scale streaming data platforms using technologies such as Apache Flink and other real-time processing frameworks. Ensure all solutions comply with industry best practices, governance frameworks, and regulatory requirements, maintaining the highest standards … Lambda, Redshift , and related data services. Proven experience designing secure architectures within sensitive or highly regulated environments. Hands-on experience with Apache Flink and real-time data processing architectures. Strong understanding of data modelling methodologies, architectural patterns, and industry standards. Expertise across ETL processes, data integration, data warehousing, distributed ...

developers, engineers, architects, and platform specialists, contributing to both business-as-usual services and innovative technology initiatives. The role offers exposure to Kubernetes, OpenShift, Apache NiFi, Identity Platforms, DevOps tooling, and open-source technologies. Key Responsibilities As an IdAM Application Developer, you will: Design, develop, test, and deploy data … enhance in-house Kubernetes clusters supporting application delivery across multiple environments. Develop and maintain Ansible automation roles, enabling users to independently deploy and manage Apache NiFi clusters. Provide technical guidance, support, and training on Apache NiFi and related technologies. Collaborate with engineering teams to deliver secure, scalable ...

ensuring best practices in data architecture, security, and cloud computing Proficiency in data modelling, ETL processes, data warehousing, distributed systems and metadata systems Utilise Apache Flink and other streaming technologies to build real-time data processing systems that handle large-scale, high-throughput data Ensure all data solutions comply … Lambda, and Redshift. Experience with other cloud providers (e.g., Azure, GCP) is a plus In-depth knowledge and hands-on experience with Apache Flink for real-time data processing Proven experience in mentoring and managing teams, with a focus on developing talent and fostering a collaborative work environment Strong ...