4 of 4 Contract Apache Spark Jobs in the South West

/CD, and Infrastructure as Code. Desirable: Experience within UK Defence, Intelligence, or National Security sectors. Knowledge of big data tools such as Kafka, Spark, or Hadoop. Support for machine learning or advanced analytics workloads. Experience working with classified datasets and Elastic Stack (ELK). If you're ready ...

Jenkins, Bamboo, Concourse etc. Monitoring utilising products such as: Prometheus, Grafana, ELK, filebeat etc. Observability - SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Edge technologies e.g. NGINX, HAProxy etc. Excellent knowledge of YAML or similar languages It is essential for this ...

data solutions across the full life cycle, including data acquisition, integration, storage, processing and analysis. Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL . Design data models, schemas, data contracts and Interface Control Documents (ICDs) to support reliable system integration. Integrate data from … Data engineering within a project or production environment. Strong Python and SQL skills, including developing, testing and optimising data pipelines. Hands-on experience with Apache NiFi for data ingestion, routing, transformation and orchestration. Data modelling and schema design, including data contracts or Interface Control Documents (ICDs). Parquet ...

Responsibilities Design, build and maintain secure and scalable data engineering solutions. Develop and maintain data ingestion and transformation pipelines using Python, SQL and Apache NiFi . Design data models, schemas and data contracts/Interface Control Documents (ICDs) . Integrate data from REST APIs and other source systems. Work … maintainable. Essential Experience Proven commercial experience delivering Data Engineering solutions in project or production environments. Strong Python and SQL skills. Hands-on experience with Apache NiFi and data pipeline orchestration. Data modelling and schema design. Parquet and S3-compatible object storage . REST API integration. OpenShift or Kubernetes . ...