6 of 6 Apache Spark Jobs in Bristol

architecture (SQL and NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team ...

Python, PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence … scale distributed data processing and performance tuningHands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar)Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite … compressed hours. The role will include: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security ...

digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite … hours.\ N \nThe role will include:\ N\nDesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.\ NImplement data ingestion, transformation, and integration processes, ensuring data quality and security.\ NCollaborate with data architects and security teams to ensure compliance with ...

ensembles (and experience applying them with packages like Pandas, sci-kit-learn, SciPy) is highly beneficialExperience processing big data, ideally in a Hadoop/Spark environment, would be beneficialProven experience implementing CI/CD and DevOps practices, with the ability to improve deployment reliability, testing standards, and release processes ...

Build and operate batch and streaming data pipelines with strong scalability, performance, and fault toleranceDevelop and manage workflow orchestration using tools such as Apache Airflow to support reliable, observable, and well-scheduled data movement and transformationsImplement and optimize data models and warehouse structures to support analytics and business intelligence … e.g., Terraform) for cloud-based data infrastructure Preferred qualifications, capabilities, and skills Data modeling skills Experience with data streaming and scalable processing frameworks (e.g., Spark, Flink, Beam, or similar)Experience automating deployment, releases, and testing in continuous integration and continuous delivery pipelinesExperience with lakehouse patterns and table formats (e.g. ...