9 of 9 Apache Spark Jobs in Bristol

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools ...

within either Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data qualityPreparing data for predictive and prescriptive modellingHands on coding experience ...

Python, PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence … scale distributed data processing and performance tuningHands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar)Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite … compressed hours. The role will include: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security ...

digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite … hours.\ N \nThe role will include:\ N\nDesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.\ NImplement data ingestion, transformation, and integration processes, ensuring data quality and security.\ NCollaborate with data architects and security teams to ensure compliance with ...

ensembles (and experience applying them with packages like Pandas, sci-kit-learn, SciPy) is highly beneficialExperience processing big data, ideally in a Hadoop/Spark environment, would be beneficialProven experience implementing CI/CD and DevOps practices, with the ability to improve deployment reliability, testing standards, and release processes ...

Build and operate batch and streaming data pipelines with strong scalability, performance, and fault toleranceDevelop and manage workflow orchestration using tools such as Apache Airflow to support reliable, observable, and well-scheduled data movement and transformationsImplement and optimize data models and warehouse structures to support analytics and business intelligence … e.g., Terraform) for cloud-based data infrastructure Preferred qualifications, capabilities, and skills Data modeling skills Experience with data streaming and scalable processing frameworks (e.g., Spark, Flink, Beam, or similar)Experience automating deployment, releases, and testing in continuous integration and continuous delivery pipelinesExperience with lakehouse patterns and table formats (e.g. ...

machine learning frameworks and programming languages such as Python, R, or similar. Experience working with large datasets and data processing technologies (e.g., Hadoop, Spark, SQL). Familiarity with anomaly detection, supervised, unsupervised learning methods, deep learning, and graph based solutions Demonstrated ability to work collaboratively in cross-functional teams ...

with stakeholders. Preferred qualifications, capabilities, and skills Experience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray) Experience of big data technologies (e.g. Spark, Hadoop)Have constructed batch and streaming microservices exposed as REST/gRPC endpointsFamiliarity with GraphQL #CIBAppliedAI ABOUT US J.P. Morgan is a global leader ...