2 of 2 Remote Apache Spark Jobs in Doncaster

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

technology (AWS or Azure). Strong experience with creation of data ingestion pipeline and ET process. Strong knowledge of big data tool such as Spark, Databricks and Python. Strong understanding of common machine learning techniques and frameworks (e.g. mlflow). Strong knowledge of Natural language processing (NPL) concepts. Strong ...