2 Permanent Batch Processing Jobs in the Midlands

deadlines. Your main responsibilities will be: Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using Databricks and Spark Streaming and Batch processes in Databricks SPARK performance\tuning\optimisation Providing technical guidance for complex geospatial problems and spark dataframes Developing scalable and re-usable frameworks for more »

Experience with Apache Spark and/or Databricks. Familiarity with BI visualization tools like Power BI Experience in managing end-to-end analytics pipelines (batch and streaming) Proficiency in root cause analysis and post-incident reporting in a production environment Excellent interpersonal skills Strong report writing, presentation, and record … Computer Science, Engineering, or related field Certifications such as Azure Data Engineer Associate are desirable. Knowledge of data ingestion methods for real-time and batch processing Proficiency in PySpark and debugging Apache Spark workloads. What’s in it for you? Annual bonus scheme – up to 10% Excellent pension more »