3 of 3 Apache Spark Jobs in the East of England

influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing … Engineer or Solutions Architect certification (desirable). Databricks Certified Data Engineer or Machine Learning Associate (desirable). Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable). Knowledge of machine learning and AI on Databricks (desirable). Benefits of working at Canada Life We believe in recognising ...

scripting skills in Python for automation, logging, and observability integrations. Expertise in SQL, especially Redshift, and in managing large-scale data processing frameworks like Spark and EMR. Hands-on experience with containerization (Docker, Kubernetes). Professional certifications such as AWS Certified Solutions Architect and AWS Certified DevOps Engineer … provisioning automation. Nice to Have Skills: Experience with Power BI, DAX, and data modeling for optimized reporting. Knowledge of big data orchestration tools like Apache Airflow. Previous exposure to regulated environments such as finance, healthcare, or gaming. Experience in a lottery or similar domain is a plus. Preferred Education ...

locally-hosted solutions. Technical Requirements Languages Expert-level Python and advanced SQL . Data Engineering Experience with ETL/ELT and orchestration tools like Apache Airflow or NiFi . On-Prem Tech Proficiency with Hadoop/HDFS , Spark , and containerization via Docker/Kubernetes (K3s/OpenShift) . ...