5 of 5 Apache Spark Jobs in the North West

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable ...

grade ML workflows. Experience designing CI/CD pipelines using Azure DevOps, GitHub Actions or similar tools. Advanced Python skills, with knowledge of SQL, Spark and testing approaches for ML. Understanding of data engineering concepts such as ETL/ELT, feature stores and lineage tracking. Awareness of security, governance ...

including data modelling, DAX and semantic model design Hands-on experience with Microsoft Fabric, particularly lakehouse architecture, notebooks and data pipelines Strong understanding of Apache Spark/PySpark and large-scale data processing Experience designing and operating modern data platforms Experience developing or enabling AI and machine learning ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...