3 of 3 Apache Spark Jobs in the East of England

source systems into our reporting solutions. Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools like Apache Spark to ensure seamless data flow. Monitoring and Failure Recovery: Implement monitoring procedures to detect failures or unusual data profiles and establish recovery processes to maintain data integrity. … in Azure data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Lake Storage/One Lake, and Azure Data Factory. Understanding of data extraction from vendor REST APIs. Spark/Pyspark or Python skills a bonus or a willingness to develop these skills. Experience with monitoring and failure recovery in data pipelines. Excellent problem-solving skills and attention More ❯

solutions from structured and unstructured data. Build data pipelines, models, and AI applications, using cloud platforms and frameworks such as Azure AI/ML Studio, AWS Bedrock, GCP Vertex, Spark, TensorFlow, PyTorch, etc. Build and deploy production grade fine-tuned LLMs and complex RAG architectures. Create and manage the complex and robust prompts across the GenAI solutions. Communicate effectively More ❯

experience deploying and maintaining machine learning models (transformer based models in production is a plus), including identifying the right KPIs and objective functions Experience working with big data systems (Spark, EMR, S3, Airflow) and programming languages (Java, Python, C++) Experience building in-production NLU and/or ASR systems Bachelor Degree required. MS in Computer Science or a Ph.D. More ❯