4 of 4 Permanent Apache Spark Jobs in Surrey

software platforms adheres to IDBS's architecture vision. What We'll Get You Doing Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark (PySpark) to support analytics and other data-driven initiatives. Support the elaboration of requirements, formulation of the technical implementation plan and backlog … refinement. Provide technical perspective to products enhancements & new requirements activities. Optimize Spark-based workflows for performance, scalability, and data integrity, ensuring alignment with GxP and other regulatory standards. Research, and promote new technologies, design patterns, approaches, tools and methodologies that could optimise and accelerate development. Apply strong software engineering ...

technology (AWS or Azure). Strong experience with creation of data ingestion pipeline and ET process. Strong knowledge of big data tool such as Spark, Databricks and Python. Strong understanding of common machine learning techniques and frameworks (e.g. mlflow). Strong knowledge of Natural language processing (NPL) concepts. Strong ...

focused organization Knowledge of LLMs, embeddings, and modern ML techniques for text, images, and multimodal data Experience working with big data ecosystems (Spark, Ray, Snowflake, BigQuery, etc.) Familiarity with statistical modeling frameworks such as Bayesian methods or probabilistic programming Why Join Gain exposure to cutting-edge AI research workflows ...

your specific data industry experience to keep our curriculum cutting-edge. This involves iterating on materials to reflect modern stacks (e.g. cloud data engineering, Apache Airflow, Kafka). You will collaborate with the Curriculum team to design highly effective educational experiences that give learners the AI and Data skills … data modelling, and managing data warehouses (experience with DBT is a plus). Pipelines & Orchestration: Familiarity with building data pipelines, covering batch processing (e.g., Apache Airflow) and an understanding of stream processing (e.g., Kafka/Spark). Data Statistics & Analytics: A strong grasp of statistical concepts (distributions, regression ...