151 to 175 of 466 Apache Spark Jobs in the UK

with Microsoft Azure a plus: Azure SQL Database, Analysis Services, Databricks, Data Lake, Logic Apps and Data Factory Understanding of Big Data technologies (Hadoop, Spark) Able to create rich visualisations that appropriately represent the underlying datasets. Review and align the reporting attributes against the Enterprise and conceptual data models ...

support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas to maintain data integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining … ease. Understanding of Data Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using ...

into a coherent operational story. Data engineering & processing systems: You have built, operated, or supported production data pipelines using distributed processing systems such as Apache Spark or Databricks, and understand common failure modes, performance tradeoffs, and operational challenges in batch and hybrid pipelines. Data platforms & analytics systems ...

Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts across financial ...

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

modeling and data transformation techniques; Knowledge of maintaining high data quality and integrity; Strong software development skills with Hadoop technologies such as Hive and Spark; Experience with Workday Studio and managing code within the Workday environment; Knowledge of SaaS applications such as Salesforce or other CRM systems; Knowledge ...

cloud platforms (GCP or Azure) and Databricks. Familiarity with deploying ML workflows using MLflow, Vertex AI, or Azure ML. Nice‐to‐have: Experience with Spark, CI/CD pipelines, and orchestration tools. Nice‐to‐have: Knowledge of Elasticsearch or digital/web analytics platforms. Nice‐to‐have: Understanding ...

complex joins, window functions, aggregation, and query optimization; Experience with a major data warehouse or big data technology such as BigQuery, Snowflake, Redshift, Hive, Spark, Vertica, or StarRocks; Experience with ETL/ELT tools or orchestration frameworks such as Airflow, dbt, or internal frameworks; Proficiency in at least ...

concept to production. Strong knowledge of Python and core ML libraries (e.g., spaCy, NumPy, SciPy, Transformers). Understanding of big data and cloud technologies (Spark, Hadoop, Azure, AWS, or GCP). Deep expertise in MLOps principles, scalable model deployment, and model governance. Excellent communication and stakeholder management skills with ...

alerting for ML systems (model drift, data quality, feature coverage, etc.) Design and scale data ingestion and feature transformation flows using batch (e.g., Spark/BigQuery) and streaming (Kafka or equivalent) Contribute to internal Python libraries and platform tooling that accelerate experimentation and deployment for all model teams Ensure ...

modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK — using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. Cloud-native AI engineering and large-scale data platforms, leveraging Google Cloud technologies and modern ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

including migration from legacy data platforms, lakehouse modernisation, performance optimisation, governance, and production‐grade delivery. Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data & AI Intelligence capabilities. Strong experience defining ...

embedded into business processes and decision-making.Technical Skills:Hands-on experience in one or more of the following areas:Programming & Analytics – Python, SQL, R, Spark, machine learning and advanced analyticsCloud & Data Platforms – Microsoft Fabric, Databricks, Snowflake, Azure, AWS and GCPAI Technologies – GenAI frameworks, Agentic AI, vector databases, semantic search ...

Data & AI programmes, including planning, mobilisation, resourcing, financial management, delivery governance, risk and dependency management. Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data & AI Intelligence capabilities. Strong experience defining ...

deploy scalable batch and streaming data pipelines, AI-ready data products, feature pipeline and analytics solutions on the Databricks Platform using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL. Deliver solutions across the full data & AI engineering lifecycle, from requirements analysis and solution design through development … scalable data platforms, data products and analytics solutions using modern lake house or cloud data warehouse technologies Proficiency writing advanced, highly optimised SQL and Spark (PySpark/Scala), with a focus on performance optimisation and maintainability. Designing and building solutions on a cloud data warehouse or lakehouse platform (e.g. ...

modern data engineering best practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence ...

skills Desirable Experience Experience with any of the following would be advantageous: Angular, React or Vue Chart.js, D3.js or other data visualisation tools Databricks, Apache Spark or distributed data processing Docker, Kubernetes or Cloud Run Terraform or other infrastructure-as-code technologies GPU or accelerated computing technologies such ...

skills Desirable Experience Experience with any of the following would be advantageous: * Angular, React or Vue * Chart.js, D3.js or other data visualisation tools * Databricks, Apache Spark or distributed data processing * Docker, Kubernetes or Cloud Run * Terraform or other infrastructure-as-code technologies * GPU or accelerated computing technologies such ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

build tools (Maven/Gradle). Desirable skills: Scala: Hands-on experience or a strong functional programming background using Scala. Big Data: Exposure to Apache Spark, Kafka, or Hadoop ecosystems. Cloud: Experience with AWS, Azure, or Google Cloud Platform. Experience in banking or financial services industry. Soft Skills ...

hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested ...