501 to 525 of 569 Apache Spark Jobs in the UK

validate transformations and trace data across platforms. Tooling/Delivery Methods Working knowledge of scheduling/orchestration tools such as Autosys and/or Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift … reporting outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Hands-on exposure to DCRM tooling and operational exception management processes. Experience with governance/catalog ...

based systems, summarization/extraction pipelines, chat/coplay experiences, and tool-using agents. Proven experience working with large datasets and distributed compute (Spark/Databricks or equivalent), with SQL fluency and an understanding of partitioning, performance, and cost. Hands-on experience using enterprise-authorized AI-assisted software development … Application Resiliency, and Security. Preferred Qualifications, Capabilities, And Skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Experience with evaluation frameworks and approaches (golden datasets, LLM-as-judge, human-in-the-loop review, red teaming). Solid grounding in data ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern web applications. With a globally distributed presence across the US, India, and Europe, the team thrives on collaboration, bringing together diverse perspectives … Computer Science or equivalent practical experience Experience developing finance applications is a plus Working experience with Terraform is a plus Experience creating workflows for Apache Airflow is a plus What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role … focuses on integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such ...

rapidly evolving world of AI. You will have the opportunity to: Design and implement robust data pipelines and ML workflows using Python, SQL, Spark and Databricks. Develop and deploy machine learning models, including NLP, deep learning and agentic AI, into production environments. Integrate data from diverse sources through both … years experience with large-scale data challenges Hands-on with Infrastructure-as-Code (Terraform, Ansible) Agile and Waterfall project experience Experience withPython, SQL, Spark, Scala What you will gain from working here: Competitive salary 25 days annual leave Private medical Career progression Extensive Training If youre passionate about data ...

gambling industry. Your work will involve shaping lakehouse and data-lake paradigms, leveraging Parquet and Iceberg for high-performance data storage, and adopting scalable Spark frameworks for large-scale processing. Ultimately, you will enable self-service analytics, personalised user experiences, and an AI-first approach to data-driven decision … DevOps practices, ensuring compliance with FDJ United’s security and regulatory standards. Modernising Data Pipelines & Workflows: Implement Scalable Frameworks, orchestrate large-scale pipelines using Spark, Docker, Kubernetes, and other DevOps tools. Accelerate Delivery: Apply standardised processes and automated testing to streamline development and reduce time-to-market while managing ...

programs. This is a hands-on leadership role for a builder who can equally command a whiteboard in front of a CTO and a Spark UI when a pipeline misbehaves. You will also shape the practice itself — mentoring engineers, creating accelerators and reusable assets, and converting delivery success into … heart — 12+ years in data engineering/architecture, 3+ on Databricks at enterprise scale, and still happiest when hands are on the keyboard. Deep Spark expertise — architecture, performance tuning, streaming, debugging, the advanced stuff that separates architects from diagram-drawers. Lakehouse fluency — Medallion architecture, Delta Lake, Unity Catalog, governance ...

quality datasets that underpin analytics, AI initiatives, and business decision-making. The emphasis is on data discovery, data architecture, and data modelling rather than Spark development, dashboard engineering, or visualisation work. Responsibilities Identify, analyse, and document data sources required to meet business and application requirements for the Operational Resilience … documentation, data governance, and high standards of data quality. Work is focused on data architecture, modelling, and integration rather than front-end visualisation or Spark-based development, enabling you to specialise in core data engineering and modelling disciplines. 2 days a week in Sheffield required Inside IR35 Umbrella rate ...

About Us The Company: Dotdigital is a thriving global community of passionate, dedicated professionals, committed to the collective success of the organization and its clients. Our core principles of innovation, teamwork, and client-focused solutions ...

building Kraken's real-time streaming infrastructureOwn the architecture and roadmap for high-volume low-frequency data systems, with focus on data stack like Spark, Kafka, Iceberg, RisingWave, Apache FlinkDesign and operate scalable data architecture that serve trading, risk, compliance, analytics and many product teams. Drive adoption … data engineering, platform engineering, or distributed systems — with at least 3 years managing engineering teamsExperience and knowledge of building data-lakes in AWS (i.e. Spark, Athena, Iceberg, Parquet, Presto), including data modeling, data quality best practices, and self-service tooling. Strong expertise in building and operating real-time data ...

Location: Northampton, UK --------------------------------------- JOB DETAILS Role Title: Sr Data Engineer • Minimum 6-8 years of experience working as Sr Data Engineer • Bigdata Technologies – PySpark, Spark • Coding Languages – Python, SQL • AWS services – Glue, Cloud formation, Lake Formation, IAM, Lambda, Redshift • Exposure to cloud technologies – Snowflake, Redshift, Apache Flink … Apache Kafka. • Work in an Agile environment and participation in scrum daily standups, sprint planning reviews and retrospectives. • Understand project requirements and translate them into technical solutions which meets the project quality standards • Ability to work in team in diverse/multiple stakeholder environment and collaborate with upstream/ ...

customers. The role requires 5+ years of production-grade software experience, proficiency in Kotlin/Java/Go, and hands-on work with Kafka, Spark/Beam, and cloud warehouses. #J-18808-Ljbffr ...

improvements in tooling and processes. Ideal candidates will have 5+ years in backend engineering for data systems, strong experience with MongoDB, Postgres, Kinesis, Flink, Spark, ElasticSearch, Redis, and query engines; plus #J-18808-Ljbffr ...

Hyperledger framework would be very appreciated A first experience using Mobile technologies in Android/iOS environment, Big Data technologies such as Cassandra/Spark/NoSQL is an asset Work within our team of passionate and talented developers Design, implement and test platforms and Apps Deal with cryptocurrencies ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking ...

Azure PaaS components, including Azure DevOps, Vault, and other security and configuration services (AKS experience is beneficial) Integrate and optimize data processing frameworks with Spark, Databricks, or Cloudera platform (Hue, Hive, HDFS) Support ETL pipelines built for master data management, customer profiles, and consent management workflows Ensure environments … release cycles Exposure to Azure Cloud and PaaS services, including Azure DevOps and Vault Familiarity with data engineering and big data ecosystems, including Spark and Cloudera stacks (hue, hive, HDFS) Excellent problem‐solving skills and ability to communicate technical solutions effectively Previous experience working on data‐driven projects ...

equivalent platforms, with the ability to partner with technology teams to shape architecture and drive enterprise adoption. Proficiency across Python, q/KDB, SQL, Spark, C#, and related engineering toolsets, with sufficient depth to guide technical decisions, challenge design options, and work effectively with engineering teams to deliver scalable … Science, or related discipline. Nice to Have Experience integrating real-time market data into Align, Endur, and Front Office trading platforms. Familiarity with Flink, Spark Streaming, and modern stream-processing technologies. Knowledge of execution algorithms, quantitative trading strategies, and best execution frameworks. Experience with MDSL, TRG Screen, DACS ...

Location: London, UK At Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer‐centric mindset guides us to create a unified platform that makes ...

At Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified platform that makes data science and ...

REQ ID: FEQ327R160Location: London, UKRecruiter: Dina HussainAt Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified ...

needs of the department. This is an exciting opportunity to work on cutting edge technologies including Big Data and NoSQL databases (Hadoop, HBase, Hive, Spark and MongoDB) to allow the business to gain advanced insight into their portfolios and valuation metrics. Key responsibilities include:* Lead business requirements elicitation, analysis … documentation efforts* Assist technology team in Azure cloud solution implementation; write Spark SQL queries to analyze and transform data; provide feedback to User Interface team; testing and documentation* Analyze solution requirements; understand project scope; determine project priorities; ensure efficient and on-time delivery of project tasks and milestones; follow ...

Applying advanced techniques in LLMs, agentic systems, vector databases, fine‐tuning, and deployment tools (e.g., HuggingFace, Langchain). Hands‐on experience working with Distributed Spark based systems. Experience with data engineering concepts or a good understanding of data engineering concepts. Pre‐sales or post‐sales experience working with external … clients across a variety of industry markets. Minimum of 5+ years of customer‐facing experience would be preferred. [Preferred] Experience working with Apache SparkTM to process large‐scale distributed datasets. Communication: Proven ability to communicate and teach complex technical concepts to both technical and non‐technical audiences. Core Traits ...

Applying advanced techniques in LLMs, agentic systems, vector databases, fine-tuning, and deployment tools (e.g., HuggingFace, Langchain). Hands-on experience working with Distributed Spark based systems Experience with data engineering concepts or a good understanding of data engineering concepts Pre-sales or post-sales experience working with external … clients across a variety of industry markets. Minimum of 5+ years of customer-facing experience would be preferred [Preferred] Experience working with Apache SparkTM to process large-scale distributed datasets Communication: Proven ability to communicate and teach complex technical concepts to both technical and non-technical audiences. Core Traits ...

strategy and promote best practices amongst engineers and data scientists. How to make an Impact Architect and Implement : Own the technical roadmap for our Spark-based data processes to ensure our Airflow pipelines are performant, cost-effective, and scalable. You will enhance our existing services through hands-on development … deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...