426 to 450 of 569 Apache Spark Jobs in the UK

sportsbook domain data (bets, offers, rewards, digital data) into well-structured datasets for downstream consumption. Implement data transformation logic using modern tooling (e.g. dbt, Spark, SQL-based frameworks) with a focus on clarity and reliability. Contribute to streaming data pipelines (Kafka/Flink or equivalent) to support near real … data engineering or a related field. Strong SQL skills and a good understanding of data modelling principles. Experience with data transformation tools (e.g. dbt, Spark) and workflow orchestration (e.g. Airflow or similar). Familiarity with cloud-based data platforms, ideally AWS. Understanding of data pipeline design and ETL/ ...

changed how you work, not just how fast. We expect you to enjoy leveraging AI and agentic systems to operate more effectively. Tech environmentScala | Spark | Databricks | AWS | Claude | Airflow | Kubernetes | Terraform | Functional ProgrammingYou will thrive here if youEnjoy end-to-end ownership, from writing the code to architecture design … Python. Ability to design, build, and maintain distributed data systems in a cloud-based environment. Hands-on experience with big data frameworks such as Spark or Databricks. Knowledge of cloud infrastructure (AWS, GCP, or Azure).Judgement to balance scalability, performance, and maintainability. Experience with data modelling and workflow orchestration. ...

The opportunity We are investing in the next generation of our data and technology capabilities to create more connected, intelligent experiences for customers. Data is central to this ambition. This is an opportunity to define ...

Python is required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation ...

Python is required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation ...

Automated system deployment using tools such as Ansible, Puppet or ChefDesirable Skills/Experience Familiar with the Big Data tech stacks, including: o Trinoo Spark o Apache Hiveo Apache Airflowo Zeppelin notebooksOur PackageTailored benefits make a real difference. That's why we offer a comprehensive range ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive ...

Good to Have Knowledge of the adTech space and particularly the identifiers used in the field. Experience with large‐scale distributed data processing (e.g., Apache Spark) and cloud data infrastructure across GCP and AWS (e.g., Dataproc, Dataflow, GKE, EMR). Experience with Temporal More about us: LiveRamp ...

seeking a Secure Data Engineer in Newcastle. This role focuses on building scalable and secure data platforms for Defence clients using Azure, Databricks, and Apache Spark. The successful candidate will engage in developing robust data applications, ensuring high security and compliance standards. Opportunities for professional growth and a flexible ...

events daily, powering trading and analytics. You will guide engineering teams in building scalable, self-service capabilities with modern data lakehouse tech, including Iceberg, Spark, Trino, and real-time streaming with Kafka and Flink. You will influence technology strategy across Trading Tech, Data Engineering and Platform Engineering, while ensuring ...

will lead requirements for the Treasury Analytics Group, support FIS Quantum & TBSM, and work on a new risk/analytics platform using Hadoop, Spark, and NoSQL tools. The role reqs 5+ years of experience, Linux/Oracle/MS SQL, ITIL, and scripting with Java/Python. Collaboration with ...

Citi is hiring for a Support Engineer to join a global team maintaining a large-scale data platform built on Hadoop, Hive, Spark, Kafka, and related tools. The role supports 24/7 availability with a follow-the-sun model and cross-region collaboration to keep systems running smoothly. ...

commercial Python development experience. Experience building or working with RESTful microservices. Experience working with large-scale datasets. Exposure to big data technologies such as Spark, Kafka, or similar. Previous experience within financial services is advantageous but not essential. A strong academic background, ideally with a degree in Computer Science ...

/or Parity Experience in using various Solidity frameworks like Truffle, Mix IDE or Intellij-Sol. Contributions to relevant open source projects; Experience Hadoop, Spark, or Docker. Basic or deep knowledge of Ethereum Whisper or IPFS/Swarm. My client is based in the City of LondonPython, Ethereum, Blockchain ...

depth actually gets used. • AWS across the board, including S3, Glue, Redshift, Lake Formation and Kinesis • Databricks and Snowflake running on top of AWS • Spark and distributed processing frameworks for large-scale data • Streaming technologies such as Kinesis or Kafka You're expected to pick the right tools ...

Building streaming data pipelines and event-driven systems Kafka, Pulsar, NATS, RabbitMQ, or similar messaging platforms Designing distributed schedulers, queues, and asynchronous processing systems Spark, Flink, Beam, or MapReduce Ad tech, social networks, search engines, or other large-scale content platforms We conduct coding interviews as part ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything-spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

conducted for several leading frontier model developers and national safety institutes, and has been featured in OpenAI's o1 system card and Meta Muse Spark's Safety & Preparedness Report. About the roleAs a Senior Research Scientist at Faculty, you will join our high-impact R&D team to lead ...

Elasticsearch, Kafka, and more across different environments Technologies We Use Different backend languages, including Java, Rust, and Go Open-source technologies like Cassandra, ElasticSearch, Spark, Kafka, Kubernetes, Flink Industry-standard build tooling, including Gradle and GitHub What We Value Demonstrated ability to collaborate and empathise with a variety ...

team. What We’re Looking For Proficient in one of Python, Scala or Java with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience … Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

London --------------------------------------- JOB DETAILS Role Title: Data Engineer (Python Enterprise Developer) Must have skills: -Python, Postgres SQL, Azure Databricks, AWS(S3), Git, Azure DevOps CICD, Apache Airflow -Preferred Domain: Energy trading Minimum number of relevant years of experience: 8+ Detailed Job Description: JD: • 8+ years of experience in python scripting. … building native python ETL pipelines for scraping/processing. • Good to have experience in processing big data. • Proficient in SQL programming, Postgres SQL. Spark knowledge is a good to have • Knowledge on DevOps CI/CD like Azure Pipelines, Jenkins, Git. • Experience working with AWS services(S3, lambda ...

remediation, or automation programs (preferably in financial services). Strong proficiency in Python for data engineering and automation. Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases. Experience designing and supporting …/ELT pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of Autosys and Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift or equivalent). Familiarity with ...

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of scheduling/orchestration tools such as Autosys and/or Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift or equivalent … reporting outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Exposure to data governance concepts: critical data elements (CDEs), lineage, data quality dimensions, audit frameworks. Experience ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering … Engineering, or a quantitative discipline (or equivalent experience) Nice to Have: Databricks certification or experience with the Databricks Platform Experience with Unity Catalog, Lakeflow Spark Declarative Pipelines, or MLflow Background at a data/AI company, cloud provider, or technical consulting firm Interview Process Recruiter Screen → Hiring Manager Screen ...