301 to 325 of 372 Apache Spark Jobs in England

collaboration and delivery. What We’re Looking For Strong proficiency in Python, Scala, or Java, with hands-on experience of Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks. Experience building real-time event streaming solutions using technologies such as Kafka, Spark Streaming, or Amazon Kinesis … solutions. Bonus Points For Expertise in data modelling, schema design, and working with structured and semi-structured datasets. Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation. Knowledge of automating ingestion and transformation frameworks within cloud-native data platforms. Experience delivering solutions within highly ...

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

queries over large-scale datasets. Pipeline & Workflow Orchestration: Proven track record building, scheduling, and monitoring production pipelines using modern workflow orchestrators such as Apache Airflow. Systems & Data Architecture Mindset: Experience handling high-throughput event data and a deep understanding of analytical databases, storage formats, and execution plans. Ownership & Pragmatic … schemas. Nice to Have: Hands-on experience with modern data transformation tools like dbt. Familiarity with distributed compute and storage platforms (e.g., Kafka, Spark, Snowflake, or Presto/Trino). Experience in product-led SaaS, fintech, or high-growth tech environments navigating complex event tracking or onboarding funnels (prior ...

Help shape the technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture … programming. Ability to design, build, and maintain distributed data systems in a cloud‐based environment. Hands‐on experience with big data frameworks such as Spark or Databricks. Knowledge of cloud infrastructure (AWS, GCP, or Azure). Judgement to balance scalability, performance, and maintainability. Experience with data modelling and workflow ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

domain data (bets, offers, rewards, digital data) into reusable, well‐defined datasets for downstream consumption. Implement data transformation logic using modern tooling (e.g. dbt, Spark, SQL‐based frameworks), ensuring consistency and testability. Build and optimise streaming data pipelines (Kafka/Flink or equivalent) to enable near real‐time data … maintaining scalable data platforms. Strong SQL and data modelling skills, with experience designing analytical datasets. Experience with modern data stack tools (e.g. dbt, Spark, Airflow, or similar orchestration and transformation frameworks). Experience with cloud‐based data platforms, ideally AWS. Understanding of medallion architecture or similar data layering approaches. ...

shared platform capabilities for data ingestion, transformation, orchestration, and processing across batch and real-time workloads. Own the integration strategy across tools such asdbt, Spark and Beam Lead the implementation and adoption of data quality, observability, security, testing and deployment including creating and sharing standards Partner closely with Analytics … excellence, and supporting others Hands on experience transitioning to an AI-native Data Engineering landscape A strong Python and SQL engineer with experience usingdbt, Spark and Terraform Strong experience working with GCP (or a comparable cloud platform), Infrastructure as Code, and CI/CD practices. Deep understanding of data ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deploymentsWorking knowledge of MLOps, ML/AI models ...

Hadoop ecosystem, collaborating with data scientists to operationalize ML models using CML. The role spans data ingestion, transformation, and full-stack tooling with Java, Spark, Python, Flask, and React, emphasizing performance tuning and efficient resource use across batch and streaming workloads. #J-18808-Ljbffr ...

seeking a Secure Data Engineer in Newcastle. This role focuses on building scalable and secure data platforms for Defence clients using Azure, Databricks, and Apache Spark. The successful candidate will engage in developing robust data applications, ensuring high security and compliance standards. Opportunities for professional growth and a flexible ...

multi-petabyte data estate, processing billions of datapoints daily from thousands of sources. The stack includes Java and Python, with Kafka, ArcticDB, Airflow and Spark, tackling complex distributed systems at scale. You will collaborate with quants and traders to translate requirements into robust platform solutions, while improving reliability ...

Building streaming data pipelines and event-driven systems Kafka, Pulsar, NATS, RabbitMQ, or similar messaging platforms Designing distributed schedulers, queues, and asynchronous processing systems Spark, Flink, Beam, or MapReduce Ad tech, social networks, search engines, or other large-scale content platforms We conduct coding interviews as part ...

bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. RESPONSIBILITIES Recruit, hire, and grow Baseten’s first SRE team ...

scale estate, processing billions of datapoints daily from thousands of data sources. Our stack combines Java and Python, with foundations of Kafka, ArcticDB, Airflow, Spark, and more. This is an opportunity to tackle complex distributed systems challenges at an exceptional scale across thousands of unique datasets. AI agents ...

scale estate, processing billions of datapoints daily from thousands of data sources. Our stack combines Java and Python, with foundations of Kafka, ArcticDB, Airflow, Spark, and more. This is an opportunity to tackle complex distributed systems challenges at an exceptional scale across thousands of unique datasets. AI agents ...

Elasticsearch, Kafka, and more across different environments Technologies We Use Different backend languages, including Java, Rust, and Go Open-source technologies like Cassandra, ElasticSearch, Spark, Kafka, Kubernetes, Flink Industry-standard build tooling, including Gradle and GitHub What We Value Demonstrated ability to collaborate and empathise with a variety ...

society's experience with software. Come join one of the fastest-growing startups, supported by best-in-class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. Pendo Core Values: Bias to Act, Hone Your Craft, The Team is Pendo, and Maniacal Focus. Location: Pendo is a hybrid ...

person attendance expected at least 3 days per week. At Robinhood, we believe in the power of in-person work to accelerate progress, spark innovation, and strengthen community. Our office experience is intentional, energizing, and designed to fully support high-performing teams. Requires participation in an on-call rotation ...

technology platforms in an enterprise environmentTechnical knowledge in data storage, processing and containerisation technology such as Cloud or On-prem Distributed File Systems, Spark, Databricks, Kubernetes, Open Search, Elastic Search.Our perks and quirks.What makes you Q will help you to realize your full potential, flourish and enjoy what ...

Liquidity Architecture Delivery team. You will develop server-side processes and help build a large-scale data platform using Java/Python/Scala, Apache Spark, on AWS/MinIO, delivering data for regulatory reporting. The role requires fluent English, Linux experience, teamwork in a globally distributed group ...

Global Liquidity Architecture Delivery (GLAD) team to build and maintain server-side processes for regulatory and liquidity reporting. The role focuses on Scala and Apache Spark on AWS, with collaboration across a global team. You will work with business users to translate requirements into robust data solutions, contribute ...

team. What We’re Looking For Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience … Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

Develop and maintain production-quality Python and SQL code.* Use Git-based source control and branch management practices across multiple deployment environments.* Work with Spark and PySpark to process and analyse large-scale datasets.* Develop and manage machine learning workflows within an MLOps environment.* Use tooling such as MLflow … Code or JupyterHub for model development, experimentation and deployment.* Work with technologies to access, manage and analyse data at scale (e.g., Apache Iceberg and Trino).**Delivery Excellence*** Participate fully in Agile ceremonies including sprint planning, backlog refinement, demonstrations and retrospectives.* Produce high-quality technical documentation and knowledge-sharing ...

Develop and maintain production-quality Python and SQL code. Use Git-based source control and branch management practices across multiple deployment environments. Work with Spark and PySpark to process and analyse large-scale datasets. Develop and manage machine learning workflows within an MLOps environment. Use tooling such as MLflow … Code or JupyterHub for model development, experimentation and deployment. Work with technologies to access, manage and analyse data at scale (e.g., Apache Iceberg and Trino). Delivery Excellence Participate fully in Agile ceremonies including sprint planning, backlog refinement, demonstrations and retrospectives. Produce high-quality technical documentation and knowledge-sharing ...