20 of 20 Permanent Apache Beam Jobs in the UK

platform at scale, ideally in a consumer or marketplace environment. Deep understanding of distributed systems and modern data ecosystems — including extensive experience with Databricks, Apache Spark, Apache Kafka, Apache Flink, Apache Beam and DBT. Demonstrated success in designing, building, and operating data platforms at scale ...

event-driven technologies, such as:Testing & Automation: PyTest, Cucumber, Behave, DBT, GreatExpectations, GCP DVT, Monte Carlo, Soda, Deequ, RestAssured etcData Engineering & Orchestration: BigQuery, Spanner, Apache Kafka, Airflow, Spark, Cloud Composer, DAGs, Apache Beam, Pub/Sub, Dataflow, DataStage, Teradata, Snowflake, ETL, SQL etcData Governance & Visualisation: Looker, Ataccama ...

Data Engineer within GCP environments Hands-on experience with BigQuery, Dataflow and Spanner Strong programming capability in Python, Java or Scala Experience with Apache Beam/Dataflow and distributed processing frameworks Experience designing and managing ETL/ELT pipelines Solid understanding of data modelling and database design Experience … with workflow orchestration tools such as Apache Airflow Strong understanding of data governance, security and scalability Desirable Skills for the GCP Data Engineer: Experience with Pub/Sub, Cloud Composer or Cloud Data Fusion Exposure to CI/CD, DevOps and Infrastructure as Code Experience with real-time streaming ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code ...

that ClickHouse seamlessly integrates with the tools our users already know and love.We're responsible for the official data connectors (Kafka, dbt, Spark, Flink, Beam, Fivetran), and integrations with leading visualization platforms (Grafana, Tableau, Power BI, Metabase).What will you do As a Senior Software Engineer specializing … focusing on building and delivering high-quality, data-intensive solutions.Proven experience with the internals of streaming or data integration frameworks, with strong preference for Apache Kafka (or similar pub/sub systems), Kafka Connect, Apache Flink or Apache Beam.Experience developing or extending connectors, sinks, or sources ...

ClickHouse seamlessly integrates with the tools our users already know and love. We're responsible for the official data connectors (Kafka, dbt, Spark, Flink, Beam, Fivetran), and integrations with leading visualization platforms (Grafana, Tableau, Power BI, Metabase).What will you do? As a Senior Software Engineer specializing … building and delivering high-quality, data-intensive solutions. Proven experience with the internals of streaming or data integration frameworks, with strong preference for Apache Kafka (or similar pub/sub systems), Kafka Connect, Apache Flink or Apache Beam. Experience developing or extending connectors, sinks, or sources ...

with Python, SQL and modern software engineering practices. Experience building large-scale batch and streaming data solutions using technologies such as Spark, Kafka, Flink, Beam or equivalent. Deep knowledge of modern data architectures including data warehouses, data lakes and lakehouse platforms. Strong understanding of analytical data modelling, dimensional modelling ...

Extensive experience working with operational data stores, data warehouse, large‐scale data technologies and data lakes Experience using distributed frameworks (Spark, Flink, Beam, Hadoop) Good knowledge of containers (Docker, Kubernetes etc) and experience with cloud platforms such as GCP, Azure or AWS Strong experience working with Kafka technologies Clear ...

data ingestion, transformation, orchestration, and processing across batch and real-time workloads. Own the integration strategy across tools such as dbt, Spark and Beam Lead the implementation and adoption of data quality, observability, security, testing and deployment including creating and sharing standards Partner closely with Analytics Engineers, Data Scientists ...

data processing solutions using BigQuery, Pub/Sub, Dataflow and Dataproc Experience developing end‐to‐end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam. Expert understanding of when to use a range of data storage technologies including relational/non‐relational, document ...

Experience with domain-specific workflow systems (e.g., Nextflow) for scaling high-throughput pipeline execution.Familiarity with general-purpose data orchestration and processing frameworks (e.g., Dagster, Apache Beam) for integrating research pipelines into a production platformFamiliarity with building and maintaining bioinformatics infrastructure on Google Cloud Platform (GCP).Familiarity with modern ...

technical program manager. Data Engineering Proficiency : Expertise in SQL, Python, Pandas, Matlab, or R. Bonus if familiarity with Influx, Grafana, Spark, Polars, Arrow, Kafka, Beam, and Flink. [Bonus] In-Field Hardware Mastery : Proficiency with hardware engineering tools such as DAQs, DDS, Nix, RTOS, MCAP, H5, TDMS, LabVIEW, Simulink ...

field. Experience with LLM agents, autonomous multi-step reasoning systems, meta-learning, or self-improving ML pipelines. Experience with large-scale data pipelines (e.g., Apache Beam) or foundation model training and fine-tuning at scale. Experience working in research environments or track record of published research in relevant ...

field. Experience with LLM agents, autonomous multi-step reasoning systems, meta-learning, or self-improving ML pipelines. Experience with large-scale data pipelines (e.g., Apache Beam) or foundation model training and fine-tuning at scale. Experience working in research environments or track record of published research in relevant ...

Google Cloud Platform, Databricks Programming Languages: Python, R, PySpark Agentic & GenAI Platforms: Microsoft Copilot Studio, OpenAI GPT-5 Agents, UiPath, Adept AI, Orby AI, Beam AI If you’ve held any of these roles or used these technologies/skills, this role could be a great fit: Data Scientist ...

Google Cloud Platform, Databricks Programming Languages: Python, R, PySpark Agentic & GenAI Platforms: Microsoft Copilot Studio, OpenAI GPT-5 Agents, UiPath, Adept AI, Orby AI, Beam AI If you've held any of these roles or used these technologies/skills, this role could be a great fit: Data Scientist ...

ready data processing solutions using BigQuery, Pub/Sub, Dataflow and DataprocExperience developing end-to-end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam.Expert understanding of when to use a range of data storage technologies including relational/non-relational, document, row-based ...

data pipelines and event-driven systems Kafka, Pulsar, NATS, RabbitMQ, or similar messaging platforms Designing distributed schedulers, queues, and asynchronous processing systems Spark, Flink, Beam, or MapReduce Ad tech, social networks, search engines, or other large-scale content platforms We conduct coding interviews as part of the process. Benefits ...

skills Degree in CS, maths, statistics, engineering, physics or similar Desirable Requirements: NoSQL databases – Elasticsearch, MongoDB...etc (bonus) Modern Data tools such as Spark/Beam (bonus) Streaming technologies such as Spark/Akka Streams (bonus) Tagged as: Industry, NLP, United Kingdom #J-18808-Ljbffr ...