26 to 50 of 138 Apache Beam Jobs

relational and non-relational databases to build data solutions, such as SQL Server/Oracle , experience with relational and dimensional data structures. Experience in using distributed frameworks ( Spark, Flink, Beam, Hadoop ). Proficiency in infrastructure as code (IaC) using Terraform . Experience with CI/CD pipelines and related tools/frameworks. Containerisation: Good knowledge of containers ( Docker, Kubernetes … AWS, or Azure . Good understanding of cloud storage, networking, and resource provisioning. It would be great if you had... Certification in GCP “Professional Data Engineer”. Certification in Apache Kafka (CCDAK). Proficiency across the data lifecycle. Working for us: Our focus is to ensure we are inclusive every day, building an organisation that reflects modern society and More ❯

relational and non-relational databases to build data solutions, such as SQL Server/Oracle , experience with relational and dimensional data structures. Experience in using distributed frameworks ( Spark, Flink, Beam, Hadoop ). Proficiency in infrastructure as code (IaC) using Terraform . Experience with CI/CD pipelines and related tools/frameworks. Containerisation Good knowledge of containers ( Docker, Kubernetes … AWS, or Azure . Good understanding of cloud storage, networking, and resource provisioning. It would be great if you had... Certification in GCP “Professional Data Engineer”. Certification in Apache Kafka (CCDAK). Proficiency across the data lifecycle. WORKING FOR US Our focus is to ensure we are inclusive every day, building an organisation that reflects modern society and More ❯

Built platforms that become reusable foundations for others Technical Capabilities 6+ years experience in data engineering or distributed systems Deep proficiency in Python and SQL Fluent in tools like Apache Spark , Beam , Flink , or Kafka Experience with cloud data warehouses (BigQuery, Snowflake) and orchestration tools (Airflow, Prefect, Dagster) Understanding of data lakehouse patterns (Iceberg, Delta Lake), vector stores More ❯

data programs. 5+ years of advanced expertise in Google Cloud data services: Dataproc, Dataflow, Pub/Sub, BigQuery, Cloud Spanner, and Bigtable. Hands-on experience with orchestration tools like Apache Airflow or Cloud Composer. Hands-on experience with one or more of the following GCP data processing services: Dataflow (Apache Beam), Dataproc (Apache Spark/Hadoop … or Composer (Apache Airflow). Proficiency in at least one scripting/programming language (e.g., Python, Java, Scala) for data manipulation and pipeline development. Scala is mandated in some cases. Deep understanding of data lakehouse design, event-driven architecture, and hybrid cloud data strategies. Strong proficiency in SQL and experience with schema design and query optimization for large datasets. More ❯

data programs. 5+ years of advanced expertise in Google Cloud data services: Dataproc, Dataflow, Pub/Sub, BigQuery, Cloud Spanner, and Bigtable. Hands-on experience with orchestration tools like Apache Airflow or Cloud Composer. Hands-on experience with one or more of the following GCP data processing services: Dataflow (Apache Beam), Dataproc (Apache Spark/Hadoop … or Composer (Apache Airflow). Proficiency in at least one scripting/programming language (e.g., Python, Java, Scala) for data manipulation and pipeline development. Scala is mandated in some cases. Deep understanding of data lakehouse design, event-driven architecture, and hybrid cloud data strategies. Strong proficiency in SQL and experience with schema design and query optimization for large datasets. More ❯

data programs. 5+ years of advanced expertise in Google Cloud data services: Dataproc, Dataflow, Pub/Sub, BigQuery, Cloud Spanner, and Bigtable. Hands-on experience with orchestration tools like Apache Airflow or Cloud Composer. Hands-on experience with one or more of the following GCP data processing services: Dataflow (Apache Beam), Dataproc (Apache Spark/Hadoop … or Composer (Apache Airflow). Proficiency in at least one scripting/programming language (e.g., Python, Java, Scala) for data manipulation and pipeline development. Scala is mandated in some cases. Deep understanding of data lakehouse design, event-driven architecture, and hybrid cloud data strategies. Strong proficiency in SQL and experience with schema design and query optimization for large datasets. More ❯

to ensure code is fit for purpose Experience that will put you ahead of the curve Experience using Python on Google Cloud Platform for Big Data projects, BigQuery, DataFlow (Apache Beam), Cloud Run Functions, Cloud Run, Cloud Workflows, Cloud Composure SQL development skills Experience using Dataform or dbt Demonstrated strength in data modelling, ETL development, and data warehousing More ❯

the next generation of personalized generative voice products at scale. What You'll Do Build large-scale speech and audio data pipelines using frameworks like Google Cloud Platform and Apache Beam Work on machine learning projects powering new generative AI experiences and helping to build state-of-the-art text-to-speech models Learn and contribute to the … distributed systems such as Hadoop, BigTable, Cassandra, GCP, AWS, or Azure You have experience with one or more higher-level Python or Java-based data processing frameworks such as Beam, Dataflow, Crunch, Scalding, Storm, Spark, Flink, etc. You have strong Python programming abilities Experience using pre-trained ML models is a plus You might have worked with Docker as More ❯

partitioning, clustering, materialized views, and optimized SQL transformations. • Automate and schedule ETL/ELT workflows with DBT, and Airflow Workflows. • Develop real-time and batch data pipelines using Dataflow, Apache Beam, and DBT for streaming and structured data ingestion. • Define and enforce data architecture standards, including naming conventions, data modelling best practices, and schema evolution policies. • Lead architectural More ❯

applications and high proficiency SQL for complex querying and performance tuning. ETL/ELT Pipelines: Proven experience designing, building, and maintaining production-grade data pipelines using Google Cloud Dataflow (Apache Beam) or similar technologies. GCP Stack: Hands-on expertise with BigQuery , Cloud Storage , Pub/Sub , and orchestrating workflows with Composer or Vertex Pipelines. Data Architecture & Modelling: Ability More ❯

applications and high proficiency SQL for complex querying and performance tuning. ETL/ELT Pipelines: Proven experience designing, building, and maintaining production-grade data pipelines using Google Cloud Dataflow (Apache Beam) or similar technologies. GCP Stack: Hands-on expertise with BigQuery , Cloud Storage , Pub/Sub , and orchestrating workflows with Composer or Vertex Pipelines. Data Architecture & Modelling: Ability More ❯

applications and high proficiency SQL for complex querying and performance tuning. ETL/ELT Pipelines: Proven experience designing, building, and maintaining production-grade data pipelines using Google Cloud Dataflow (Apache Beam) or similar technologies. GCP Stack: Hands-on expertise with BigQuery , Cloud Storage , Pub/Sub , and orchestrating workflows with Composer or Vertex Pipelines. Data Architecture & Modelling: Ability More ❯

applications and high proficiency SQL for complex querying and performance tuning. ETL/ELT Pipelines: Proven experience designing, building, and maintaining production-grade data pipelines using Google Cloud Dataflow (Apache Beam) or similar technologies. GCP Stack: Hands-on expertise with BigQuery , Cloud Storage , Pub/Sub , and orchestrating workflows with Composer or Vertex Pipelines. Data Architecture & Modelling: Ability More ❯

applications and high proficiency SQL for complex querying and performance tuning. ETL/ELT Pipelines: Proven experience designing, building, and maintaining production-grade data pipelines using Google Cloud Dataflow (Apache Beam) or similar technologies. GCP Stack: Hands-on expertise with BigQuery , Cloud Storage , Pub/Sub , and orchestrating workflows with Composer or Vertex Pipelines. Data Architecture & Modelling: Ability More ❯

ML/DL libraries like TensorFlow, PyTorch, or JAX. Knowledge of data analytics concepts, including data warehouse technical architectures, ETL and reporting/analytic tools and environments (such as Apache Beam, Hadoop, Spark, Pig, Hive, MapReduce, Flume). Customer facing experience of discovery, assessment, execution, and operations. Demonstrated excellent communication, presentation, and problem solving skills. Experience in project More ❯

Kafka Focus on disaster recovery aspects Knowledge of Kafka resiliency and new features like KRAFT Experience with real-time technologies such as Spark Required Skills & Experience Extensive experience with Apache Kafka and real-time architecture including event-driven frameworks. Strong knowledge of Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink, and Beam. Experience with cloud platforms such as More ❯

ML/DL libraries like TensorFlow, PyTorch, or JAX. Knowledge of data analytics concepts, including data warehouse technical architectures, ETL and reporting/analytic tools and environments (such as Apache Beam, Hadoop, Spark, Pig, Hive, MapReduce, Flume). Customer facing experience of discovery, assessment, execution, and operations. Demonstrated excellent communication, presentation, and problem solving skills. Experience in project More ❯

ML/DL libraries like TensorFlow, PyTorch, or JAX. Knowledge of data analytics concepts, including data warehouse technical architectures, ETL and reporting/analytic tools and environments (such as Apache Beam, Hadoop, Spark, Pig, Hive, MapReduce, Flume). Customer facing experience of discovery, assessment, execution, and operations. Demonstrated excellent communication, presentation, and problem solving skills. Experience in project More ❯

architectures (Lambda, Fargate, Cloud Run, et al.) and a clear understanding of when not to use them. Experience with message queues (SQS, PubSub, RabbitMQ etc.) and data pipelines (Kafka, Beam, Kinesis, etc.) You are an effective team player with effective communication, presentation and influencing skills. You have a passion for improving coding and development practices. You have worked with More ❯

architectures (Lambda, Fargate, Cloud Run, et al.) and a clear understanding of when not to use them. Experience with message queues (SQS, PubSub, RabbitMQ etc.) and data pipelines (Kafka, Beam, Kinesis, etc.) You are an effective team player with effective communication, presentation and influencing skills. You have a passion for improving coding and development practices. You have worked with More ❯

architectures (Lambda, Fargate, Cloud Run, et al.) and a clear understanding of when not to use them. Experience with message queues (SQS, PubSub, RabbitMQ etc.) and data pipelines (Kafka, Beam, Kinesis, etc.). Effective team player with excellent communication, presentation, and influencing skills. Passion for improving coding and development practices. Experience working with microservices communicating over a range of More ❯

for real-world applications such as fraud detection, network analysis, and knowledge graphs.  - Optimize performance of graph queries and design for scalability.  - Support ingestion of large-scale datasets using Apache Beam, Spark, or Kafka into GCP environments.  - Implement metadata management, security, and data governance using Data Catalog and IAM.  - Work across functional teams and clients in diverse EMEA More ❯

for real-world applications such as fraud detection, network analysis, and knowledge graphs. - Optimize performance of graph queries and design for scalability. - Support ingestion of large-scale datasets using Apache Beam, Spark, or Kafka into GCP environments. - Implement metadata management, security, and data governance using Data Catalog and IAM. - Work across functional teams and clients in diverse EMEA More ❯

machine learning purposes. Expertise in design of data solutions for BigQuery. Expertise in logical and physical data modelling. Hands-on experience using Google Dataflow, GCS, cloud functions, BigQuery, DataProc, Apache Beam (Python) in designing data transformation rules for batch and data streaming. Solid Python programming skills and using Apache Beam (Python). Structure of CI/ More ❯

Kubernetes). Preferred Skills: Experience with feature stores (e.g., Feast, Tecton). Knowledge of distributed training (e.g., Horovod, distributed PyTorch). Familiarity with big data tools (e.g., Spark, Hadoop, Beam). Understanding of NLP, computer vision, or time series analysis techniques. Knowledge of experiment tracking tools (e.g., MLflow, Weights & Biases). Experience with model explainability techniques (e.g., SHAP, LIME … . Familiarity with reinforcement learning or generative AI models. Tools & Technologies: Languages: Python, SQL (optionally: Scala, Java for large-scale systems) Data Processing: Pandas, NumPy, Apache Spark, Beam Model Serving: TensorFlow Serving, TorchServe, FastAPI, Flask Experiment Tracking & Monitoring: MLflow, Neptune.ai, Weights & Biases #J-18808-Ljbffr More ❯