7 of 7 Apache Airflow Jobs in Berkshire

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

Escalate complex requests to specialized engineering pods and track to closure. TECHNICAL IMPLEMENTATION & ENABLEMENT Provision and configure research, data, and trading environments (Python, AWS, Airflow, GitHub, Coder, etc.). Coordinate bespoke data ingestion (market, alternative, proprietary) via APIs, direct feeds, and file transfers. Deliver live, role-based training ...

Engineering, Data Quality, or Data Governance roles with production systems. Strong SQL and hands-on experience with one or more: Spark/Databricks, dbt, Airflow/Databricks Workflows, Kafka/Kinesis, Python. Experience implementing data quality frameworks and observability (e.g., Great Expectations, Soda, Monte Carlo, Databricks DQ, BigQuery ...

like Spark, Flink etc., Solid grasp of Kubernetes, CI/CD and Terraform Working knowledge of ETL and ELT frameworks and orchestration tools like Airflow Hiring, developing, and retaining cyber data platform talent Ability to deliver clear input, guide, empower the team to achieve desired outcomes Developing working relationships ...

Marketing, Product, and Engineering, including influencing senior and executive partners. Nice to have: Experience with marketing tech stacks, event-level data, Spark/Databricks, Airflow, generative AI for CRM applications, or two-sided marketplace environments. Please see the Talent Privacy Notice for information regarding how eBay handles your personal ...

MLOps tools (e.g., Vertex Pipelines, Kubeflow, Weights & Biases) for experiment tracking, model registry, and automated deployment. Leveraging Docker/Kubernetes and workflow orchestration tools (Airflow, Prefect, Dagster). Collaborating with product, design, and engineering teams to deliver ML features that directly impact customer experience. Translating model performance into business … significant scale (e.g., web-scale, distributed systems, cloud-native environments). Brings expertise in MLOps: CI/CD pipelines, Docker, Kubernetes, workflow orchestration (Airflow, Prefect), and automation. Has experience across and understands the full ML lifecycle. Can design for long-term scalability, reliability, and resilience. Has strong programming skills ...