576 to 600 of 867 Permanent Apache Airflow Jobs

analytics and governance while promoting self-service insights for business users. Key skills include data lakehouse design (Iceberg), AWS/Azure data services, dbt, Airflow, and BI/dashboard experience. The role emphasizes automation, quality, and collaboration with architects. #J-18808-Ljbffr ...

smart meter readings daily, enabling customised, market-leading tariffs and a scalable energy ecosystem. Our stack includes Python, Terraform, Kubernetes, Argo CD and Airflow on S3-based data lakes. Join us as a Platform Engineer to design, implement, and maintain scalable data pipelines and services, collaborating with product ...

join our dynamic, expanding team. Over the last few years we've built a solid data foundation using best in class technologies (dbt, BigQuery, Airflow) and are just at the start of our journey to leverage this data in a more sophisticated way. About Zapp: Zapp is London ...

results post-migration (before/after comparison) -Release management — experience managing releases in an enterprise environment Nice to have - Experience with other orchestration tools (Airflow, AWS Glue Workflows) - AWS certification (Solutions Architect/DevOps Engineer) - Knowledge of AWS Lambda, EventBridge (commonly used- alongside Step Functions) - Experience in financial domain ...

platforms - execution roles, service roles, cross-account access, Lake Formation, SSO/SAML MLflow or equivalent experiment tracking SageMaker Pipelines or similar workflow orchestration (Airflow, Step Functions) Model serving - real-time endpoints, batch transform, auto-scaling, endpoint monitoring Snowflake as a data source for ML pipelines Kubernetes ...

ideally within a large‐scale or regulated environment. AWS Experience: Strong working knowledge of AWS and cloud‐first data pipeline orchestration tools such as Airflow, Glue, or EMR. A Security Mindset: Understanding of compliance and security when managing sensitive data pipelines. Strong Automation Skills: Comfortable using Python and Shell ...

platforms - execution roles, service roles, cross-account access, Lake Formation, SSO/SAML MLflow or equivalent experiment tracking SageMaker Pipelines or similar workflow orchestration (Airflow, Step Functions) Model serving - real-time endpoints, batch transform, auto-scaling, endpoint monitoring Snowflake as a data source for ML pipelines Kubernetes ...

narratives Entrepreneurial mentality: You can take work beyond the analysis and drive outcomes. Useful additional experience: Experience in Fintech domain Knowledge of utilities like Airflow/Github/DBT Experience with user-facing products and data Please note: we are not able to offer visa sponsorship for this role ...

labeling workflows, ontologies, consensus systems, QA, or label-store integrations Experience with Go, Rust, or TypeScript for services Experience with orchestration tools such as Airflow or Prefect Experience with metadata and lineage tools such as MLflow or Weights & Biases Exposure to edge or sensor data, including audio, sonar, video ...

labeling workflows, ontologies, consensus systems, QA, or label-store integrations Experience with Go, Rust, or TypeScript for services Experience with orchestration tools such as Airflow or Prefect Experience with metadata and lineage tools such as MLflow or Weights & Biases Exposure to edge or sensor data, including audio, sonar, video ...

large‐scale or regulated environment. AWS Experience: You have a strong working knowledge of AWS and cloud‐first data pipeline orchestration tools (like Airflow, Glue, or EMR). A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines. Strong Automation Skills ...

will include Design, build and maintain robust, scalable data pipelines that support data science workflows and advanced analytics use cases Develop and manage complex Airflow orchestration, ensuring pipelines are reliable, observable and maintainable Work hands-on with Databricks suite to deliver high-quality data engineering solutions for data science ...

what's at stake for real people, and that's what gives our intensity its purpose. Technologies we use Claude Code, Kubernetes, AWS, Airflow, dlt, dbt, Python, TypeScript, React, Go Please note: we are unable to sponsor or take over sponsorship of an employment visa at this time. Compensation ...

what's at stake for real people, and that's what gives our intensity its purpose. Technologies we use Claude Code, Kubernetes, AWS, Airflow, dlt, dbt, Python, TypeScript, React, Go Please note: we are unable to sponsor or take over sponsorship of an employment visa at this time. Compensation ...

large-scale or regulated environment.* AWS Experience: You have a strong working knowledge of AWS and cloud-first data pipeline orchestration tools (like Airflow, Glue, or EMR).* A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines.* Strong Automation Skills ...

development, dashboard creation, or data ingestion pipelines. Containerization & Orchestration: Hands-on experience with Kubernetes or Rancher architectures. Workflow Automation: Experience orchestrating data pipelines using Apache Airflow. The Benefits Package Wyetech believes in generously supporting employees as they prepare for retirement. The company automatically contributes 20% of each employee ...

Expert‐level Python development skills with production experience at scale. Strong Kubernetes experience with production workloads. Hands‐on experience with workflow orchestration tools (Temporal, Airflow, Kubeflow, or similar). Cloud platform expertise (AWS required, Azure & GCP beneficial). Experience with data processing frameworks (Spark, Athena, Ray, or similar). ...

idiomatic Python used for reliable, well-tested systems (not exploratory notebooks). Design and operation of large-scale ETL/ELT pipelines, including orchestration (Airflow, Prefect, or similar) and reliability patterns such as retries, backfills, and exactly-once semantics. Deep experience with Azure, including AKS, Helm, EventHub, observability tools … Advanced experience operating Delta tables at scale, covering transaction semantics, schema evolution, partitioning, and compaction, alongside highly optimised analytical query execution using DuckDB or Apache Spark. Production-grade relational database expertise, particularly PostgreSQL (query tuning, indexing, partitioning, replication, and schema design across OLTP and analytics). Strong KDB experience ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

applications using TypeScript, HTML, and CSS Experience with JavaScript frameworks such as React, Angular, and Vue Familiar with Distributed Big Data processing engines including Apache Spark Experience using Jupyter Notebook Experience with data wrangling and preprocessing using tools such as pandas, NumPy Experience working with structured, semi-structured … Parquet, JSON, CSV, XML Familiarity with data quality concepts, data validation, and anomaly detection Experience with Git Source Control System. Required Skills Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience with SQL technologies such as MySQL, MariaDB, and PostgreSQL ...

applications using TypeScript, HTML, and CSS Experience with JavaScript frameworks such as React, Angular, and Vue Familiar with Distributed Big Data processing engines including Apache Spark Experience using Jupyter Notebook Experience with data wrangling and preprocessing using tools such as pandas, NumPy Experience working with structured, semi-structured … Parquet, JSON, CSV, XML Familiarity with data quality concepts, data validation, and anomaly detection Experience with Git Source Control System. Required Skills Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience with SQL technologies such as MySQL, MariaDB, and PostgreSQL ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

Senior Data Engineer to design, build, and operate enterprise-scale data solutions. You will leverage Snowflake with hands-on Python, PySpark, AWS and Airflow to create scalable data platforms and robust analytics capabilities. The role emphasizes technical leadership, governance, and collaboration with cross-functional teams to deliver high-performance ...

devices and platforms to surface key metrics. The role emphasizes coding in Python, strong SQL, and experience with large data ecosystems (HDFS, Hive, Spark, Airflow). You will influence architecture and own initiatives from concept to delivery. Hybrid work from Cambridge Office. #J-18808-Ljbffr ...

technical direction and influencing engineering practices across teams; Hands-on experience with cloud ML platforms such as AWS SageMaker and orchestration tools such as Airflow and Step Functions; Strong communication skills and ability to collaborate across Data Science, Engineering, and Data Platforms; Comfortable operating at the boundary between experimentation ...