201 to 225 of 239 Permanent Apache Airflow Jobs

This position will be posted for a minimum of five calendar days internally. All internal applicants are required to inform your current supervisor/manager prior to applying. Internal applicants are required to be in ...

Bristol 50000 - 72000 per annum + Excellent benefits PermanentSenior Data Engineer – Python, SQL, Airflow, Azure, Pyspark, KubernetesSalary: Up to 72,000 per annumBenefits: Excellent benefits package including annual bonus, free onsite gym, and company pension scheme.About the RoleWe are seeking an experienced Senior Data Engineer to join an innovative … ResponsibilitiesData Pipeline DevelopmentData Platform ManagementStakeholder CollaborationLeadership & MentorshipTechnology StackThe successful candidate will have experience working with some or all of the following technologies:Data Processing: Apache Spark/PySparkWorkflow Orchestration: Apache AirflowContainerisation: KubernetesCloud Platform: Microsoft AzureData APIs & Semantic Layer: CubeJSProgramming Languages: Python, SQLEssential Skills & ExperienceDegree in a STEM discipline ...

Engineering, or MLOps environments Experience building, deploying, and supporting machine learning solutions at scale Experience working with AWS and ML tooling such as MLflow, Airflow, Metaflow, or Snowflake Ability to balance technical leadership, strategic thinking, and hands-on engineering contribution SALARY AND BENEFITS Up to £125,000 base salary ...

technical design reviews. Must Haves (Hard Skills)Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… Avro).Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow).Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python, SQL, or an Object-Oriented ...

Experience designing data products across source-aligned, aggregate and consumer-aligned patterns. Proven ability to architect and scale modern cloud data platforms using Snowflake, Apache Iceberg, Amazon Redshift, and S3.Hands-on expertise in developing robust data transformation pipelines and orchestrating complex workflows using dbt and Apache Airflow. Strong ...

data infrastructure, while contributing to strong engineering practices and standards. The environment is heavily focused on Google Cloud Platform (GCP), with BigQuery, Dataform and Airflow/Cloud Composer forming key parts of the technology stack. WHAT YOU'LL BE DOING * Designing and implementing complex data flows across operational, analytics … clearly defined. ESSENTIAL EXPERIENCE * Strong hands-on experience with Google Cloud Platform (GCP). * Strong experience with BigQuery. * Experience using Dataform. * Experience with Airflow/Cloud Composer. * Strong data engineering fundamentals, including building scalable and robust production data solutions. * Experience working with data quality, governance and appropriate data controls. ...

maintain standardized data models, schemas, and metadata that make commodities datasets easy to understand, discover, and reuse. Schedule, monitor, and manage data pipelines using Airflow or similar workflow orchestration tools to ensure reliable, timely data delivery. Implement validation, reconciliation, and anomaly-detection controls to maintain data completeness, accuracy … building and maintaining ETL pipelines. Strong Python and SQL skills, including experience working with large datasets and complex data transformations. Hands-on experience with Airflow or comparable workflow orchestration tools. Experience with Git, CI/CD pipelines such as GitHub Actions, and automated testing frameworks such as PyTest. Strong ...

experience with cloud platforms (AWS, Azure, GCP)Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow)Practical experience with NLP and/or Search techniquesPrompt engineering, and LLMs for enterprise-scale applications. You have team lead experienceStrong ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

elevate our current data pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg). Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork … Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg). dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background in designing ...

frameworks and API design (FastAPI, Flask, or Django) for building internal data services and RESTful APIs.Familiarity with enterprise workload scheduling tools (e.g., Autosys, Airflow, Control-M) and enterprise messaging systems (e.g., Apache Kafka, IBM MQ).Familiarity with containerization technologies (Docker, OpenShift, Kubernetes) and cloud infrastructure concepts.Experience in Linux ...

integration failures, data inconsistencies, and production issues Key Skills • Python & SQL (production-grade pipeline development) • REST APIs, webhooks, OAuth, event-driven architecture • Orchestration tools: Airflow, Prefect, or Dagster • Cloud platforms: AWS, GCP, or Azure • Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills • Vector databases and embedding pipelines • Real-time … robust, production-grade data pipelines from scratch. • Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. • Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). • Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

sequenced engineering plan, and keep the team aligned to it. Own the technical quality of the platform: Snowflake infrastructure and RBAC, ingestion patterns, orchestration (Airflow), CI/CD, observability and cost controls. Make and record technical decisions. Run the decision-document process. Make the final call when the team … enterprise-grade dbt model development. Experience designing end-to-end data pipelines. Combining batch, file-based, and streaming ingestion (Kafka/CDC) managed with Airflow orchestration. A good understanding of data security and governance, including PII handling and access control. Strong infrastructure-as-code skills and CI/ ...

with cloud platforms (AWS, Azure, GCP)Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow)Practical experience with NLP and/or Search techniquesPrompt engineering, and LLMs for enterprise-scale applications. You have team lead experienceStrong ...

with cloud platforms (AWS, Azure, GCP) Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

with cloud platforms (AWS, Azure, GCP) Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

Why Sony Interactive Entertainment? Sony Interactive Entertainment isn't just the Best Place to Play — it's also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Managed Airflow (Composer) to orchestrate ETL pipelines and enables reliable movement of data from source systems through the platform into downstream vendor systems. You'll guide … contribution where needed (critical ETLs, framework components, design spikes, performance fixes).ETL orchestration & data movement Design and develop robust ETL pipelines orchestrated in Airflow, including scheduling strategies, dependency management, retries, backfills, and processing. Establish patterns for onboarding new source systems and delivering data to downstream vendor systems with strong ...

deliver new capabilities Build and maintain robust MLOps pipelines to support scalable, reproducible, and automated model development, deployment, and monitoring Leverage tools such as Airflow for workflow orchestration and MLflow for experiment tracking, model registry, and lifecycle management, ensuring strong CI/CD practices and model governance Essential Skills … development of machine learning models and/or deep learning to solve complex science and engineering problems Experience with MLOps tools and practices, including Airflow, MLflow, and containerization (e.g., Docker) A passion for gaining insight into real-world datasets and clearly communicating through data visualization techniques Interest in material ...