1 to 25 of 162 Permanent Apache Airflow Jobs

solving, organizational, and analytical skills. Design: Strong foundation in algorithms, design patterns, and writing efficient code. Workflow: Knowledge of orchestration and scheduling tools (e.g., Apache Airflow). Data: Understanding of data processing technologies such as Apache Spark and Apache Kafka. Autonomy: Able to work independently ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

reviews, pairing, and sharing best practices. What You'll Bring Technical Expertise: Proficiency in Python and SQL; experience with data processing frameworks such as Airflow, Spark, or TensorFlow. Data Engineering Fundamentals: Strong understanding of data architecture, data modelling, and scalable data solutions. You have hands-on experience deploying … running ML systems in production, including model serving and monitoring. Cloud Platforms: Familiarity with AWS or Azure, including services like Apache Airflow, Terraform, or SageMaker. Data Quality Management: Experience with data versioning and quality assurance practices. You enjoy working in fast-moving environments where you can have real ...

reviews, pairing, and sharing best practices. What You’ll Bring Technical Expertise: Proficiency in Python and SQL; experience with data processing frameworks such as Airflow, Spark, or TensorFlow. Data Engineering Fundamentals: Strong understanding of data architecture, data modelling, and scalable data solutions. You have hands-on experience deploying … running ML systems in production, including model serving and monitoring. Cloud Platforms: Familiarity with AWS or Azure, including services like Apache Airflow, Terraform, or SageMaker. Data Quality Management: Experience with data versioning and quality assurance practices. You enjoy working in fast-moving environments where you can have real ...

/GCP beneficial) Strong data modelling knowledge Understanding of data governance and security best practices Ability to translate business needs into scalable solutions Desirable Airflow, dbt, Kafka, CI/CD, BI tools, governance platforms, financial services experience. Interested in building modern, enterprise-scale data platforms in a highly regulated ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

similar, for pipeline development and automation Cloud experience across AWS, Azure, or GCP Familiarity with orchestration and analytics engineering tools such as dbt, Airflow, or equivalents Good habits around governance, security, documentation, version control (Git), and CI/CD The kind of person who thrives here Confident, curious ...

provisioning automation. Nice to Have Skills: Experience with Power BI, DAX, and data modeling for optimized reporting. Knowledge of big data orchestration tools like Apache Airflow. Previous exposure to regulated environments such as finance, healthcare, or gaming. Experience in a lottery or similar domain is a plus. Preferred Education ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

preferred. Required to have experience with one or more AI automations platforms like Kubeflow pipeline, MLFlow, Azure Pipeline, AWS Sage Maker Pipeline, Airflow, Jenkins, Spark, Hadoop, Kafka, Jira and GIT. We provide market-competitive compensation packages, inclusive of base pay, incentives, and benefits. The base pay rate for full ...

delivering customer-facing Enterprise SaaS products at scale. Strong hands-on or architectural experience with modern data platform tooling, including but not limited to Apache Airflow, dbt, Kafka, AWS Glue, AWS Redshift, AWS Sagemaker, AWS Dynamo, AWS Quick Sight. Success leading a data engineering team through complex challenges ...

Cassandra) Expertise in building and maintaining ETL pipelines and data workflows Familiarity with cloud data platforms (AWS, Azure, GCP) and data pipeline orchestration tools (Airflow, Prefect, etc.) Understanding of data modelling, schema design, and performance optimisation Experience with agile development methodologies, including Scrum and Kanban Familiarity with version control ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

days per week in the office. • A collaborative culture and clear opportunities for technical growth. • Exposure to modern tooling including AWS, Redshift, Airflow, Terraform, and dbt. • The chance to shape a data platform and have meaningful influence on engineering standards. How to Apply If you are interested in this ...

based data engineering, including Redshift, S3, IAM and related services Building and maintaining data pipelines using Python and SQL Orchestration tools such as Airflow Data modelling and warehouse architecture in modern cloud environments Git-based development and Infrastructure as Code (e.g. Terraform) Desirable experience includes exposure to regulated industries ...

Experience working in multi-cloud environments (GCP/Azure). Proficiency with platforms such as BigQuery, Dataflow, Dataform, Cloud Run, DBT, Dataproc, SQL, Python, Airflow, Pub/Sub, and equivalent Azure tooling such as ADLS, Azure Functions, and Databricks. Experience with microservices architectures (Kubernetes, Docker). Deep experience with ...

Excellence Lead the evolution of the enterprise data landscape using modern platforms such as Databricks, Snowflake, Azure Synapse, and BigQuery. Oversee data engineering with Apache Airflow, dbt, and Prefect, ensuring performance, governance, and alignment with enterprise metadata standards (Collibra, Alation, Microsoft Purview). Drive adoption of Delta Lake ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

with a strong focus on ETL/ELT. • Advanced SQL and Python development skills. • Hands-on experience with DBT, GIT, Terraform, Docker, IAM, and Airflow (Composer). • Proven experience working on cloud platforms – ideally GCP (BigQuery), but Azure or AWS also considered. • Strong understanding of Data Mesh, Test Driven ...

role Strong proficiency in SQL, Python and Spark (PySpark) Experience designing data models and implementing reliable pipeline patterns Knowledge of orchestration tools such as Airflow Experience with low-code or no-code integration tools Hands-on experience with cloud platforms (AWS preferred), including services for storage, transformation and warehousing ...