26 to 50 of 221 Apache Airflow Jobs

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

/CD for data pipelines. Good to have skills Although not essential, the following skills are desired by the client: Kafka/Kinesis, Airflow, CodePipeline. BI tools (Power BI/Tableau). Docker/OpenShift; metadata driven pipelines. 3-8+ years Data Engineering experience. Cloud data engineering ...

with a strong focus on ETL/ELT. • Advanced SQL and Python development skills. • Hands-on experience with DBT, GIT, Terraform, Docker, IAM, and Airflow (Composer). • Proven experience working on cloud platforms – ideally GCP (BigQuery), but Azure or AWS also considered. • Strong understanding of Data Mesh, Test Driven ...

role Strong proficiency in SQL, Python and Spark (PySpark) Experience designing data models and implementing reliable pipeline patterns Knowledge of orchestration tools such as Airflow Experience with low-code or no-code integration tools Hands-on experience with cloud platforms (AWS preferred), including services for storage, transformation and warehousing ...

detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such … MongoDB. Proven delivery of streaming pipelines and real-time data processing solutions. Improved job efficiency and reduced runtimes through Apache Spark optimisation and development. Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines. Delivered streaming pipelines and real-time data processing solutions. Experience replacing legacy ETL tools ...

detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such … MongoDB. Proven delivery of streaming pipelines and real-time data processing solutions. Improved job efficiency and reduced runtimes through Apache Spark optimisation and development. Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines. Delivered streaming pipelines and real-time data processing solutions. Experience replacing legacy ETL tools ...

1classification ) Strong SQL&Python Hands-onETL/ELTexperience(e.g.,Matillion,Talend,Fivetran,ADF) Experiencewith Snowflake (AWS/Azure/GCPbeneficial) Strongdatamodellingknowledge Understandingofdatagovernanceandsecuritybestpractices Abilitytotranslatebusinessneedsintoscalablesolutions Desirable Airflow,dbt,Kafka,CI/CD,BItools,governanceplatforms,financialservicesexperience. Interestedinbuildingmodern,enterprise-scaledataplatformsinahighlyregulatedenvironment Applynow ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

Skills and Experience * Strong commercial experience working with AWS in production environments. * Proficiency in Python and SQL for data processing and orchestration. * Experience with Airflow for workflow management. * Knowledge of cloud data warehousing, ideally Redshift. * Experience with infrastructure as code and CI/CD tooling such as Terraform ...

with hands on Snowflake experience • Proven dbt experience at scale • Strong dimensional modelling using Kimball principles • Python for data engineering tasks • Experience with Airflow or similar orchestration tools • Git based workflows • Familiarity with BI tools such as Tableau, Looker, or Power BI • Experience working in cloud environments including ...

Experienced building real pipelines — not just notebooks Curious about how financial systems actually work Bonus points for: Exposure to ML workflows Familiarity with dbt, Airflow, Docker Experience handling financial or semi-structured data Interest in LLM infrastructure and vector databases Finance background is not required. Intellectual horsepower and ownership ...

with SQL and modern data warehousing technologies (e.g., Snowflake, BigQuery, Redshift, Databricks). Proven experience building production-ready pipelines using tools such as dbt, Airflow, Dagster, Prefect, or similar. Experience integrating data from HRIS and talent systems (e.g., Workday, BambooHR, Greenhouse, Lever, HiBob, Personio). Ability to work with ...

Experience working with complex, multi source data environments, ideally including ecommerce and subscription data. Experience with data pipeline orchestrations tools such as Cloud Composer, Airflow or equivalent. Comfort working in fast moving environments with imperfect starting points. Ability to balance best practice with pragmatism and delivery speed. Strong communication ...

tasks, working with the Snowflake Data Marketplace, Performance Tuning and Knowledge of ETL/ELT pipelines and data integration tools (e.g., Informatica, Matillion, DBT, Airflow). Experience with APIs, stored procedures, and automation scripts is a plus. Strong analytical and problem-solving mindset. Clear communicator, able to work across ...

SENIOR DATA ENGINEER £75,000 + BENEFITS MANCHESTER (Hybrid) This is an opportunity to take real ownership of a modern data platform within a growing, technology-driven bank. You will join a collaborative data team ...

SENIOR DATA ENGINEER £75,000 + BENEFITS PRIMARILY REMOTE This is an opportunity to take real ownership of a modern data platform within a growing, technology-driven bank. You will join a collaborative data team ...

tooling Good scripting skills (Python/SQL/Bash) Strong understanding of AWS security, IAM, and networking Comfortable operating in Agile, delivery-led environments Apache Airflow for DAG Date-Led Platform/Cloud Engineering Key requirements: Active SC Clearance (essential) Permanent role Ideally you will be based around ...

EXPERIENCE The successful Lead Data Engineer will have the following skills and experience: Strong Python and SQL skills AWS or GCP Orchestration, ideally with Airflow or Dagsta but open to others Data warehousing - Snowflake, Redshift, BigQuery (NOT Databricks) Terraform or Palumi BENEFITS The successful Lead Data Engineer will receive ...

EXPERIENCE The successful Lead Data Engineer will have the following skills and experience: Strong Python and SQL skills AWS or GCP Orchestration, ideally with Airflow or Dagsta but open to others Data warehousing - Snowflake, Redshift, BigQuery (NOT Databricks) Terraform or Palumi BENEFITS The successful Lead Data Engineer will receive ...

EXPERIENCE The successful Lead Data Engineer will have the following skills and experience: Strong Python and SQL skills AWS or GCP Orchestration, ideally with Airflow or Dagsta but open to others Data warehousing - Snowflake, Redshift, BigQuery (NOT Databricks) Terraform or Palumi BENEFITS The successful Lead Data Engineer will receive ...

Familiarity with cloud-based data tools and modern data ecosystems. Understanding of data modeling, warehousing concepts, and best practices Nice-to-Have Experience with Airflow, dbt, or similar orchestration/modeling tools. Experience with BI tools Knowledge of GCP services (Cloud Storage, Dataflow, Pub/Sub, etc.). Social ...

Familiarity with cloud-based data tools and modern data ecosystems. Understanding of data modeling, warehousing concepts, and best practices Nice-to-Have Experience with Airflow, dbt, or similar orchestration/modeling tools. Experience with BI tools Knowledge of GCP services (Cloud Storage, Dataflow, Pub/Sub, etc.). Social ...

technical expertise with mentoring, guidance, and collaboration across teams. Core technologies include: Google Cloud Platform (BigQuery, Dataflow, Dataproc, Data Fusion, Data Streams, Cloud Functions, Airflow/Composer), SQL, Python, CI/CD tooling About you You'll have 10+ years experience in Data Engineering Experience operating in a Lead ...

grade data pipelines for multiple internal teams. Develops and optimises central data models supporting analytics, ML, and operational workloads. Works with Databricks, AWS, dbt, Airflow, and Looker as part of a modern data stack. Leads complex technical initiatives and improves efficiency through CI/CD, observability, and tooling. Ensures ...

Time-series data experience (market data, telemetry, pricing, events) Streaming exposure (Kafka/Event Hubs/Kinesis) Experience with Temporal (or similar orchestrators like Airflow/Dagster/Prefect) Any exposure to AI agents/automation tooling 👉 Apply ...