26 to 50 of 70 Apache Airflow Jobs in the City of London

Strong experience with Snowflake, including building full solutions from scratch with security and user access controls. Knowledge of DBT and general data modelling; Vault experience is desirable. Experience with Airflow and Python. Proficiency with AWS services such as Lambda, S3, SNS, and CDK for DevOps. Ability to build, deploy, and manage infrastructure using Terraform. Benefits Bonus opportunity - up to More ❯

rules, set up monitoring/logging, and support architecture decisions What You Bring Strong SQL & Python (PySpark); hands-on with GCP or AWS Experience with modern ETL tools (dbt, Airflow, Fivetran) BI experience (Looker, Power BI, Metabase); Git and basic CI/CD exposure Background in a quantitative field; AI/ML interest a plus Bonus: Experience in manufacturing More ❯

actionable for operational and ML-driven use cases About you: Strong background in software and data engineering leadership Proficient in Python, SQL, and modern ELT practices (e.g. dbt, Fivetran, Airflow) Deep knowledge of data warehousing (Snowflake), AWS services (e.g. Lambda, Kinesis, S3), and IaC (Terraform) Experienced in building data platforms with a focus on governance, reliability, and business value More ❯

experience in ML libraries like scikit-learn, PyTorch, TensorFlow , or XGBoost Hands-on experience building and deploying ML models into production environments Familiarity with ML Ops workflows (e.g., MLflow, Airflow, Weights & Biases, or Kubeflow) Experience working with structured data (credit, payments, customer behaviour) and applying feature engineering at scale Understanding of model performance metrics, calibration, A/B testing More ❯

Lead Data Engineer - Snowflake, DBT, Airflow - London - Up to £100k I'm working with a key client of ours here at TRG who are looking to grow out their Data & Analytics function. My client are globally renowned for being a leader within their relative field. Whilst they are a very well recognised house hold brand, they are also known … happy employee means higher productivity! So they offer very flexible working arrangements through both WFH options and flexi working hours. Experience required... * Expert in Snowflake * Strong DBT experience * Strong Airflow experience * Expert knowledge and understanding of Data Warehousing * Strong AWS experience This is a great opportunity to join outstanding organisation who pride themselves on being one of the best … is of an interest then get in touch ASAP. Send across your CV to t.shahid@nigelfrank.com or alternatively, give me a call on 0191 3387551. Keywords: Snowflake, DBT, SQL, Airflow, AWS, Engineer, DWH, Data Warehouse, Data Warehousing, Architecture, London #J-18808-Ljbffr More ❯

technologies – Azure, AWS, GCP, Snowflake, Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis/MQ/Event Hubs, Kafka … Dataflow/Airflow/ADF etc. Excellent consulting experience and ability to design and build solutions, actively contribute to RfP response. Ability to be a SPOC for all technical discussions across industry groups. Excellent design experience, with entrepreneurship skills to own and lead solutions for clients Excellent ETL skills, Data Modeling Skills Excellent communication skills Ability to define the … skills. A minimum of 5 years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF Desirable Skills Designing Databricks based More ❯

technologies – Azure, AWS, GCP, Snowflake, Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis/MQ/Event Hubs, Kafka … Dataflow/Airflow/ADF etc. Excellent consulting experience and ability to design and build solutions, actively contribute to RfP response. Ability to be a SPOC for all technical discussions across industry groups. Excellent design experience, with entrepreneurship skills to own and lead solutions for clients Excellent ETL skills, Data Modeling Skills Excellent communication skills Ability to define the … skills. A minimum of 5 years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF Desirable Skills Designing Databricks based More ❯

technologies – Azure, AWS, GCP, Snowflake, Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis/MQ/Event Hubs, Kafka … Dataflow/Airflow/ADF etc. Excellent consulting experience and ability to design and build solutions, actively contribute to RfP response. Ability to be a SPOC for all technical discussions across industry groups. Excellent design experience, with entrepreneurship skills to own and lead solutions for clients. Excellent ETL skills, Data Modeling Skills. Excellent communication skills. Ability to define the … minimum of 5 years’ experience in a similar role. Ability to lead and mentor the architects. Required Skills : Mandatory Skills [at least 2 Hyperscalers]: GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF. Preferred Skills : Designing Databricks based More ❯

technologies – Azure, AWS, GCP, Snowflake, Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis/MQ/Event Hubs, Kafka … Dataflow/Airflow/ADF etc. Excellent consulting experience and ability to design and build solutions, actively contribute to RfP response. Ability to be a SPOC for all technical discussions across industry groups. Excellent design experience, with entrepreneurship skills to own and lead solutions for clients Ability to define the monitoring, alerting, deployment strategies for various services. Experience providing … skills. A minimum of 5 years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF Designing Databricks based solutions for More ❯

technologies – Azure, AWS, GCP, Snowflake, Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis/MQ/Event Hubs, Kafka … Dataflow/Airflow/ADF etc. Excellent consulting experience and ability to design and build solutions, actively contribute to RfP response. Ability to be a SPOC for all technical discussions across industry groups. Excellent design experience, with entrepreneurship skills to own and lead solutions for clients Excellent ETL skills, Data Modeling Skills Excellent communication skills Ability to define the … skills. A minimum of 5 years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF Desirable Skills: Designing Databricks based More ❯

driven decision-making. What we'd like to see from you: 3–5 years of experience in data integration, orchestration, or automation roles Solid experience with orchestration tools (e.g., Apache Airflow, MuleSoft, Dell Boomi, Informatica Cloud). Familiarity with cloud data platforms (e.g., AWS, Microsoft Azure, Google Cloud Platform) and related data movement technologies, including AWS Lambda and More ❯

platforms Deep understanding of tools like Kafka , Flink , Spark Streaming , etc. Proficiency in Python , Java , or Scala Cloud experience with AWS , GCP , or Azure Familiarity with orchestration tools like Airflow , Kubernetes Collaborative, solutions-focused mindset and a willingness to lead from the front 🌟 What’s on Offer Competitive base salary + quarterly bonus Hybrid working with a central London More ❯

financial accounting Experience manipulating, processing, modelling, and extracting value from large, disconnected datasets Familiarity with data warehouse design and modelling Familiarity with cloud data services (preferably Azure) Experience with Airflow ? Please note : unfortunately, this role does not offer VISA sponsorship. #J-18808-Ljbffr More ❯

data warehouses (Snowflake, BigQuery), lakes, and CDPs. Understanding of data pipelines, contracts, lineage, APIs, and data governance frameworks. Comfortable navigating GDPR, CCPA and other compliance requirements. Familiar with dbt, Airflow, and cloud platforms (AWS, GCP, Azure). Proven ability to manage backlogs, engage stakeholders and deliver in Agile environments. Analytical mindset with a strong commercial awareness. Why Apply? Excellent More ❯

and compliance practices. Tech Stack: Languages: Python, SQL Cloud: Google Cloud Platform (BigQuery, Dataflow, Vertex AI, Cloud Run, Pub/Sub) Databases: PostgreSQL, BigQuery, Pinecone, FAISS, Chroma Tools: dbt, Airflow, Terraform, Docker, GitHub Actions AI Frameworks: LangChain, LangGraph, LangFlow, CrewAI, OpenAI APIs What We’re Looking For: Strong experience building and maintaining data systems on GCP Direct experience working More ❯

analytics platforms. Experience managing data products or platforms, ideally customer data platforms (CDPs), data warehouses (e.g. Snowflake, BigQuery), or data lakes. Familiarity with data engineering workflows, data tools (dbt, Airflow), and cloud data platforms (AWS, GCP, Azure). Familiarity with data modelling, data pipelines, ETL/ELT processes, and APIs. Knowledge of GDPR, data ethics, and governance practices. Strong More ❯

data warehouses (Snowflake, BigQuery), lakes, and CDPs. Understanding of data pipelines, contracts, lineage, APIs, and data governance frameworks. Comfortable navigating GDPR, CCPA and other compliance requirements. Familiar with dbt, Airflow, and cloud platforms (AWS, GCP, Azure). Proven ability to manage backlogs, engage stakeholders and deliver in Agile environments. Analytical mindset with a strong commercial awareness. Why Apply? Excellent More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting edge AR and VR technologies, 3D printing, etc. They … will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring data integrity through … rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in Apache Airflow -Experience with ETL tools, data architecture, and data warehousing solutions -Strong More ❯

data modeling, and database management. Strong understanding of cloud-based data solutions. Experience with the tools we work with, or other equivalent ones (dbt, BigQuery, Stitch, Segment, Fivetran, Metabase, Apache Airflow). Education A bachelor’s degree in a relevant field such as Computer Science, Information Technology, Data Science, Engineering (e.g., Computer Science Engineering), Mathematics, or Statistics. *** Only More ❯

like ICE, CME, Reuters, Bloomberg). Looking for 3-6 years experience in a relevant role and ideally some exposure to commodities data sets. Strong Python and exposure to Airflow is a must. Tech Stack: Python, AWS, Airflow, SQL, ML, ETL Please apply if you want to be a part of this unique build out. More ❯

years experience in Python and SQL. Experience working with large datasets. Exceptional communication and collaboration skills. Advantageous: Any operational experience supporting real-time systems. Working knowledge of Amazon S3, Airflow, Kafka, Docker. More ❯

platform, ensuring scalability, reliability, and security. Drive modernisation by transitioning from legacy systems to a lean, scalable platform. Act as a lead expert for technologies such as AWS, DBT, Airflow, and Databricks. Establish best practices for data modelling, ingestion, storage, streaming, and APIs. Governance & Standards Ensure all technical decisions are well-justified, documented, and aligned with business needs. Lead … in data engineering and cloud engineering, including data ingestion, transformation, and storage. Significant hands-on experience with AWS and its data services. Expert-level skills in SQL, Python, DBT, Airflow and Redshift. Confidence in coding, scripting, configuring, versioning, debugging, testing, and deploying. Ability to guide and mentor others in technical best practices. A product mindset, focusing on user needs More ❯

Role: Data Engineer Tech stack: Python, SQL, AWS, data processing frameworks (e.g. Airflow, Tensorflow) Salary: £40,000–£70,000 + big equity Location: London (on site) This fast-growing team is addressing a real-world challenge that's impacted a huge number of people across the UK - especially those balancing work with study or other commitments, born out of More ❯

Flink, Spark Streaming) Solid hands-on knowledge of cloud platforms (AWS, GCP or Azure) Strong proficiency in languages like Python, Java or Scala Familiarity with orchestration tools such as Airflow or Kubernetes Strong stakeholder management and communication skills Passion for mentoring and developing engineering talent 🎁 What’s On Offer Competitive salary + quarterly bonus scheme Hybrid working with a More ❯

development of a new global data platform (PaaS) Ensure scalable storage solutions (data lakes, data warehouses) to handle structured and unstructured data. Implement ETL/ELT pipelines using Dagster, Airflow, or similar tools. Optimize performance and scalability for large data volumes. Govern data security, compliance, and access controls. Development & DevOps: Strong programming and scripting skills in Python. Knowledge of More ❯