1 to 25 of 153 Apache Airflow Jobs in the UK

highly regulated financial settings. Key Technologies & Tools Core ML: scikit-learn, PyTorch, TensorFlow, XGBoost, LightGBM Data Manipulation: Pandas, NumPy, LangGraph MLOps & Orchestration: MLflow, Apache Airflow Databases: Neo4j, MongoDB API Development: FastAPI Big Data (Beneficial): Apache Hadoop, Apache Spark (PySpark) What We Need From ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

architecture, modelling, pipelines) (please be clear on years and/or projects) Strong Python and SQL Experience with orchestration and transformation frameworks such as Airflow, dbt, Dagster , or similar Experience designing and delivering data platforms or lakehouse-style architectures Solid understanding of CI/CD for data engineering (e.g. ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and Terraform. What you get in return: Up to £60,000 per annum + benefits Hybrid working ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

/ELT pipeline. Proficiency with at least one scripting/programming language (e.g., Python). Familiarity with modern data engineering tools like dbt, Airflow, Prefect, or similar is a plus. Knowledge of cloud platforms (AWS/Azure/GCP). Understanding of data modelling, quality controls and best practices. ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

similar, for pipeline development and automation Cloud experience across AWS, Azure, or GCP Familiarity with orchestration and analytics engineering tools such as dbt, Airflow, or equivalents Good habits around governance, security, documentation, version control (Git), and CI/CD The kind of person who thrives here Confident, curious ...

similar, for pipeline development and automation Cloud experience across AWS, Azure, or GCP Familiarity with orchestration and analytics engineering tools such as dbt, Airflow, or equivalents Good habits around governance, security, documentation, version control (Git), and CI/CD The kind of person who thrives here Confident, curious ...

similar, for pipeline development and automation Cloud experience across AWS, Azure, or GCP Familiarity with orchestration and analytics engineering tools such as dbt, Airflow, or equivalents Good habits around governance, security, documentation, version control (Git), and CI/CD The kind of person who thrives here Confident, curious ...

similar, for pipeline development and automation Cloud experience across AWS, Azure, or GCP Familiarity with orchestration and analytics engineering tools such as dbt, Airflow, or equivalents Good habits around governance, security, documentation, version control (Git), and CI/CD The kind of person who thrives here Confident, curious ...

similar, for pipeline development and automation Cloud experience across AWS, Azure, or GCP Familiarity with orchestration and analytics engineering tools such as dbt, Airflow, or equivalents Good habits around governance, security, documentation, version control (Git), and CI/CD The kind of person who thrives here Confident, curious ...

Data Engineer: Experience with Databricks , Kafka , or Kinesis for real-time data streaming Knowledge of containerisation (Docker, ECS) and modern orchestration tools such as Airflow Familiarity with machine learning model deployment pipelines or data lakehouse architectures Data Engineer, AWS Data Engineer ...

/ELT pipeline. Proficiency with at least one scripting/programming language (e.g., Python). Familiarity with modern data engineering tools like dbt, Airflow, Prefect, or similar is a plus. Knowledge of cloud platforms (AWS/Azure/GCP). Understanding of data modelling, quality controls and best practices. ...

communicate clearly with technical and non-technical stakeholders Nice to Have Experience with dbt or other transformation frameworks Exposure to data orchestration tools (Airflow, etc.) Understanding of data governance, lineage, and metadata management Experience supporting BI tools (Power BI, Tableau, Looker, etc.) Python experience for data tooling or automation ...

PostgreSQL, or similar databases. 2–4 years of experience with Java or Groovy. 2+ years of experience with orchestration and ingestion tools, such as Airflow or Airbyte. 2+ years of experience working with web service APIs (REST and/or SOAP). Experience with cloud data warehouse ...

Serving, Unity Catalog Feature stores (Databricks Feature Store, Feast) Model monitoring and governance platforms Data Engineering Databricks Lakehouse, Spark Kafka/Confluent Delta Lake Airflow/Databricks Workflows Cloud Platforms Azure, AWS, or GCP (experience with at least one) Data Science & Optimisation Python ecosystem (Pandas, NumPy, SciPy, Statsmodels) Scikit ...

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR ...

integration. Documentation. Your Profile Essential Skills/Knowledge/Experience Strong working and technical knowledge of GCP (GCS Buckets, BigQuery, Dataproc, Pub/Sub), Airflow, Spark, PySpark, Spark SQL, Hive. Good knowledge of Python (Pandas, NumPy, Wiresafe), SQL, Linux CLI, BigQuery. Experience with DBMS, Delta Tables, Oracle, MS Access ...

them usable Strong SQL and scripting/programming skills (Python required; Java/Scala/C++ a plus) Experience with orchestration and workflow tools (Airflow, Cloud Composer, Dagster, etc.) Familiarity with modern data stack components (BigQuery, dbt, Kafka/PubSub, Spark, etc.) Knowledge of data security, access control ...

tooling Good scripting skills (Python/SQL/Bash) Strong understanding of AWS security, IAM, and networking Comfortable operating in Agile, delivery-led environments Apache Airflow for DAG Date-Led Platform/Cloud Engineering Key requirements: Active SC Clearance (essential) Permanent role Ideally you will be based around ...

skills and experience with relational databases Python development experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best ...

skills and experience with relational databases Python development experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best ...