1 to 25 of 115 Apache Airflow Jobs in England

highly regulated financial settings. Key Technologies & Tools Core ML: scikit-learn, PyTorch, TensorFlow, XGBoost, LightGBM Data Manipulation: Pandas, NumPy, LangGraph MLOps & Orchestration: MLflow, Apache Airflow Databases: Neo4j, MongoDB API Development: FastAPI Big Data (Beneficial): Apache Hadoop, Apache Spark (PySpark) What We Need From ...

similarly regulated industry. Strong coding skills in Python or Java (Spring Boot); React experience is a plus. Proficiency with modern data tools: Airflow, Spark, Kafka, dbt, Snowflake or similar. Experience with cloud platforms (AWS, Azure, GCP), containerization (Docker, Kubernetes), and CI/CD. Data Quality: Proven ability to validate ...

technical stakeholders. • Experience working in agile environments with cross-functional teams. • Self-motivated with the ability to manage multiple projects simultaneously. • Experience with Kafka, Airflow, or other data orchestration tools would be advantageous. • Knowledge of distributed computing frameworks (Spark, Hadoop) would be advantageous. • Exposure to AI/Deep Learning ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

architecture, modelling, pipelines) (please be clear on years and/or projects) Strong Python and SQL Experience with orchestration and transformation frameworks such as Airflow, dbt, Dagster , or similar Experience designing and delivering data platforms or lakehouse-style architectures Solid understanding of CI/CD for data engineering (e.g. ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and Terraform. What you get in return: Up to £60,000 per annum + benefits Hybrid working ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services ( AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage … . Exposure to workflow orchestration tools ( Apache Airflow, Prefect, or Dagster ). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

/ELT pipeline. Proficiency with at least one scripting/programming language (e.g., Python). Familiarity with modern data engineering tools like dbt, Airflow, Prefect, or similar is a plus. Knowledge of cloud platforms (AWS/Azure/GCP). Understanding of data modelling, quality controls and best practices. ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

Data Engineer: Experience with Databricks , Kafka , or Kinesis for real-time data streaming Knowledge of containerisation (Docker, ECS) and modern orchestration tools such as Airflow Familiarity with machine learning model deployment pipelines or data lakehouse architectures Data Engineer, AWS Data Engineer ...

/ELT pipeline. Proficiency with at least one scripting/programming language (e.g., Python). Familiarity with modern data engineering tools like dbt, Airflow, Prefect, or similar is a plus. Knowledge of cloud platforms (AWS/Azure/GCP). Understanding of data modelling, quality controls and best practices. ...

communicate clearly with technical and non-technical stakeholders Nice to Have Experience with dbt or other transformation frameworks Exposure to data orchestration tools (Airflow, etc.) Understanding of data governance, lineage, and metadata management Experience supporting BI tools (Power BI, Tableau, Looker, etc.) Python experience for data tooling or automation ...

communicate clearly with technical and non-technical stakeholders Nice to Have Experience with dbt or other transformation frameworks Exposure to data orchestration tools (Airflow, etc.) Understanding of data governance, lineage, and metadata management Experience supporting BI tools (Power BI, Tableau, Looker, etc.) Python experience for data tooling or automation ...

Serving, Unity Catalog Feature stores (Databricks Feature Store, Feast) Model monitoring and governance platforms Data Engineering Databricks Lakehouse, Spark Kafka/Confluent Delta Lake Airflow/Databricks Workflows Cloud Platforms Azure, AWS, or GCP (experience with at least one) Data Science & Optimisation Python ecosystem (Pandas, NumPy, SciPy, Statsmodels) Scikit ...

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR ...

integration. Documentation. Your Profile Essential Skills/Knowledge/Experience Strong working and technical knowledge of GCP (GCS Buckets, BigQuery, Dataproc, Pub/Sub), Airflow, Spark, PySpark, Spark SQL, Hive. Good knowledge of Python (Pandas, NumPy, Wiresafe), SQL, Linux CLI, BigQuery. Experience with DBMS, Delta Tables, Oracle, MS Access ...

them usable Strong SQL and scripting/programming skills (Python required; Java/Scala/C++ a plus) Experience with orchestration and workflow tools (Airflow, Cloud Composer, Dagster, etc.) Familiarity with modern data stack components (BigQuery, dbt, Kafka/PubSub, Spark, etc.) Knowledge of data security, access control ...

skills and experience with relational databases Python development experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best ...

skills and experience with relational databases Python development experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best ...

Athena, Lambda, RDS/Aurora). - Expertise in data modeling, schema design, normalization, and multidimensional modeling. - Experience with ETL/ELT tools (Glue, Informatica, Airflow). - Knowledge of Big Data technologies (Spark, Hadoop, Kafka). - Strong understanding of API-based integrations. Preferred Skills: - NoSQL experience (MongoDB, DynamoDB). - Snowflake ...

Python (and ideally C#) programming skills You have experience with data modelling, ETL pipelines and data integration You have a good knowledge of Dagster, Airflow or similar orchestration tool You're collaborative and pragmatic with great stakeholder communication skills What's in it for you: Salary to £120k Pension ...

role. Strong proficiency with Python and SQL . Palantir Foundry experience ideally up to and at/around 2+ years Hands-on experience with Airflow or equivalent orchestration frameworks. Solid understanding of data warehouses, data lakes, and integration tools . Proficiency with AWS and Azure cloud ecosystems. Experience building ...