26 to 50 of 154 Apache Airflow Jobs

Experience with CI/CD, microservices, TDD/BDD. Knowledge of secure system design. Desirable (AI & Data) Vector DBs: Milvus, Weaviate, Pinecone Data orchestration: Airflow, Prefect, Dagster Search: ElasticSearch/OpenSearch AI frameworks: LangChain, Hugging Face MLOps: MLflow, Kubeflow If you are looking for an opportunity to work ...

across ML Ops, data architecture and data engineering best practices and scalable data solutions. Proficiency in Python and SQL, ideally with frameworks such as Airflow, PyTorch or Spark. Good understanding of supervised and unsupervised learning and how to choose/apply the right model to a business problem. Familiarity ...

cloud data platforms (AWS preferred) Deep experience with AWS, Databricks, Power Platform, and Redshift (Snowflake a plus) Proficiency in AWS Glue, Qlik Talend, DBT, Airflow, and modern data integration tools. Excellent knowledge of Python, SQL, PowerQuery (M), and preferably Scala or PySpark Working knowledge with enterprise architecture frameworks (e.g. ...

performance marketing data Experience working on large-scale data transformation or platform modernisation projects Tech Stack Warehouse: Snowflake Transformation: dbt BI: Looker Orchestration: Airflow Version Control: GitHub Other Tools: Adverity, OpenMetadata, Jira, Confluence Benefits Hybrid working (minimum of 3 days per week in office) 25 days annual leave plus ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

Experience with CI/CD, microservices, TDD/BDD. Knowledge of secure system design. Desirable (AI & Data) Vector DBs: Milvus, Weaviate, Pinecone Data orchestration: Airflow, Prefect, Dagster Search: ElasticSearch/OpenSearch AI frameworks: LangChain, Hugging Face MLOps: MLflow, Kubeflow If you are looking for an opportunity to work ...

data/AI platforms Deep knowledge of GCP (BigQuery, Dataflow, Pub/Sub, Vertex AI) Experience with LLMs (e.g. Gemini or similar) Familiarity with Airflow/Composer and dbt Understanding of data governance and financial services regulation Strong stakeholder management skills What’s on Offer Competitive salary + bonus ...

familiarity with data preparation, transformation, visualisation, and cloud deployment. Experience with Google Cloud Platform and tools such as Google Pub/Sub, Dataflow, Dataproc, Airflow, Dataprep, Data Fusion. Experience with CI/CD and DevOps tooling (e.g., Jenkins, Kubernetes). Experience working within Agile delivery models (Scrum, Kanban), including ...

/lending/risk/portfolio datasets Time series familiarity (trend/seasonality, cohorts over time, monitoring) Experience with data tools such as dbt, Airflow, Snowflake/BigQuery/Databricks (or similar) Experience creating automated data quality frameworks or reconciliation checks Benefits: £45,000 – £55,000 salary Hybrid working ...

Deep understanding of data platforms, orchestration , metadata management , CI/CD and architectural components such as DevOps, Azure Data Factory, BigQuery, Databricks, MS Fabric, Airflow, Snowflake etc. Demonstrated experience architecting solutions using medallion architectures , including curated gold-layer models designed for analytics consumption. Strong knowledge of unity catalog ...

associated services. Strong programming skills in Python, Java, or Scala for data engineering. Hands-on experience with data orchestration and workflow management tools (e.g., Airflow, Dagster, Prefect). Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion …/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). Deep knowledge of data transformation and ETL pipelines and APIs. Security cleared or ability to obtain (SC or above). Desirable Skills ...

experience. Hands-on experience with Hadoop - configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

efficient methods * Multi-modal AI systems combining text, image, and structured data * Reinforcement Learning from Human Feedback (RLHF) for model alignment Production ML Systems: * Apache Airflow/Dagster for ML workflow orchestration and ETL pipeline management * Model versioning and experiment tracking (MLflow, Weights & Biases) * Real-time model serving ...

processes with strong monitoring, logging and error handling Develop and maintain data models to support reporting and analytics Manage workflow orchestration and scheduling (e.g. Airflow or similar) Support CI/CD, version control and deployments across dev, test and production environments Implement data quality checks, governance and documentation Collaborate ...

discipline Solid Python programming skills Good working knowledge of SQL Comfortable using Git for version control Desirables: Exposure to workflow orchestration tools (e.g. Prefect, Airflow, Dagster) Experience with cloud data warehouses (Azure SQL, Snowflake) or dbt Basic familiarity with Docker and BI tools (Power BI, Tableau) Interest in shipping ...

pipeline design Mastery of data modeling techniques Skilled in data ingestion, integration, and transformation using modern tools (Data Factory, Databricks, SQL DB, Synapse, Glue, Airflow, Kinesis, Redshift) Hands-on experience with DevOps tooling: GitHub Actions, Azure DevOps, SonarQube, PyTest Proven ability to coordinate incident, problem, and change management processes ...

learn new systems and business contexts Technical Requirements Experience with Azure SQL Databases Familiarity with Azure-hosted environments (including Ubuntu VMs) Working knowledge of Apache Airflow for pipeline orchestration Strong Power BI skills (data modelling, report building, optimisation) Proficiency in Python for data processing and automation Experience integrating ...

reporting alongside project work Strong communication skills, particularly around handover and knowledge transfer Technology in use: Azure SQL Databases Azure Ubuntu Virtual Machines Apache Airflow Power BI Python REST & SOAP APIs To apply for this role please submit your CV or contact David Airey on or at . ...

architecture to support data consumption for reporting systems, operational applications, and analytics platforms. Implement automation, orchestration, and scalable pipeline frameworks using tools such as Apache Airflow. Collaborate with business stakeholders and technical teams to align the data platform with enterprise data strategy. Support hospitality analytics use cases, including guest ...

understanding ML pipelines and models in a production setting. Direct experience with Google Cloud Platform, BigQuery, and associated tooling. Experience with workflow tools like Airflow or Kubeflow. Familiarity with dbt (Data Build Tool). Please send your CV for more information on these roles. Reasonable Adjustments: Respect and equality ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8–10 years ’ experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...