1 to 25 of 26 Apache Airflow Jobs in Scotland

following tools: NiFi, Hbase, Bash, Assist, Putty, Neo4J, Spark , Kafka, HDFS, Oozie, Git Hub, Unix, Hadoop, Impala, DoJo, Flume, Elastic, Logstash, Kibana, Airflow, Glue, Big Query, Athena, CML, Hive, Informatica, CuteFTP Ability to explain and communicate technical concepts in non-technical language. Explain the types of communication that ...

following tools: NiFi, Hbase, Bash, Assist, Putty, Neo4J, Spark , Kafka, HDFS, Oozie, Git Hub, Unix, Hadoop, Impala, DoJo, Flume, Elastic, Logstash, Kibana, Airflow, Glue, Big Query, Athena, CML, Hive, Informatica, CuteFTP Ability to explain and communicate technical concepts in non-technical language. Explain the types of communication that ...

following tools: NiFi, Hbase, Bash, Assist, Putty, Neo4J, Spark , Kafka, HDFS, Oozie, Git Hub, Unix, Hadoop, Impala, DoJo, Flume, Elastic, Logstash, Kibana, Airflow, Glue, Big Query, Athena, CML, Hive, Informatica, CuteFTP Ability to explain and communicate technical concepts in non-technical language. Explain the types of communication that ...

millions of developers, data practitioners, and AI agents worldwide from high-level data visualization plugins (Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tooling. Our work directly shapes how companies process massive datasets: real-time analytics platforms ingesting … should build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse ...

millions of developers, data practitioners, and AI agents worldwide from high-level data visualization plugins (Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tooling. Our work directly shapes how companies process massive datasets: real-time analytics platforms ingesting … should build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse ...

prompt engineering, RAG architectures, fine-tuning, Agentic AI frameworks (LangChain, LlamaIndex, AutoGen). Data Engineering : ETL/ELT patterns, data modeling, Snowflake, Databricks, dbt, Airflow, Kafka/streaming architectures. Cloud Platforms : AWS (SageMaker, Bedrock, Glue), Azure (OpenAI Service, Synapse), GCP (Vertex AI, BigQuery). MLOps : Model deployment, monitoring, versioning ...

Knowledge of regulatory reporting and treasury operations in retail banking Exposure to Python, Go or similar languages. Experience working with orchestration frameworks such as Airflow/Luigi Have previously used dbt, dataform or similar tooling. Used to AGILE ways of working (Kanban, Scrum) The Interview Process: Our interview process ...

Knowledge of regulatory reporting and treasury operations in retail banking Exposure to Python, Go or similar languages. Experience working with orchestration frameworks such as Airflow/Luigi Have previously used dbt, dataform or similar tooling. Used to AGILE ways of working (Kanban, Scrum) The Interview Process: Our interview process ...

Proven experience writing highly maintainable, performant Python and JavaScript code; Experience building modern data pipelines using dbt, Kafka, Spark, AWS Kinesis, AWS Lambda, and Apache Airflow (or similar); Understanding of Data Modelling patterns; Deep knowledge of complex SQL, with emphasis on Common Table Expressions, window functions, and their ...

. Deep understanding of neural networks, reinforcement learning, and generative models . Experience with cloud platforms (AWS, GCP, Azure) and MLOps tools (MLflow, Kubeflow, Airflow, Docker, Kubernetes). Strong knowledge of SQL/NoSQL databases and data engineering practices . Soft Skills: Excellent problem-solving, communication, and leadership abilities. ...

. Deep understanding of neural networks, reinforcement learning, and generative models . Experience with cloud platforms (AWS, GCP, Azure) and MLOps tools (MLflow, Kubeflow, Airflow, Docker, Kubernetes). Strong knowledge of SQL/NoSQL databases and data engineering practices . Soft Skills: Excellent problem-solving, communication, and leadership abilities. ...

with cross-functional teams to deliver high-impact data solutions. Key Responsibilities: Architect and maintain robust data pipelines using AWS services (Glue, Lambda, S3, Airflow) Lead the migration and optimisation of data workflows into Snowflake. Collaborate with analysts, data scientists, and product teams to deliver clean, reliable data. Define ...

habits. Is highly capable with Python and SQL in real data engineering environments. Has experience with modern data tooling such as Snowflake, dbt, Dagster, Airflow, AWS, or similar. Understands data modelling principles and can apply them pragmatically. Cares about testing, monitoring, alerting, and data quality. Thrives in a startup ...

habits. Is highly capable with Python and SQL in real data engineering environments. Has experience with modern data tooling such as Snowflake, dbt, Dagster, Airflow, AWS, or similar. Understands data modelling principles and can apply them pragmatically. Cares about testing, monitoring, alerting, and data quality. Thrives in a startup ...

habits. Is highly capable with Python and SQL in real data engineering environments. Has experience with modern data tooling such as Snowflake, dbt, Dagster, Airflow, AWS, or similar. Understands data modelling principles and can apply them pragmatically. Cares about testing, monitoring, alerting, and data quality. Thrives in a startup ...

design, including data quality testing, cleansing, and monitoring, and data warehousing and data modelling capabilities Experience of cloud technologies including AWS, Snowflake, SQL, and Apache Airflow, SQL and programming languages such as PL/SQL and Python, Spark, and Pyspark Good knowledge of modern code development practices Strong ...

data modelling Experience integrating messy external APIs and bulk datasets Strong Postgres Experience with a transformation framework ( dbt or SQLMesh ) and an orchestrator ( Dagster , Airflow, Prefect, or similar). Modern Python data tooling; DuckDB for analytical compute. Financial Services/Wealth domain would be ideal but not essential ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

Looking For Strong experience designing and building scalable data pipelines Proficiency in SQL, Python, Databricks or AWS Experience with tools such as Git and Airflow Proven ability to test, troubleshoot and improve data platforms Experience delivering high-quality, reliable data solutions Strong communication skills and ability to work ...

culture of engineering excellence, continuous learning, and cross-pollination of knowledge. Own the technical roadmap for Airalo’s data infrastructure (GCP, BigQuery), orchestration (Dagster, Airflow), and ingestion (Fivetran, custom APIs). Drive data-platform architecture decisions, turning ambiguous business problems into scalable, production-grade technical designs. Bridge the data … background with major cloud platforms (GCP preferred) and cloud-native data warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering ...

culture of engineering excellence, continuous learning, and cross-pollination of knowledge. Own the technical roadmap for Airalo’s data infrastructure (GCP, BigQuery), orchestration (Dagster, Airflow), and ingestion (Fivetran, custom APIs). Drive data-platform architecture decisions, turning ambiguous business problems into scalable, production-grade technical designs. Bridge the data … background with major cloud platforms (GCP preferred) and cloud-native data warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering ...

building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement robust ETL/… solutions for both structured and unstructured data. Automate workflows and orchestrate jobs using Airflow and GitHub Actions. Integrate data with third-party APIs to support real-time marketing insights. Collaborate closely with cross-functional teams including Data Science, Software Engineering and Product. Champion best practices in data governance, observability ...

building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement robust ETL/… solutions for both structured and unstructured data. Automate workflows and orchestrate jobs using Airflow and GitHub Actions. Integrate data with third-party APIs to support real-time marketing insights. Collaborate closely with cross-functional teams including Data Science, Software Engineering and Product. Champion best practices in data governance, observability ...

services and tools, such as Salesforce, Segment, HubSpot, Stripe is beneficial Experience with SQL and data warehousing/pipelining tools such as BigQuery, Airflow, etc Proven experience in navigating and transforming legacy codebases, as well as leading greenfield projects ...