5 of 5 Apache Airflow Jobs in Renfrewshire

millions of developers, data practitioners, and AI agents worldwide from high-level data visualization plugins (Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tooling. Our work directly shapes how companies process massive datasets: real-time analytics platforms ingesting … should build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse ...

prompt engineering, RAG architectures, fine-tuning, Agentic AI frameworks (LangChain, LlamaIndex, AutoGen). Data Engineering : ETL/ELT patterns, data modeling, Snowflake, Databricks, dbt, Airflow, Kafka/streaming architectures. Cloud Platforms : AWS (SageMaker, Bedrock, Glue), Azure (OpenAI Service, Synapse), GCP (Vertex AI, BigQuery). MLOps : Model deployment, monitoring, versioning ...

habits. Is highly capable with Python and SQL in real data engineering environments. Has experience with modern data tooling such as Snowflake, dbt, Dagster, Airflow, AWS, or similar. Understands data modelling principles and can apply them pragmatically. Cares about testing, monitoring, alerting, and data quality. Thrives in a startup ...

design, including data quality testing, cleansing, and monitoring, and data warehousing and data modelling capabilities Experience of cloud technologies including AWS, Snowflake, SQL, and Apache Airflow, SQL and programming languages such as PL/SQL and Python, Spark, and Pyspark Good knowledge of modern code development practices Strong ...

building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement robust ETL/… solutions for both structured and unstructured data. Automate workflows and orchestrate jobs using Airflow and GitHub Actions. Integrate data with third-party APIs to support real-time marketing insights. Collaborate closely with cross-functional teams including Data Science, Software Engineering and Product. Champion best practices in data governance, observability ...