19 of 19 Apache Airflow Jobs in the South East

databases utilised in modern AI applications. Strong Python development skills. Data & Infrastructure: Experience with some of the following technologies is highly desirable: Frameworks & Infrastructure: Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch Databases: PostgreSQL MongoDB Vector databases such as Qdrant, Milvus, or pgvector Python Ecosystem: Pandas Polars ...

Experience using Git/version control Strong communication skills with both technical and non-technical stakeholders Desirable Skills: Microsoft Fabric Azure Synapse Analytics Databricks Apache Spark/PySpark dbt Apache Airflow Power BI Azure DevOps CI/CD pipelines Terraform What's On Offer: £65k+ salary package ...

Experience with cloud platforms such as AWS, Azure, or GCP. Knowledge of Data Warehouse and Data Lake architectures. Experience with orchestration tools such as Airflow, ADF, or equivalent. Understanding of data governance, data quality, and security practices. Familiarity with CI/CD pipelines and version control tools. Preferred Skills ...

Data pipeline development and optimisation Lakehouse architectures ETL/ELT frameworks Enterprise data platform implementation Cloud Technologies (AWS) Experience with: Amazon S3 AWS Glue Apache Airflow IAM AWS Lambda (desirable) Data Architecture & Modelling Knowledge and practical experience of: Data Vault Kimball methodology Dimensional modelling Data warehousing principles Enterprise ...

Requirements Passion for building scalable, resilient cloud data platforms. Strong experience with Databricks or Snowflake on AWS. Proven Python skills, including Spark and Airflow expertise. Advanced SQL skills and end-to-end data modelling experience. Experience building batch and real-time data integrations. Hands-on CICD skills with ...

Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments CI/CD experience with a focus on production ownership and reliability ...

maintaining scalable data pipelines and integrations. Developing robust ETL and data transformation processes to support analytics and reporting requirements. Working with Snowflake, DBT, Apache Airflow, AWS, SQL and Python to deliver high-quality data solutions. Collaborating with Data Architecture, DevOps and business stakeholders to ensure trusted and accessible … ideally within AWS. Data modelling, transformation and orchestration methodologies. Delivering high-quality, well-documented and thoroughly tested data solutions. Experience with Snowflake, DBT and Apache Airflow. Interest in AI and emerging technologies within the data engineering landscape. THE BENEFITS: You will receive a salary ...

preparation specifically geared toward AI model integration Technical Fluency: A strong conceptual understanding of modern data ecosystems, data warehousing, pipeline orchestration tools (e.g., dbt, Airflow), and data governance standards Cross-Functional Communication: Outstanding stakeholder management skills, with a demonstrated ability to foster collaboration between technical developers and non-technical ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Familiarity with Kafka concepts (producers ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

modern cloud based data platforms. Solid understanding of data modelling and data transformation principles. Ability to work effectively within collaborative engineering teams. Desirable Experience: Airflow Databricks Experience supporting AI or machine learning initiatives. Semantic layer experience. Exposure to Azure and multi cloud environments. Understanding of Inmon 3NF data modelling ...

join their London-based team. Key Responsibilities: Design and maintain scalable data pipelines across diverse sources. Automate and optimise workflows using tools like Airflow, dbt, and Spark. Support data modelling for analytics, dashboards, and A/B testing. Collaborate with cross-functional teams to deliver data-driven insights. Work ...

learning services. Experience with machine learning frameworks such as TensorFlow, PyTorch, or scikit-learn. Knowledge of pipeline orchestration and data workflow tools such as Airflow, Dataform, or dbt. Experience delivering advanced measurement and modelling solutions, including marketing mix modelling, incrementality testing, causal impact analysis, attribution modelling, propensity modelling, churn ...

Backend Engineering background Experience building APIs and system integrations Experience working within complex, data-intensive environments Strong understanding of software engineering best practices Desirable Airflow Snowflake AWS Exposure to data warehousing and modern analytics platforms Experience supporting reporting, commercial analytics, or performance data solutions IDEAL BACKGROUND We're especially ...

We are recruiting for an Analytical Engineer on behalf of one of the UK's most recognised and celebrated personal lines insurers. The successful candidate will be working alongside a talented team of Data & AI ...

building and optimising Semantic Layers for BI, reporting and analytics consumption Expert Level Data Modelling Strong hands-on experience with: Snowflake AWS dbt Databricks Airflow Ability to work in a fast-paced delivery environment and remain close to the technical implementation Desirable -Customer Loyalty, Rewards or Membership Programme experience ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs. Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB … lakehouse architectures.- Familiarity with big data processing frameworks like Spark. Data Engineering Knowledge:- Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow, Azure Data Factory).- Understanding of JSON, XML, APIs, and data ingestion technologies. Cloud Experience:- Familiarity with cloud platforms (Azure ...

/CD experience. Highly Desirable Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Order book reconstruction Tick-to-bar aggregation Airflow, Dagster, Prefect or MLflow Experience supporting quantitative research or trading systems Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs) What We're Looking … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

include: Designing and building scalable data pipelines using Python and SQL Developing solutions with Snowflake, dbt and modern orchestration tools such as Dagster or Airflow Building and improving cloud-native data infrastructure on AWS (or transferable GCP experience) Developing scalable data models and supporting the wider platform architecture Improving … platforms in fast-moving technology businesses. You'll ideally have: Strong Python and SQL experience Experience with Snowflake Experience using dbt Experience with Dagster, Airflow or another modern orchestration framework AWS experience (GCP also considered) Strong understanding of data modelling principles Experience building end-to-end data platforms ...