12 of 12 Contract Apache Airflow Jobs in the UK

Data Engineer in secure government, defence, or intelligence settings. * Strong Python and SQL development skills. * Hands-on experience with data integration tools like Airflow, NiFi, Azure Data Factory, or AWS Glue. * Expertise in Elasticsearch management, indexing, and query optimisation. * Knowledge of data modelling, warehousing, and modern data platform architectures. ...

reviews to maintain high engineering standards. As a Data Engineer you will have the following key skills Cloud: GCP Warehousing: BigQuery/Snowflake Orchestration: Airflow Backend: Python, SQL, Shell Languages: Scala/Java DevOps: Terraform, Docker, Kubernetes Observability: Monte Carlo BI: Looker AI/ML: MLOps London | 12 Months ...

Snowflake Toolkit: Hands-on with Snowpipe, Streams, Tasks, Dynamic Tables, and Stored Procedures. Nice to Have: Exposure to DBT, Snowpark, Snowflake AI/ML, Airflow, and Snowflake administration (RBAC, warehouse sizing). Mindset: Strong problem-solving abilities with an ownership mindset for end-to-end delivery. This ...

/SD/PP etc.) required for December cutover. Build scalable ingestion pipelines using Delta Lake/Parquet and modern orchestration tools (ADF, dbx, Airflow, etc.). Collaborate with SAP functional teams to validate business logic, deltas, and data quality. Implement CI/CD and DevOps practices for data ...

with experience handling large datasets Experience building and maintaining ETL/ELT pipelines in production environments Hands-on experience with workflow orchestration tools (e.g. Airflow ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

with Snowflake. Knowledge of MongoDB or other NoSQL databases. Experience working within AWS or other cloud environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience with market data providers such as Bloomberg, Refinitiv or ICE. Exposure to enterprise data management platforms. Knowledge of Kafka ...

abstraction layers Model the cost, feasibility and risk of different LLMs and implementation approaches THE STACK AWS, Bedrock, LangGraph, LangChain, Python, FastAPI, PostgreSQL, Snowflake, Airflow, React, Next.js, Kubernetes. YOUR EXPERIENCE 15+ years in enterprise software engineering with a background in product engineering organisations or technology companies Deeply technical: recent ...

Python, Bash, Ansible, Puppet) Familiarity with containerisation (Docker, Kubernetes) Nice to have: Experience with CI/CD pipelines (GitLab, Jenkins) Exposure to data tooling (Airflow, NiFi, Elasticsearch) Experience with Cloudera/Hadoop ecosystems (Spark, Hive, HDFS) Why join? Work on modern data platforms at scale Collaborate with data scientists ...

Ensure solutions meet security, compliance, and regulatory standards Key Skills Required: Experience in solution design/architecture Strong background in big data tools (Spark, Airflow, Trino, etc.) Knowledge of cloud platforms (AWS/Azure/GCP) Understanding of cybersecurity principles (IAM, encryption, secure design) Ability to work hands ...

This is a great opportunity to contribute greatly to an extremely engineering-focused organization and further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform, Google Cloud Platform, etc; You will own engineering efforts to provide partner teams within the organisation with data sources that build … opportunity for the role to go permanent. What You'll Do: Build and maintain scalable data pipelines for BigQuery using Cloud Composer and Airflow; Provide observability and monitoring using Monte Carlo and Looker as well as operational tools such as NewRelic and Splunk, driving reliability, data quality, data quality ...

focus areas: Building scalable data pipelines using Apache Spark & Airflow Working within a cloud-based, containerised environment (Kubernetes) Contributing to a wider data platform modernisation programme Supporting both batch and streaming workloads Tech: Python, Java, Spark, Airflow, SQL, Cloud (AWS/Azure), Kubernetes (Kafka/streaming nice ...