18 of 18 Contract Apache Airflow Jobs in England

applications, working closely with data engineers, analysts, and platform teams. Day to day duties Developing and maintaining data pipelines and processing jobs using Apache Spark and Python Orchestrating workflows with Apache Airflow Provisioning and managing cloud infrastructure on AWS using Terraform (Infrastructure as Code) Containerising applications … teams in an Agile environment, contributing to design reviews, code reviews, and documentation Essential Skills & Experience Strong hands-on experience with Python Experience with Apache Spark for large-scale data processing Practical knowledge of AWS services (eg S3, Lambda, EC2, Glue, RDS - adjust as relevant) Experience writing and maintaining ...

Optimise Python performance through effective use of loops , vectorization , threading , and understanding of the Global Interpreter Lock (GIL) . Desirable Experience Experience with Kubeflow , Airflow , or dbt . Hands-on experience with Google Cloud Platform (GCP) . Knowledge of containerisation (Docker, Kubernetes) and CI/CD practices. Familiarity with ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Familiarity with Kafka concepts (producers ...

communication skills. Highly Desirable: Experience with Spark/PySpark. Familiarity with Azure Data Lake, Snowflake, or similar platforms. Knowledge of orchestration tools like Airflow or Azure Data Factory. Exposure to financial services environments. Experience supporting large-scale data transformation or migration programmes. What We're Looking For: A senior ...

data platforms, migrating legacy environments into cloud-native solutions using the latest AWS and Snowflake technologies. Key skills: Snowflake, Teradata & Informatica Python & SQL AWS, Airflow & DBT GitHub & CI/CD ETL & Data Warehousing Control-M, Unix/Linux Data Modelling & Performance Optimisation You'll design and optimise scalable data ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

modern cloud based data platforms. Solid understanding of data modelling and data transformation principles. Ability to work effectively within collaborative engineering teams. Desirable Experience: Airflow Databricks Experience supporting AI or machine learning initiatives. Semantic layer experience. Exposure to Azure and multi cloud environments. Understanding of Inmon 3NF data modelling ...

both Terraform and Terragrunt deploying data platforms (preferably Databricks Experience designing scalable data pipelines for machine learning workloads Experience implementing orchestration tools such as Apache Airflow or equivalent. Knowledge of data modelling, dimensional modelling and data warehousing principles Whether you're developing new products, building scalable platforms ...

learning services. Experience with machine learning frameworks such as TensorFlow, PyTorch, or scikit-learn. Knowledge of pipeline orchestration and data workflow tools such as Airflow, Dataform, or dbt. Experience delivering advanced measurement and modelling solutions, including marketing mix modelling, incrementality testing, causal impact analysis, attribution modelling, propensity modelling, churn ...

demonstrable experience designing, deploying, and operating big data platforms at scale. You will have hands-on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies). You will have knowledge of cloud platforms and traditional on-premise hosting; hands-on experience with Azure ...

Backend Engineering background Experience building APIs and system integrations Experience working within complex, data-intensive environments Strong understanding of software engineering best practices Desirable Airflow Snowflake AWS Exposure to data warehousing and modern analytics platforms Experience supporting reporting, commercial analytics, or performance data solutions IDEAL BACKGROUND We're especially ...

background as a Machine Learning Engineer. Technical Skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib). Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services). Expertise: Solid understanding of computer science fundamentals and time-series ...

building and optimising Semantic Layers for BI, reporting and analytics consumption Expert Level Data Modelling Strong hands-on experience with: Snowflake AWS dbt Databricks Airflow Ability to work in a fast-paced delivery environment and remain close to the technical implementation Desirable -Customer Loyalty, Rewards or Membership Programme experience ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs. Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB … lakehouse architectures.- Familiarity with big data processing frameworks like Spark. Data Engineering Knowledge:- Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow, Azure Data Factory).- Understanding of JSON, XML, APIs, and data ingestion technologies. Cloud Experience:- Familiarity with cloud platforms (Azure ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills: - Proficient in Python and SQL, with experience in Apache Spark. - Strong experience building backend services and APIs. Database Expertise: - Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB … lakehouse architectures. - Familiarity with big data processing frameworks like Spark. Data Engineering Knowledge: - Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow, Azure Data Factory). - Understanding of JSON, XML, APIs, and data ingestion technologies. Cloud Experience: - Familiarity with cloud platforms (Azure ...

driving engineering best practices. Ability to influence technical direction and architectural decisions. Comfortable remaining hands-on while operating in a leadership capacity. Desirable Skills Airflow Databricks AI-focused development experience Semantic layer experience Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse ...

management * Software testing lifecycle/defect management * CI/CD & DevOps practices Desirable: HMRC/public sector experience, AWS/Azure/SAS PaaS, Airflow or SAS Migration Factory experience. If you are interested in this role or wish to apply, please submit your CV. ...

Months contract 2 Days a month onsite - Expenses to Newcastle depends on their hub. Start 3 from offer Experience required AWS Services Terraform Apache Spark Python Airflow Docker Nice to haves Gitlab Security Scanning Agents (eg Trivy, Trend Micro, Wiz etc) Jupyter Notebooks Integration with DWP Integration Team ...