251 to 275 of 468 Apache Airflow Jobs in England

Cloud Run), AWS (Bedrock, SageMaker)* Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK* Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar.* Ability to design ...

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

skills and fluency in Python or R, with experience applying analytical and statistical methods to business problems. Experience with orchestration tools (e.g., DBT, Airflow) for deploying reliable data workflows. Strong communication and empathy - ability to translate complex data concepts for non-technical stakeholders. Collaborativeproduct mindset- comfort working closely with ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

secureCost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignorePlatform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)AI Native: experienced in agentic coding workflows (context, harness, loop ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

models, schemas to support business requirementsDevelop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL)You have framework experience within either Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data qualityPreparing ...

Data Engineering. Strong commercial experience with Python. Advanced hands-on knowledge of Databricks, including notebooks, workflows, jobs, and Delta Lake. Strong experience with Apache Spark/PySpark. Proven track record building and maintaining scalable ETL pipelines. Strong SQL and data modelling expertise. Experience working within AWS, Azure … environments. Exposure to modern data warehousing platforms such as Snowflake, BigQuery, or Redshift. Familiarity with orchestration tools including Airflow or Databricks Workflows. Excellent communication skills with the ability to engage technical and non-technical stakeholders. Strong problem-solving mindset and a passion for delivering high-quality data solutions. ...

Shape the engineering approach across Google Cloud Platform (GCP), working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure‐as‐code tooling . A solid understanding of data ...

Shape the engineering approach across Google Cloud Platform (GCP) , working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure-as-code tooling . A solid understanding of data ...

will be used across the organisation, including taking ownership of our data transformation and orchestration tooling; batch and streaming infrastructure and exploration tools (Databricks, Airflow, dbt) and look after our Datalake (ingestion, storage, governance, privacy). You’ll work with a modern, cutting‐edge data stack and play … analytics engineers and have a strong grasp of their needs and how they operate Big data technologies, with expertise in tools & platforms such as Airflow, dbt, Kafka, Databricks and data observability & catalogue) solutions (e.g. Monte Carlo, Atlan, Datahub) Cloud Platform proficiency: familiarity with AWS, GCP, or Microsoft Azure, with ...

Data Engineer Location: Remote – with requirement to visit London based office twice a month Hours: 37.5 Hours a week (Monday – Friday 9:00am – 17:00pm) Salary Range: £40,000 - £80,000 per annum (Depending on ...

understanding of data management and data quality principles, with experience developing data quality checks or reports Prior experience using the Bloomberg Terminal Experience with Apache Airflow or similar tools For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy ...

business intelligence in a security-conscious environment. The role requires 3+ years in data engineering, strong experience with Spark, Python, Kafka, SQL and Airflow, and a willingness to work hybrid in London or Cheltenham. You will contribute to client engagements and growth of the company. #J-18808-Ljbffr ...

data products. You will leverage SQL, Python, and cloud data warehouses (BigQuery, Snowflake, or Redshift) and orchestrate batch or streaming pipelines with Airflow, Prefect, or Dagster, partnering with analytics, data science, and product engineering. #J-18808-Ljbffr ...

maintain scalable systems for application development and large-scale data processing, architecting production-grade data products and APIs using Java/Scala, SQL, Spark, Airflow, and Kubernetes. You will collaborate with Finance, Accounting, Analytics, and engineering teams to translate business requirements into robust solutions, while driving data quality ...

development experience, with strong skills in Python. Hands‐on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow #J-18808-Ljbffr ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added advantage. ...

development experience, with strong skills in Python. Hands-on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

Fluent English – for international communication and collaboration. Additional Information Nice to have: Experience with LLMs, or Time Series, Familiarity with tools like Docker, MLflow, Airflow, AWS/GCP/Azure, Background in research or R&D, Experience working in cross-functional or international teams. Soft Skills: Curiosity and problem ...

degree, preferably in Computer Science, Engineering, Mathematics, or similar technical discipline. Nice to Have Snowflake SQL expertise. AWS Cloud experience – including EC2 & S3. Airflow/MWAA or equivalent job scheduler experience. Kubernetes, EKS, Lambda, or serverless experience in AWS. Personal Attributes Team player with strong analytical and problem‐solving ...

similar databases. Experience designing data models, schemas and transformation layers for complex datasets. Experience with directed acyclic graph–based orchestration tools such as Airflow, Dagster, Prefect or similar systems. Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure. Experience with data quality testing, lineage, observability, versioning ...

including building and publishing images to internal registries Implement Infrastructure as Code using Terraform (including network, security and compute resources) Support workflow orchestration, including Airflow Contribute to platform security, auditing and cost management best practices Participate as a core contributor within the team and support architecture discussions/working ...