51 to 75 of 94 Apache Airflow Jobs in the UK

data quality checks and monitoring processes. KEY SKILLS Proficiency in SQL and data querying validation and testing purposes. Hands on experience with Snowflake or Airflow or DBT. Familiarity with data integration, ETL processes, and data governance frameworks. Solid understanding of data structures, relational databases and data modelling concepts. Experience more »

learning systems. Expert in Python. Knowledge of common ML libraries like Pandas, NumPy and Scikit-Learn. Experience with ML workflow orchestration tools (Kubeflow, MLFlow, Airflow etc). Experience with Vertex AI Understanding of CI/CD pipelines and DevOps processes. Ability to collaborate with data scientists and software engineers. more »

algorithms Expertise in popular data science platforms such as Alteryx and Python, including libraries and frameworks like NumPy, SciPy, Pandas, NLTK, TensorFlow, PyTorch, and Airflow Strong understanding of statistical analysis, encompassing distributions, statistical testing, regression, and other techniques Experience handling unstructured data sets Familiarity with software engineering principles and more »

TMS (Tealium IQ, GTM and Adobe Dynamic Tag Manager) changes. Integrate data sources via web and REST APIs. Data pippingand modelling using SQL, DBT, Airflow, ETL, Data Warehousing, Redshift and Python. Transfer knowledge of the business processes and requirements to the development teams. Collaborate with Product, Marketing and Development more »

diverse environments, leveraging Azure and other modern technologies. Proven ability to orchestrate complex data workflows and manage Kubernetes clusters on AKS , utilizing tools like Airflow, Kubeflow, Argo, and Dagster. Familiarity with data ingestion tools such as Airbyte and Fivetran, accommodating a wide array of data sources. Mastery of large more »

Proven experience in MLOps and deploying machine learning models on Kubernetes. Proficiency in cloud technologies, AWS, GCP, Azure Experience with data orchestration tools (e.g., Apache Airflow). Familiarity with Terraform for CI/CD and infrastructure as code. Strong programming skills in software development. A cloud-agnostic mindset more »

S3, Lambda, Aurora, Docker/Kubernetes. AWS Cloud Management and Optimisation: Manageandoptimize cloud-based infrastructureonAWS, ensuring security,scalability, reliability, and cost-efficiency. MUST HAVE: Apache Airflow, AWS networking concepts such as VPN, VPC peering, subnets, security groups, NAT gateways and how to implement scalable and fault tolerant network more »

data manipulation and analysis Experience with Kubernetes and Docker for scalable infrastructure deployment Hands on experience with CI/CD and Orchestration tools eg. Airflow, Jenkins, Bamboo, etc. Experience with MongoDB is highly beneficial Knowledge of data pipeline development and optimization Familiarity with data science principles and workflows is more »

Langchain, Semantic Kernel, and tools like MS tooling, Co-Pilot Studio, ML Studio, Prompt flow, Kedro, etc. Proficiency in pipeline orchestration with tools like Airflow, Kubeflow, and Argo . Exceptional communication skills, with the ability to articulate complex statistical concepts clearly. Personal Competencies: A results-oriented professional with a more »

for advanced content analysis and indexing & developing RAG services . Experience in managing data workflows and Kubernetes clusters on AKS, utilizing tools such as Airflow, Kubeflow, Argo, and Dagster. Familiarity using scripting languages and tools such as Bash, PowerShell, Azure CLI, Terraform, and Helm Charts. Additional Information Location : This more »

4.Monitoring and Logging: Implement and maintain monitoring, logging, and alerting solutions. Key technologies: AWS, VPN, VPC Peering, EC2, S3, Lambda, Aurora, Docker/Kubernetes. Apache Airflow, AWS networking concepts such as VPN, VPC peering, subnets, security groups, NAT gateways.AWS CloudWatch or equivalent. Kafka or similar data streaming components. more »

on with projects but no requirements, previous experience in this is essential SKILLS AND EXPERIENCE NEEDED: Experience in Redshift database (AWS) Experience with DBT, Airflow and Fivetran Working collaboratively with multiple teams across the business Management/mentoring experience of a team INTERVIEW PROCESS: 1st Stage- Initial Chat 2nd more »

tooling: containers (e.g., Docker) container orchestration (Kubernetes/K8s) CI/CD experience Version control (Git, Github, Gitlab) Orchestration/DAGs tools (e.g., Argo, Airflow, Kubeflow) Infrastructure as Code (Terraform, etc.) HOW TO APPLY Please register your interest by sending your CV to niall.wharton@@xcede.com or click the Apply more »

and maintain scalable data architectures, including pipelines and cloud-based data warehouses. Tech: Python (NumPy, Pandas), SQL, ETL, Cloud (AWS, Azure or GCP), Snowflake, Airflow, BigQuery, PowerBI/Tableau Industry: Fintech, Maritime trading Immersum are supporting the growth of a specialist consultancy who solely specialise in the Maritime trading … developing and optimising ETL pipelines. Version Control: Experience with Git for code collaboration and change tracking. Data Pipeline Tools: Proficiency with tools such as Apache Airflow. Cloud Platforms: Familiarity with AWS, Azure, Snowflake, and GCP. Visualisation: Tableau or PowerBI Delivery Tools: Familiarity with agile backlogs, code repositories, automated builds more »

Experienced creating data pipelines on a cloud (preferably AWS) environment CI/CD experience Containerization experience (Docker, Kubernetes, etc.) Experience with SQS/SNS, Apache Kafka, RabbitMQ Other interesting/bonus skills – Airflow, Trino, Apache Iceburg, Postgres, MongoDB You *must* be eligible to work in your chosen more »

Key technical experience: Ability to operate in a fast changing environment. Fluent in English Previous cloud based infrastructure experience, particularly with AWS. Experience using Airflow and dbt Expert SQL knowledge Solid understanding of Dimensional Data Modelling. Experience with at least one or more of these programming languages: Python, Scala …/Java Experience with distributed data and computing tools, mainly Apache Spark & Kafka Understanding of critical path approaches, how to iterate to build value, engaging with stakeholders actively at all stages. Able to deal with ambiguity and change. A self-starter who's able to work independently where necessary more »

is data analysis, building enterprise data platform within asset management or Financial Services is required Expert level Java/Python Development skills AWS expert - Apache Iceberg/Spark and Airflow Please apply for immediate consideration more »

hosted data platform that would be used by the entire firm. Tech Stack: Python/Java, Dremio, Snowflake, Iceberg, PySpark, Glue, EMR, dbt and Airflow/Dagster. Cloud: AWS, Lambdas, ECS services This role would focus on various areas of Data Engineering including: End to End ETL pipeline development … Implementing Data Curation, metadata management and data quality tooling. Requirements: Strong Python/Java Software Engineering skills Excellent AWS knowledge, ideally with exposure to Airflow, Glue, Iceberg and Snowflake Previous experience with Dremio, dbt, EMR or Dagster Good Computer Science fundamentals knowledge with strong knowledge of software and data more »

CD, and model monitoring. Proficiency in Python and relevant data manipulation and analysis libraries (e.g., pandas, NumPy). Experience with distributed computing frameworks like Apache Spark is a plus. Apache Spark and Airflow would be a bonus. Role overview: If you're looking to work with a more »

CD, and model monitoring. Proficiency in Python and relevant data manipulation and analysis libraries (e.g., pandas, NumPy). Experience with distributed computing frameworks like Apache Spark is a plus. Apache Spark and Airflow would be a bonus. Role overview: If you're looking to work with a more »

includes: building greenfield market data processing systems that will handle very large amounts of market/exchange data, data lake development using AWS Glue, Apache Kafka/Airflow and dbt, and more traditional end-to-end Data Pipeline development. Lastly, part of this role would look at the … Java or Python skills - ability to write production ready code Experience deploying code to AWS utilising relevant tools. Previous experience with AWS Glue, dbt, Airflow and Dagster Good communication skills, and ability to work both independently and collaboratively with a team. Excellent SQL skills For more information, please apply more »

projects and new innovations to support company growth and profitability. Our Tech Stack Python Scala Kotlin Spark Google PubSub Elasticsearch Bigquery, PostgresQL Kubernetes, Docker, Airflow Ke y Responsibilities Designing and implementing scalable data pipelines using tools such as Apache Spark, Google PubSub etc. Optimising data storage and retrieval … Data Infrastructure projects, as well as designing and building data intensive applications and services. Experience with data processing and distributed computing frameworks such as Apache Spark Expert knowledge in one or more of the following languages - Python, Scala, Java, Kotlin Deep knowledge of data modelling, data access, and data more »

Senior AWS Data Engineer - London (Hybrid) - Permanent - £65,000-£70,000 Media Agency, Snowflake, SQL, Python, dbt, Airflow, Marketing *You must be based in London, and have full permanent right to work in the UK to apply for this role* I'm currently working with a leading media agency … ASAP and I will be in touch. Senior AWS Data Engineer - London (Hybrid) - Permanent - £65,000-£70,000 Media Agency, Snowflake, SQL, Python, dbt, Airflow, Marketing more »

and you’ll build the solution. They work in a mix of SQL, Python programming but also operate with modern tech such as Snowflake, Airflow, DBT, Kubernetes/Docker and cubeJS whilst also using Tableau CRM and Jupyter notebooks for gaining data insights. If you think there’s a … to be a SQL Server expert with strong architecture skills and also strong skills with Python, and DBT. The more experience with things like Airflow, cubeJS and Snowflake, plus other data viz tools would be a huge plus. Total compensation for this role is up to £250,000, split more »

VERY OCCASIONAL TRAVEL TO CLIENT SITES AND OFFICE. Would you like to the opportunity to expand your skillset across Java, Python, Spark, Hadoop, Trino & Airflow across the Banking & Financial Services industries? How about if you worked with an Innovation Partner of the Year Winner (2023 Oracle EMEA Partner Awards … trends and best practices, and share knowledge with the team. Qualifications You will have expertise within the following: Java, Python, Spark, Hadoop (Essential) Trino, Airflow (Desirable) Architecture and capabilities. Designing and implementing complex solutions with a focus on scalability and security. Excellent communication and collaboration skills. Additional Information Why more »