76 to 100 of 103 Permanent Apache Airflow Jobs

hosted data platform that would be used by the entire firm. Tech Stack: Python/Java, Dremio, Snowflake, Iceberg, PySpark, Glue, EMR, dbt and Airflow/Dagster. Cloud: AWS, Lambdas, ECS services This role would focus on various areas of Data Engineering including: End to End ETL pipeline development … Implementing Data Curation, metadata management and data quality tooling. Requirements: Strong Python/Java Software Engineering skills Excellent AWS knowledge, ideally with exposure to Airflow, Glue, Iceberg and Snowflake Previous experience with Dremio, dbt, EMR or Dagster Good Computer Science fundamentals knowledge with strong knowledge of software and data more »

CD, and model monitoring. Proficiency in Python and relevant data manipulation and analysis libraries (e.g., pandas, NumPy). Experience with distributed computing frameworks like Apache Spark is a plus. Apache Spark and Airflow would be a bonus. Role overview: If you're looking to work with a more »

CD, and model monitoring. Proficiency in Python and relevant data manipulation and analysis libraries (e.g., pandas, NumPy). Experience with distributed computing frameworks like Apache Spark is a plus. Apache Spark and Airflow would be a bonus. Role overview: If you're looking to work with a more »

includes: building greenfield market data processing systems that will handle very large amounts of market/exchange data, data lake development using AWS Glue, Apache Kafka/Airflow and dbt, and more traditional end-to-end Data Pipeline development. Lastly, part of this role would look at the … Java or Python skills - ability to write production ready code Experience deploying code to AWS utilising relevant tools. Previous experience with AWS Glue, dbt, Airflow and Dagster Good communication skills, and ability to work both independently and collaboratively with a team. Excellent SQL skills For more information, please apply more »

projects and new innovations to support company growth and profitability. Our Tech Stack Python Scala Kotlin Spark Google PubSub Elasticsearch, Bigquery, PostgresQL Kubernetes, Docker, Airflow Key Responsibilities Designing and implementing scalable data pipelines using tools such as Apache Spark, Google PubSub etc. Optimising data storage and retrieval systems … Data Infrastructure projects, as well as designing and building data intensive applications and services. Experience with data processing and distributed computing frameworks such as Apache Spark Expert knowledge in one or more of the following languages - Python, Scala, Java, Kotlin Deep knowledge of data modelling, data access, and data more »

Senior AWS Data Engineer - London (Hybrid) - Permanent - £65,000-£70,000 Media Agency, Snowflake, SQL, Python, dbt, Airflow, Marketing *You must be based in London, and have full permanent right to work in the UK to apply for this role* I'm currently working with a leading media agency … ASAP and I will be in touch. Senior AWS Data Engineer - London (Hybrid) - Permanent - £65,000-£70,000 Media Agency, Snowflake, SQL, Python, dbt, Airflow, Marketing more »

and you’ll build the solution. They work in a mix of SQL, Python programming but also operate with modern tech such as Snowflake, Airflow, DBT, Kubernetes/Docker and cubeJS whilst also using Tableau CRM and Jupyter notebooks for gaining data insights. If you think there’s a … to be a SQL Server expert with strong architecture skills and also strong skills with Python, and DBT. The more experience with things like Airflow, cubeJS and Snowflake, plus other data viz tools would be a huge plus. Total compensation for this role is up to £250,000, split more »

VERY OCCASIONAL TRAVEL TO CLIENT SITES AND OFFICE. Would you like to the opportunity to expand your skillset across Java, Python, Spark, Hadoop, Trino & Airflow across the Banking & Financial Services industries? How about if you worked with an Innovation Partner of the Year Winner (2023 Oracle EMEA Partner Awards … trends and best practices, and share knowledge with the team. Qualifications You will have expertise within the following: Java, Python, Spark, Hadoop (Essential) Trino, Airflow (Desirable) Architecture and capabilities. Designing and implementing complex solutions with a focus on scalability and security. Excellent communication and collaboration skills. Additional Information Why more »

VERY OCCASIONAL TRAVEL TO CLIENT SITES AND OFFICE. Would you like to the opportunity to expand your skillset across Java, Python, Spark, Hadoop, Trino & Airflow across the Banking & Financial Services industries? How about if you worked with an Innovation Partner of the Year Winner (2023 Oracle EMEA Partner Awards … trends and best practices, and share knowledge with the team. Qualifications You will have expertise within the following: Java, Python, Spark, Hadoop (Essential) Trino, Airflow (Desirable) Architecture and capabilities. Designing and implementing complex solutions with a focus on scalability and security. Excellent communication and collaboration skills. Additional Information Why more »

analysis solutions. You will collaborate closely with cross-functional teams to design, implement, and optimize Python-based applications, with a particular emphasis on utilizing Airflow and also pandas for efficient data manipulation and analytics. This role offers an exciting opportunity to contribute to the transformation of their business through … reviews, identify areas for improvement, and implement best practices to ensure code quality and maintainability. Stay updated on emerging trends and technologies in Python, Airflow, pandas, and data engineering, and incorporate them into our development practices. Qualifications Proven experience as a Python Developer, with a strong focus on data … analytics. Solid understanding of pandas for data manipulation and analysis in big data environments. Proficiency in SQL for querying and manipulating relational databases. Strong Airflow experience Strong problem-solving skills and the ability to work effectively in a collaborative team environment. Excellent communication skills with the ability to convey more »

Role: Senior Data Engineer 💾 Stack: Snowflake, AWS, Airflow, DBT and Tableau Excited to combat climate change and make a real impact? 🌍 We've teamed up with a startup dedicated to creating a net-zero product! 🚀 Over the past three years, they've built an outstanding in-house engineering team … Process: Two stages 💸 Salary: Base salary can go to £85k (bonus + up to 22% pension) 📊Key Skills Required: Experience in Snowflake (essential), AWS, Airflow, DBT and Tableau Experience in Agile environment Experience in start-up or scale-up environment No recent CV handy? No sweat! Just shoot me more »

the above project of redesigning the Creditsafe platform into the cloud space. You will be expected to work with technologies such as Python, Linux, Airflow, AWS DynamoDB, S3, Glue, Athena, Redshift, lambda, API Gateway, Terraform, CI/CD. KEY DUTIES AND RESPONSIBILITIES You will actively contribute to the codebase … and participate in peer reviews. Design and build metadata driven, event based distributed data processing platform using technologies such as Python, Airflow, Redshift, DynamoDB, AWS Glue, S3. As an experienced Engineer, you will play a critical role in the design, development, and deployment of our business-critical system. You more »

In this role you will be working within a new, greenfield division of the business, using a brand-new technology stack including Snowflake, dbt, Airflow and AWS. This function provide data for Machine Learning and Artificial Intelligence capabilities, helping them to provide the best possible service offering to their … Experience collaborating with Architects to design and lead the implementation of new data solutions Extensive experience with Snowflake Experience or working knowledge of dbt, Airflow and AWS is highly desirable Experience leading and managing a high-performing technical team Very strong stakeholder management experience, ideally including external suppliers You more »

In this role you will be working within a new, greenfield division of the business, using a brand-new technology stack including Snowflake, dbt, Airflow and AWS. This function provide data for Machine Learning and Artificial Intelligence capabilities, helping them to provide the best possible service offering to their … Experience collaborating with Architects to design and lead the implementation of new data solutions Extensive experience with Snowflake Experience or working knowledge of dbt, Airflow and AWS is highly desirable Experience leading and managing a high-performing technical team Very strong stakeholder management experience, ideally including external suppliers You more »

In this role you will be working within a new, greenfield division of the business, using a brand-new technology stack including Snowflake, dbt, Airflow and AWS. This function provide data for Machine Learning and Artificial Intelligence capabilities, helping them to provide the best possible service offering to their … looking for the following experience: Experience liaising with data consumers to understand their requirements Extensive experience with Snowflake Experience or working knowledge of dbt, Airflow and AWS is highly desirable Experience developing, testing and maintaining practical data architectures Experience resolving complex issues and identifying improvements in the testing and more »

In this role you will be working within a new, greenfield division of the business, using a brand-new technology stack including Snowflake, dbt, Airflow and AWS. This function provide data for Machine Learning and Artificial Intelligence capabilities, helping them to provide the best possible service offering to their … looking for the following experience: Experience liaising with data consumers to understand their requirements Extensive experience with Snowflake Experience or working knowledge of dbt, Airflow and AWS is highly desirable Experience developing, testing and maintaining practical data architectures Experience resolving complex issues and identifying improvements in the testing and more »

Job Title: Data Engineer Job Type: Full Time, Permanent Working location: London, Hybrid Role Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more »

Role: Graduate Data Engineer Type: 12 months fixed-term Location: Peterborough Ready to utilise your skills to process and extract value from large datasets? Are you passionate about performing root cause analysis on various data? We have an exciting role more »

enhance their systems. You will be involved with project migration and business-critical upgrades. You must have experience with Azure Data Factory – Google Cloud – Airflow, along with SQL and ETL experience. This could be a great career move with personal development plans to grow your career. You will require … a blend of the following: - Strong SQL Query experience Knowledge of Azure Data Factory is desirable Python Development knowledge Knowledge of Apache Airflow. Experience with enterprise DBMS and, ideally, Google BigQuery. Experience in Google Cloud Platform services. Any knowledge of Power BI and Tableau is useful. In return, our more »

North America. - Build trusted client relationships. - Solve problems creatively and scope projects effectively. - Expertise in Google Cloud and modern data stack technologies (Fivetran, DBT, Airflow, BigQuery, etc). - Lead and sometimes work within project teams. - Manage multiple concurrent projects and meet client deadlines. What you need: - Have hands-on … experience with Google Cloud. - Have experience working with a mixture of Fivetran, BT, Airflow, Cube, Rudderstack, Snowflake, and Looker. - Possess outstanding analytical and problem-solving skills. - Demonstrate proficiency in SQL/Python. - Excel in consulting roles with the ability to manage multiple engagements – client-facing experience is a must more »

a MedTech company with a growing reputation in their field. The Data Architect will use their skills and experience of technologies including Kubernetes, Kafka, Airflow, Linux, AWS and Python to spearhead the evolution and enhancement of their existing data infrastructure, as they scale up the adoption of their software … the Software as a Medical Device industry Experience of designing and implementing scalable and secure data infrastructure Hands-on experience with the Kubernetes, Kafka, Airflow, Python, Ubuntu Linux, AWS (or similar cloud platforms) Strong database skills using SQL - Any experience with PostgreSQL or MongoDB would be advantageous An understanding more »

a MedTech company with a growing reputation in their field. The Data Architect will use their skills and experience of technologies including Kubernetes, Kafka, Airflow, Linux, AWS and Python to spearhead the evolution and enhancement of their existing data infrastructure, as they scale up the adoption of their software … the Software as a Medical Device industry Experience of designing and implementing scalable and secure data infrastructure Hands-on experience with the Kubernetes, Kafka, Airflow, Python, Ubuntu Linux, AWS (or similar cloud platforms) Strong database skills using SQL - Any experience with PostgreSQL or MongoDB would be advantageous An understanding more »

Are you ready to be a game-changer in the world of sports data? Do you want to be part of a team that's rewriting the playbook on sports analytics? This company is on a mission to revolutionise the more »

data practices Possess strong knowledge of data tools, data management tools, and various data and information technologies. E.g. DAMA DMBOK, Microsoft SQL Server, Couchbase, Apache Druid, Spark, Kafka, Airflow, etc In-depth understanding of modern data principles, methodologies, and tools Excellent communication and collaboration skills, with the ability … native computing concepts and experience working with hybrid or private cloud platforms is a plus. Demonstrable technical experience working with a Microsoft, Redhat, and Apache data and software engineering environment. A team-oriented individual with a passion for engineered excellence and the ability to lead and motivate a team more »

using Docker, Kubernetes and Cloud services. Experience with Azure stack will be an asset. Experience designing and implementing event-driven/microservices applications using Apache Kafka, Flink, etc. Exposure to model deployment and serving tools like Seldon Core, KServe, etc. Experience with drift detection and adaptation techniques as well … and familiarity with tools to manage infrastructure as code, like Terraform and package managers like Helm Charts. Proficiency with pipeline orchestration tools, such as Airflow, Kubeflow, and Argo Workflows. Outstanding communication skills, ability to convey complex technical concepts to non-technical stakeholders and collaborate with cross-functional teams. A more »