15 of 15 Apache Airflow Jobs in West Yorkshire

Tableau (or similar) Strong knowledge of SQL and experience in testing databases and data warehouses with dBt (e.g., Snowflake - Preferred, Redshift, BigQuery) Strong Knowledge of workload automation platforms like Apache Airflow and dBt (Data Build Tool) Familiarity with CI/CD tools (e.g. Azure DevOps – Preferred, Jenkins) and experience integrating automated tests into pipelines. Experience with cloud platforms … for developing test automation scripts and frameworks. Proficiency with automation testing frameworks (Cucumber, Gherkin, TestNG, or similar) for data testing workloads. Knowledge of performance testing and load testing tools (Apache JMeter or Gatling) Experience: Proven track record in supporting and improving test processes in data-related projects. Experience in leadership, mentoring, or training roles is highly advantageous. Desirable Qualifications More ❯

Python, SQL, the dialect of data. Libraries Tools : Terraform, Flask, Pandas, FastAPI, Dagster, GraphQL, SQLAlchemy, GitLab, Athena. Your Trusted Companions : Docker, Snowflake, MongoDB, Relational Databases (eg MySQL, PostgreSQL), Dagster, Airflow/Luigi, Spark, Kubernetes. Your AWS Kingdom : Lambda, Redshift, EC2, ELB, IAM, RDS, Route53, S3-the building blocks of cloud mastery. Your Philosophy : Continuous integration/deployments, (CI/… Proficiency in statistics and/or machine learning libraries like NumPy, matplotlib, seaborn, scikit-learn, etc. Experience in building ETL/ELT processes and data pipelines with platforms like Airflow, Dagster, or Luigi. What's important for us: Academically Grounded : Bachelor's or Master's degree in Computer Science, Data Engineering, or related field. Seasoned Practitioner : 5+ years of More ❯

or in a similar role. Strong proficiency in SQL and experience with database management systems (e.g., MySQL, PostgreSQL, MongoDB). Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Luigi). Familiarity with cloud platforms and services and a particular knowledge of Google Cloud would be preferable. Proficiency in programming languages such as Python, Java, or More ❯

current data solution as well as advancing it to the next level. We have created an initial gem of a Data Lake and Lakehouse (Azure Data Lake, ADF, Databricks, Airflow, DBT) to enable Business Intelligence and Data Analytics (Superset, RStudio Connect). Our Data Lake is fully metadata driven, cost efficient, documented, and reproducible. We need our one-source … Engineering experience (Preferably Databricks/Azure - or Snowflake/Redshift/BigQuery) (Required) Experience with infrastructure as code (e.g. Terraform) (Required) Proficiency in using Python both for scheduling (e.g. Airflow) and manipulating data (PySpark) (Required) Experience building deployment pipelines (e.g. Azure Pipelines) (Required) Deployment of web apps using Kubernetes (Preferably ArgoCD & Helm) (Preferred) Experience working on Analytics and Data More ❯

have experience with GCP, Data Products, Data Mesh, ETL, EDW Composer, BigQuery, DataProc, DBT, Spark, Hive, Kafka, Pub-Sub, Jira, GCP Dataflow, BigTable, Python, Scala, Java, .NET, ANSI SQL, Airflow, Shell Scripts, Control-M, Git, CI & CD, HDFS, Unix File System, RDBMS, Azure DevOps, Harness. More about the role: Define Data Product Definition, Architecture, and Roadmap. Lead the delivery More ❯

description: Proficiency in GCP, Data Products, Data Mesh, ETL, EDW Composer, BigQuery, DataProc, DBT, Spark, Hive, Kafka, Pub-Sub, Jira, GCP Dataflow, BigTable, Python, Scala, Java, .NET, ANSI SQL, Airflow, Shell Scripts, Control-M, Git, CI & CD, HDFS, Unix File System, RDBMS, Azure DevOps, Harness. More about the role: Able to define Data Product Definition, Architecture, and Roadmap. Define More ❯

the ability to translate data into business context A proactive, detail-oriented mindset and a drive to build systems that scale Nice to Have Experience using Azure Data Factory, Airflow, or similar orchestration tools Exposure to CI/CD pipelines and data quality testing frameworks Previous work with international logistics, supply chain data, or operational analytics Comfort working in More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting edge AR and VR technologies, 3D printing, etc. They … will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring data integrity through … rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in Apache Airflow -Experience with ETL tools, data architecture, and data warehousing solutions -Strong More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting edge AR and VR technologies, 3D printing, etc. They … will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring data integrity through … rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in Apache Airflow -Experience with ETL tools, data architecture, and data warehousing solutions -Strong More ❯

common life sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker, Kubernetes). ZONTAL is an More ❯

common life sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker, Kubernetes). ZONTAL is an More ❯

common life sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker, Kubernetes). ZONTAL is an More ❯

Code • Create Terraform scripts for the infrastructure provisioning • Set up infrastructure security • Implementation of IAM policy and permissions for applications and GitHub • Changes of environment variables for applications and Airflow DAGS If you are interested please hit apply with your updated CV and we will arrange a call to further your application. More ❯

Code • Create Terraform scripts for the infrastructure provisioning • Set up infrastructure security • Implementation of IAM policy and permissions for applications and GitHub • Changes of environment variables for applications and Airflow DAGS If you are interested please hit apply with your updated CV and we will arrange a call to further your application. More ❯

designer and metdata-driven approach Implement scalable and efficient data models (e.g star schema, scd typ2) for data marts and analytics layer Integrate Prophecy pipelines with orchestration tools like Airflow, data catalog tools for lineage Implement version control, automated testing and deployment pipelines using Git and CI/CD (e.g GitHub and Jenkins) Monitor and tune performance of Spark More ❯