426 to 450 of 885 Apache Airflow Jobs

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

with expertise in Python and experience building reusable tooling and platform services. Experience with modern platform and engineering tooling such as GitHub Actions, Apigee, Airflow, and related cloud‐native technologies. Expertise with observability platforms such as Data Dog, Prometheus, Grafana, ELK, Splunk or equivalent, including monitoring, logging, tracing, reliability ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

data storytelling. A Data Architect: You have a high bar for data quality and have experience working within modern data stacks (e.g BigQuery, dbt, Airflow) What’s In It For You A range of flexible working options to dedicate time to what matters to you Competitive compensation package + ...

Batch, ECS, EKS, S3, RDS or Postgres, Athena, SageMaker, Bedrock, or equivalent services. Scientific workflow and data orchestration frameworks such as Nextflow, Dagster, Airflow or similar. Platform security, privacy or governance controls for sensitive biomedical or clinical data. Identity platforms and federation technologies such as Entra ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

data storytelling. A Data Architect: You have a high bar for data quality and have experience working within modern data stacks (e.g BigQuery, dbt, Airflow) What’s In It For You Competitive compensation package + bonus 25 days holiday per year, increasing to 28 days after 2 years ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

cloud‐native data architecture (AWS/GCP/Snowflake). Architect and implement scalable ingestion and transformation pipelines using Fivetran, Databricks, and Airflow/DBT. Design and code modular data services and APIs that unify campaign, conversion, and audience datasets across 20+ marketing platforms. Build and optimize schemas, transformations … Hands‐on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases. Expert‐level proficiency in Python, SQL, and Apache Spark for data pipeline development. Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks. Solid understanding of data modeling, dimensional design ...

platformsAdvanced Python and SQL skills, with experience in modern engineering practices such as CI/CD and TDDExperience with big data frameworks such as Apache Spark or similar technologiesSolid understanding of data modelling, data warehousing, OLTP and data lake architecturesExperience working with cloud ecosystems such as AWS, Azure … GCPFamiliarity with tools such as Kafka, Airflow, Hive, Delta, or similar technologiesStrong understanding of software engineering principles and production supportExperience with Linux, containerisation and enterprise scheduling toolsKnowledge of data governance, security and data privacy principles including GDPRConfident communicator, able to present complex technical concepts to varied audiencesA proactive, self ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security … with business and technical stakeholders. Preferred qualifications, capabilities, and skills Knowledge of regulatory reporting and financial data aggregation techniques. Experience with data orchestration tools (Airflow, Step Functions, etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet ...

files, webhooks and event streams; Shape the engineering approach across Google Cloud Platform, including BigQuery, Cloud Storage, Pub/Sub, Datastream, Cloud Composer/Airflow, Terraform and infrastructure‐as‐code; Set and champion engineering standards for data modelling, dbt transformations, testing, monitoring, documentation, deployment, version control and cost optimisation … production‐ready data solutions Strong experience with cloud data warehouses, ideally BigQuery, and working knowledge of the wider Google Cloud Platform ecosystem Experience with Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure‐as‐code tooling Understanding of data modelling, dimensional modelling ...

Shape the engineering approach across Google Cloud Platform (GCP) , working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure-as-code tooling . A solid understanding of data ...

database design principles Experience with Microsoft Azure and Azure Data Lake Storage (ADLS) Experience with data engineering tools such as Azure Data Factory, Databricks, Apache Spark, dbt or Airflow Experience with cloud data platforms such as Azure Synapse Analytics or Snowflake Experience using Git for version control Experience ...

Kubernetes - e.g. RDMA, DPUs, Infiniband, many-core CPUs Experience with declarative CI/CD tools such as ArgoCD Experience with workflow engines such as Apache Airflow or Argo Workflows Experience with infrastructure automation Cloud engineering experience with at least one cloud service provider Experience ...

infrastructure. Well versed and proven experiencein programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Longstanding experience ...

join our Data Engineering team in the UK. You will build and govern data infrastructure powering our platforms, using Python ingestion pipelines, Airflow orchestration, and Snowflake/Databricks to move data securely from source to decision-ready output. You will implement end-to-end ingestion sources, investigate ...

hybrid role bridging software and data engineering. You will architect and maintain production-grade data products and APIs using Java/Scala, SQL, Spark, Airflow, and Kubernetes to deliver reliable, high-performance solutions. You will collaborate with Finance, Accounting, and Analytics, drive data quality, and contribute to AI-powered ...

ideal candidate will have strong experience in a lead data engineering position, extensive GCP know-how (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow), and proficiency in Python and SQL. A focus on scalable data pipelines and CI/CD is essential. #J-18808-Ljbffr ...

platform products at scale Excellent written and verbal communication; comfortable engaging directly with clients Right to work in the UK Required skills Snowflake dbt Airflow Python Benefits at SREEYAMSOFT Top‐of‐market pay with annual reviews and performance bonuses. £2,000/year for certifications, conferences and learning. Modern ...

Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added advantage. ...

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle ...

happens next. Technical Mastery: You bring expert-level SQL skills and are fully comfortable building and maintaining your own robust data pipelines using dbt, Airflow, and Snowflake. Beyond querying, you use Python for advanced automation and forecasting, and you apply statistical rigor (e.g., Bayesian statistics, causal inference) to solve ...

integration and deployment Experience supporting production systems Experience with any of the following: gRPC microservices, Postgres, Pandas, Golang, R, Git, Jenkins, Bazel, Prometheus, Grafana, Airflow, Kubernetes “Friends of Voleon” Candidate Referral Program If you have a great candidate in mind for this role and would like to have ...

focused on what works in production. Comfortable owning ambiguity and making trade-overs. Languages: English A plus, but not required: workflow orchestration tools (Temporal, Airflow, Step Functions); experience building internal tools or agents; familiarity with regulated or enterprise environments; exposure to MLOps or AI evaluation frameworks. Why Planet Planet ...