201 to 225 of 250 Apache Airflow Jobs

/security implications. You might also have: Experience with Kubernetes for cloud-burst or elastic compute scenarios. Familiarity with workflow orchestration tools (e.g. Prefect, Airflow, Dagster) — not required to design flows, but useful for understanding deployment/runtime needs. Exposure to high-performance/scientific/technical computing environments ...

/security implications. You might also have: Experience with Kubernetes for cloud-burst or elastic compute scenarios. Familiarity with workflow orchestration tools (e.g. Prefect, Airflow, Dagster) — not required to design flows, but useful for understanding deployment/runtime needs. Exposure to high-performance/scientific/technical computing environments ...

develop. On the data pipeline side, we value awareness across both established tools such as SSIS/SSRS and newer approaches such as dbt, Airflow, or AWS Glue — dbt experience is essential, with the others a strong plus. DesirablePrevious experience in working in cloud environments (AWS preferred … definite plus. Familiarity with legacy SSIS/SSRS, useful for maintaining existing packages, as well as newer orchestration/ingestion tools such as Airflow or AWS Glue. Exposure to cloud data warehouse/data fabric platforms such as Snowflake or Databricks. A good understanding of financial markets. Background ...

Docker and Kubernetes experience, including deployment and management Previous experience working within air-gapped environments Strong understanding of APIs and microservices Experience with Prefect, Airflow or similar data pipeline/orchestration technologies Experience configuring and integrating workflow tooling such as Jira Experience with database and storage technologies such ...

deploying, monitoring and maintaining ML systems at scale Ability to balance hands-on engineering with people management responsibilities Experience with AWS, Snowflake, MLflow, Airflow or similar tooling would be advantageous SALARY AND BENEFITS £110,000 - £125,000 base salary Annual bonus Hybrid working model Private medical insurance Enhanced family ...

than one-off scripts or analysis tooling. Strong Python development skills, with experience building performant, maintainable applications. Experience with workflow orchestration tools such as Airflow, Dagster, or Prefect. Understanding of retries, dependency management, idempotency, backfills, and operational recovery. Experience with analytical or columnar databases such as ClickHouse or similar ...

data analysis packages such as Numpy, Pandas, etc. Experience with SQL and/or other database technologies Knowledge of modern technologies such as AWS, Airflow, Docker, GitLab and Redis Ideally working with data visualization technologies such as Dash, Plotly, You will be joining a progressive and exciting company committed ...

coursework in financial engineering, or industry internshipsExperience developing in a production-facing environment and familiarity with standard concepts and tooling, e.g., CI/CD, Airflow“Friends of Voleon” Candidate Referral ProgramIf you have a great candidate in mind for this role and would like to have the potential ...

This position will be posted for a minimum of five calendar days internally. All internal applicants are required to inform your current supervisor/manager prior to applying. Internal applicants are required to be in ...

Bristol 50000 - 72000 per annum + Excellent benefits PermanentSenior Data Engineer – Python, SQL, Airflow, Azure, Pyspark, KubernetesSalary: Up to 72,000 per annumBenefits: Excellent benefits package including annual bonus, free onsite gym, and company pension scheme.About the RoleWe are seeking an experienced Senior Data Engineer to join an innovative … ResponsibilitiesData Pipeline DevelopmentData Platform ManagementStakeholder CollaborationLeadership & MentorshipTechnology StackThe successful candidate will have experience working with some or all of the following technologies:Data Processing: Apache Spark/PySparkWorkflow Orchestration: Apache AirflowContainerisation: KubernetesCloud Platform: Microsoft AzureData APIs & Semantic Layer: CubeJSProgramming Languages: Python, SQLEssential Skills & ExperienceDegree in a STEM discipline ...

Engineering, or MLOps environments Experience building, deploying, and supporting machine learning solutions at scale Experience working with AWS and ML tooling such as MLflow, Airflow, Metaflow, or Snowflake Ability to balance technical leadership, strategic thinking, and hands-on engineering contribution SALARY AND BENEFITS Up to £125,000 base salary ...

technical design reviews. Must Haves (Hard Skills)Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… Avro).Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow).Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python, SQL, or an Object-Oriented ...

Experience designing data products across source-aligned, aggregate and consumer-aligned patterns. Proven ability to architect and scale modern cloud data platforms using Snowflake, Apache Iceberg, Amazon Redshift, and S3.Hands-on expertise in developing robust data transformation pipelines and orchestrating complex workflows using dbt and Apache Airflow. Strong ...

data infrastructure, while contributing to strong engineering practices and standards. The environment is heavily focused on Google Cloud Platform (GCP), with BigQuery, Dataform and Airflow/Cloud Composer forming key parts of the technology stack. WHAT YOU'LL BE DOING * Designing and implementing complex data flows across operational, analytics … clearly defined. ESSENTIAL EXPERIENCE * Strong hands-on experience with Google Cloud Platform (GCP). * Strong experience with BigQuery. * Experience using Dataform. * Experience with Airflow/Cloud Composer. * Strong data engineering fundamentals, including building scalable and robust production data solutions. * Experience working with data quality, governance and appropriate data controls. ...

data infrastructure, while contributing to strong engineering practices and standards. The environment is heavily focused on Google Cloud Platform (GCP), with BigQuery, Dataform and Airflow/Cloud Composer forming key parts of the technology stack. WHAT YOU'LL BE DOING * Designing and implementing complex data flows across operational, analytics … clearly defined. ESSENTIAL EXPERIENCE * Strong hands-on experience with Google Cloud Platform (GCP). * Strong experience with BigQuery. * Experience using Dataform. * Experience with Airflow/Cloud Composer. * Strong data engineering fundamentals, including building scalable and robust production data solutions. * Experience working with data quality, governance and appropriate data controls. ...

maintain standardized data models, schemas, and metadata that make commodities datasets easy to understand, discover, and reuse. Schedule, monitor, and manage data pipelines using Airflow or similar workflow orchestration tools to ensure reliable, timely data delivery. Implement validation, reconciliation, and anomaly-detection controls to maintain data completeness, accuracy … building and maintaining ETL pipelines. Strong Python and SQL skills, including experience working with large datasets and complex data transformations. Hands-on experience with Airflow or comparable workflow orchestration tools. Experience with Git, CI/CD pipelines such as GitHub Actions, and automated testing frameworks such as PyTest. Strong ...

experience with cloud platforms (AWS, Azure, GCP)Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow)Practical experience with NLP and/or Search techniquesPrompt engineering, and LLMs for enterprise-scale applications. You have team lead experienceStrong ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

development API gateway technologies, including Tyk, Apigee or DataPower Cloud and application hosting using PCF, TAS or AWS Integration and batch processing using StreamSets, Airflow, ESP Services or GSS CI/CD pipelines and automated testing Technical design, code reviews and production troubleshooting You'll have the opportunity ...

elevate our current data pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg). Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork … Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg). dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background in designing ...

frameworks and API design (FastAPI, Flask, or Django) for building internal data services and RESTful APIs.Familiarity with enterprise workload scheduling tools (e.g., Autosys, Airflow, Control-M) and enterprise messaging systems (e.g., Apache Kafka, IBM MQ).Familiarity with containerization technologies (Docker, OpenShift, Kubernetes) and cloud infrastructure concepts.Experience in Linux ...