51 to 75 of 112 Permanent Apache Airflow Jobs

technical trade-offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with (or strong advocacy for) AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. ...

Experience working within cloud environments (AWS or Azure) Strong SQL skills and experience working with large-scale datasets Familiarity with orchestration tools such as Airflow or similar Good understanding of data modelling and distributed data processing concepts Comfortable working across multiple client environments and stakeholders 🎁 What’s on Offer ...

Experience with CI/CD, microservices, TDD/BDD. Knowledge of secure system design. Desirable (AI & Data) Vector DBs: Milvus, Weaviate, Pinecone Data orchestration: Airflow, Prefect, Dagster Search: ElasticSearch/OpenSearch AI frameworks: LangChain, Hugging Face MLOps: MLflow, Kubeflow If you are looking for an opportunity to work ...

/ELT development, orchestration frameworks, cloud-based data platforms, and data quality management. You will be comfortable working with technologies such as dbt, Airflow, PostgreSQL, cloud warehouses or lakehouses, and modern data architecture patterns. Most importantly, you'll be someone who enjoys ownership, takes pride in building reliable solutions ...

/ELT development, orchestration frameworks, cloud-based data platforms, and data quality management. You will be comfortable working with technologies such as dbt, Airflow, PostgreSQL, cloud warehouses or lakehouses, and modern data architecture patterns. Most importantly, you'll be someone who enjoys ownership, takes pride in building reliable solutions ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

across the data engineering domain. Engage with data engineering technical detail when needed, understanding the data platform landscape including data ingestion, transformation (DBT), orchestration (Airflow), streaming (Kafka), and data warehousing (Snowflake), and able to articulate technical choices and trade‐offs to different stakeholders. Work at both conceptual and detailed … despite complexity in data requirements and priorities. What I need to know: Strong understanding of data engineering technologies including data ingestion, transformation (DBT), orchestration (Airflow/Astronomer). Experience with cloud data platforms (AWS data services, Azure data services) including cost management and optimisation Knowledge of data quality frameworks ...

prompt engineering skills and the ability to critically evaluate AI-generated code for correctness, security, and financial accuracy Desirable, but no essential: Experience with Apache Airflow or similar workflow schedulers REST or GraphQL API design experience Knowledge of fixed income, FX, or derivatives products and associated ...

responsibilities Own, assess, improve, or rebuild existing ETL/ELT pipelines Build reliable data workflows using Python Work with orchestration tools such as Airflow or similar Improve data quality, validation, lineage, and troubleshooting Collaborate with consultants and non-technical stakeholders Help translate messy business/client data into structured ...

data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform or CDK) Experience with workflow orchestration tools such as Airflow or similar Ability to design scalable architectures, make pragmatic trade-offs, and communicate effectively with both technical and non-technical stakeholders Processing Your Data ...

retrieval augmented generation (RAG) architectures. * Strong knowledge of cloud platforms including AWS or Azure. * Experience with data engineering and orchestration tools such as Databricks, Airflow, Docker, and Kubernetes is desirable. * Must have previous experience working within a small to medium-sized business (SME), start-up, or scale-up environment. ...

experience with Databricks, Spark and distributed data processing technologies Cloud expertise across AWS, Azure and/or GCP Experience using orchestration tools such as Airflow A strong understanding of data platform reliability, scalability and governance Proven experience influencing technical strategy and customer decision-making The ability to lead, mentor ...

engineering, model evaluation, and experiment design (A/B testing, holdout strategies, stratification). Experience developing production-quality data pipelines and automated workflows using Airflow or similar orchestration tools. Practical familiarity with graph databases and/or graph frameworks (Neo4j, AWS Neptune, GraphFrames, DGL, PyTorch Geometric) and graph algorithms ...

libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like private healthcare & dental insurance, a generous work from ...

/CD terminology and pipelines in GIT, Jira Cloud & Infrastructure: AWS BedRock, SageMaker AI, Azure Machine Learning, GCP Vertex AI, Kubernetes, Terraform Data Engineering: Apache Kafka, Spark, Airflow, Parquet, SQL, NoSQL and graph databases. Optimization & Scheduling: Constraint optimization (eg. LP), Reinforcement Learning. Education Minimum Bachelors in Computer Science ...

requires a candidate with 8+ years of software and data engineering experience, focusing on API development using Spring Boot and big data technologies like Apache Spark and Airflow. Responsibilities include designing scalable systems, collaborating with cross-functional teams, and mentorship. Candidates should have strong programming skills, SQL expertise ...

development experience, with strong skills in Python. Hands‐on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

skills ✔ Experience working with cloud platforms (AWS preferred) ✔ Knowledge of data modelling, data architecture and governance ✔ Experience with orchestration tools such as Airflow or Databricks Workflows ✔ Comfortable working closely with stakeholders, working independantly delivering on high profile projects and mentoring team members ✔ Experience delivering complex data solutions from scratch ...

skills ✔ Experience working with cloud platforms (AWS preferred) ✔ Knowledge of data modelling, data architecture and governance ✔ Experience with orchestration tools such as Airflow or Databricks Workflows ✔ Comfortable working closely with stakeholders, working independantly delivering on high profile projects and mentoring team members ✔ Experience delivering complex data solutions from scratch ...

XGBoost, or equivalent tools Experience building and deploying software services, APIs, data pipelines, or internal platforms using tools such as FastAPI, Flask, SQL, Spark, Airflow, dbt, or similar technologies Experience working with time-series, sensor, event, equipment, MES, historian, quality, or other industrial data Experience training, validating, and deploying ...

APIs can be exposed to LLM-based agents. Experience with search relevance & recommendation systems — ranking algorithms, semantic search, personalisation. Hands-on with Snowflake, Databricks, Airflow, Kafka . Why Join Us Work on firm-wide platforms used by thousands of data scientists, ML engineers, and analysts across JPMC Shape ...

love to see: Knowledge of Database Systems and trade offs in the distributed systems Familiarity with API Designs Familiarity with Orchestration Frameworks such as Apache Airflow, Argo Workflows, Dagster etc. Experience working with and designing systems utilizing AWS Bloomberg is an equal opportunity employer and we value diversity ...

understanding of functional paradigms and type theory. Confident JVM knowledge. Modern programming languages knowledge in our primary languages (Java, Python, Clojure). Experience with Airflow or other Python-based workflow orchestration tools. Proficiency in domain-driven design and domain modeling. Exposure to Kubernetes, Docker, Linux, Kafka, RabbitMQ, or git. ...

Collaborating closely with consultants, developers, and project teams Helping shape the future of the company’s data capability Tech You’ll Work With Python Airflow Git & GitHub RDF/SPARQL/Semantic Web technologies Cloud and modern data tooling What They’re Looking For Strong experience building scalable data ...