76 to 100 of 179 Apache Airflow Jobs in England

modern cloud based data platforms. Solid understanding of data modelling and data transformation principles. Ability to work effectively within collaborative engineering teams. Desirable Experience: Airflow Databricks Experience supporting AI or machine learning initiatives. Semantic layer experience. Exposure to Azure and multi cloud environments. Understanding of Inmon 3NF data modelling ...

Platforms Databricks Delta Lake Unity Catalog dbt Spark/PySpark Data pipeline development Lakehouse and modern data platform architectures Cloud Technologies AWS S3 Glue Airflow Data Modelling & Architecture Data Vault Kimball Data warehousing concepts and best practices Modernisation Experience Experience supporting migrations from technologies such as: Teradata Hadoop Oracle ...

implementing batch and stream data processing systems. Experience with Datadog, SQL, Terraform, Docker, AWS (ECS, Kinesis, SQS, CloudWatch), GCP (BigQuery). Knowledge of Airflow, Kafka, Flink, dbt, Kubernetes is highly beneficial. Good understanding of AI platforms, experience building infrastructure for LLM orchestration (e.g., vector databases, RAG pipelines, GPU scaling ...

MLOps principles with experience deploying models in production Proficiency in Python and tools/platforms such as Azure, AWS, GCP, Databricks, MLFlow, and Airflow Strong communication skills with the ability to engage business and technical stakeholders We offer/Benefits EPAM Employee Stock Purchase Plan (ESPP) Protection benefits including ...

database querying languages Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, Observability tools (Dynatrace, Splunk, Grafana), and Orchestration tools (Airflow, Temporal) Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test acceleration ...

cluster management and ML training workloads (NVIDIA, CUDA, distributed training). Familiarity with MLOps tooling: Experiment tracking (MLflow, Weights & Biases). Workflow orchestration (Airflow, Prefect, Argo). Data versioning (DVC). Background in scientific computing or HPC environments. Prior experience at a deep tech or computational science company. ...

when it isn’t. The ability to communicate model behaviour and limitations clearly to non‐technical stakeholders. Familiarity with data pipeline tooling (e.g. Airflow, dbt, Spark) and an understanding of how ML fits within a broader data platform. There’s always room to grow and learn with our roles ...

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Preferred: Experience in one or more relevant ...

technical trade‐offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with or strong advocacy for AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. ...

seeking a data engineer to design, build, and scale data pipelines powering its next-generation digital content platform. You’ll work with DataBricks, Snowflake, Airflow, dbt (SQL), and Python within AWS to deliver robust data solutions. Join a collaborative team and contribute to platform optimization, governance, and scalable architectures ...

Strong hands‐on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast‐paced fintech or technology environment A collaborative leadership style with the ability ...

Experience Data Modeling Data Integration & Ingestion Data Manipulation and Processing GitHub, GitHub Actions, Azure DevOps Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest AI based Code developed and AI-built automated data pieplines Key Characteristics Technology expert who constantly pursues knowledge enhancement ...

Python Experience with AWS and modern data technologies Familiarity with version control systems such as Git Experience with orchestration tools such as Airflow Ability to troubleshoot and improve data platform performance Strong attention to detail and focus on data quality Excellent communication and collaboration skills Willingness to learn ...

Terraform, Bicep or Pulumi) Implementing monitoring and observability for production systems, including metrics, alerting, logging, and dashboarding (e.g. Prometheus, Grafana) Pipeline orchestration using Dagster, Airflow, Prefect, or similar Bonus points if you have: Practical experience with model serving infrastructure – batch and/or real‐time inference at scale. Experience ...

data modelling Databricks as our datalake platform Kubernetes for data services Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data ...

management. Data Center Optimization Collaborate with facilities and operations teams to optimize infrastructure deployments based on available rack space, power draw, thermal dynamics, and airflow efficiency. Contribute to the planning and provisioning of data center resources, ensuring support for both traditional workloads and AI‐intensive environments. Evaluate emerging technologies ...

operating large‐scale data transfer pipelines at multi‐TB or PB scale. Experience orchestrating parallelised transfer jobs: Flyte preferred; other workflow orchestration tools (e.g. Airflow, Prefect) will be considered. Experience building reliable, observable data systems , including retry strategies, backfills, data integrity checks, lifecycle/state tracking and operational alerting. ...

unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java Python or Scala* Experience in modern engineering practices and technologies* Good working knowledge of data ...

talks, or threads about building with AI. MLOps and model serving experience (BentoML, MLflow, Vertex AI, SageMaker). Streaming and batch ingestion pipelines (Spark, Airflow, Beam, Glue). Healthcare or life sciences domain exposure. AWS Professional certification or other relevant industry certifications. What We Offer At Newpage ...

bring mission‐critical software, analytics, and AI to life and is the company behind Astro, the industry‐leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data‐driven applications. Trusted by more than … building and scaling AE and Sales Engineering functions in a high‐growth environment. Preferred Qualifications Experience selling into the modern data stack ecosystem (Airflow, dbt, Snowflake, Databricks, Kafka, or similar). Familiarity with open‐source‐led or product‐led growth (PLG) sales motions — understanding how to convert community users ...

join them 5+ years’ experience in Data Engineering Experience designing scalable data pipelines for machine learning workloads Experience implementing orchestration tools such as Apache Airflow or equivalent. Knowledge of data modelling, dimensional modelling and data warehousing principles. Whether you’re developing new products, building scalable platforms or unlocking ...