226 to 250 of 476 Apache Airflow Jobs in the UK

ensure data accuracy and reliability. Required Technical SkillsStrong proficiency in Python. Experience with monitoring and observability tools. Hands-on experience with workflow orchestration tools - Apache Airflow. Familiarity with data pipelines and ETL processes. Proficiency with containerization and orchestration technologies - Docker, KubernetesStrong understanding of CI/CD pipelines and toolsStrong ...

issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline. PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift #J-18808-Ljbffr ...

that make vendor data consistent and useful to downstream consumers.- Improve the reliability and observability of existing services and pipelines.- Help migrate and strengthen Airflow-based workflows.- Participate in the shared on-call rota for systems used by the trading teams.- Hold engineering standards across the team: review, testing ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

MLOps principles with experience deploying models in production Proficiency in Python and tools/platforms such as Azure, AWS, GCP, Databricks, MLFlow, and Airflow Strong communication skills with the ability to engage business and technical stakeholders We offer EPAM Employee Stock Purchase Plan (ESPP) Protection benefits including life assurance ...

data ecosystem, particularly BigQuery. Advanced SQL skills and strong experience with Python. Strong experience with modern data engineering and orchestration technologies such as dbt, Airflow/Cloud Composer, and related tools. Experience designing scalable data platforms, pipelines, data models, and integration patterns. Strong understanding of data governance, security, quality ...

pipeline development, not just scripting Experience with distributed data frameworks like Spark to write and debug data pipelines Experience with workflow orchestration tools (e.g. Airflow, Flyte, or equivalent) Proven ability to design, implement, and maintain production-grade data pipelines and dashboards Good understanding of development processes and best practices ...

data modelling Databricks as our datalake platform Kubernetes for data services Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

interpersonal skills Preferred qualifications: Experience in the gaming or entertainment industry Experience working with tools and technologies such as Spark, Snowflake, Qwak, MLFlow, Airflow, etc Contributions to open‐source AI/ML projects or participation in AI/ML competitions (e.g., Kaggle) Experience with training and fine‐tuning large ...

interpersonal skills Preferred qualifications: Experience in the gaming or entertainment industry Experience working with tools and technologies such as Spark, Snowflake, Qwak, MLFlow, Airflow, etc Contributions to open‐source AI/ML projects or participation in AI/ML competitions (e.g., Kaggle) Experience with training and fine‐tuning large ...

MLflow) Experience with cloud‐based ML tools (e.g., SageMaker), data and model versioning (e.g., DVC), CI/CD (e.g., GitHub Actions), workflow orchestration (e.g., Airflow/Dagster) and containerised solutions (e.g., Docker, ECS) nice to have Experience in code testing (unit, integration, end‐to‐end tests) Strong data engineering ...

business environments is a plusWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureFamiliarity with Generative ...

languages and database querying languagesExperience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, Observability tools (Dynatrace, Splunk, Grafana), and Orchestration tools (Airflow, Temporal)Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test ...

data challengesWhile we advocate using the right tech for the right task, we often leverage the following technologies: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureExperience with Generative ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift Why else ...

stakeholders to the right data solutions Familiarity with using AI tools to accelerate your workflow Nice to Have Experience with orchestration tools like Dagster, Airflow, or similar Experience building real-time or near-real-time data pipelines Familiarity with machine learning workflows and feature engineering Experience embedded in teams ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing. Discover more about our Analytics Community ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systemsWhat ...

Cloud Run), AWS (Bedrock, SageMaker) Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar. Ability to design ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Preferred: Experience in one or more relevant ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Benefits and Perks We value the total ...

well as present complex findings clearly to both technical and non-technical audiencesProficient in using Tech Stack such as Amazon S3, Lambda function, Kafka, Apache Airflow or Bloomberg Teck Stack in production environmentExceptional problem-solving skills, numerical proficiency and high attention to detailAbility to work independently as well ...