76 to 100 of 224 Apache Airflow Jobs in London

.Databricks Mastery: Hands-on experience with production workloads on Databricks (Azure/AWS), PySpark, Spark SQL, Delta Lake, Unity Catalog, Python, SQL, orchestration tools (Airflow, Workflows, ADF), and streaming (Kafka/Event Hubs).Professional Skills: Strong consultative mindset, excellent communication, and ability to interface with traders and senior stakeholders. ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

tools. Experience building and optimising scalable data pipelines and data warehouses. Knowledge of AWS technologies such as Redshift, S3 and Athena. Experience with Airflow, Fivetran or similar orchestration and integration tools. Strong Linux/Unix and Bash scripting skills. A solid understanding of data quality, governance, lineage and monitoring. ...

technical issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline.PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams.Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones.What we can offerBonusHybrid working ...

systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

business environments is a plusWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureFamiliarity with Generative ...

data challengesWhile we advocate using the right tech for the right task, we often leverage the following technologies: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureExperience with Generative ...

robust engineering practices.Optimise performance, scalability, and reliability of quantitative applications.Skills required:Python data engineering stack: Pandas/Parquet/FastAPI/Jupyter/Airflow/Streamlit/RayExperience with high-performance data stores and query engines like Trino/SnowflakeExperience with financial concepts such as Equities/Options/ ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

pipelines, data lake patternsArchitecture: Event-driven systems, microservices, API-first designDevOps: CI/CD (GitHub Actions, Jenkins), Terraform, containerization (Docker/Kubernetes)Orchestration - : Airflow, Step Functions, BPM toolsObservability (desirable): GrafanaAI (desirable): AI/ML or GenAI adoption for automation and operational efficiencyExperience & SkillsProven experience as a Solutions Architect/ ...

Styled Components), React, Redux, Redux Hooks, and other modern state management libraries. Kotlin on Android and Swift on iOS. Data: SQL, Python, Snowflake, dbt, Airflow & Looker AI at the company: Sierra.ai for conversational AI, Python and LangChain for agentic AI solutions, Sagemaker and MLflow for Machine Learning, Cursor ...

experience with data pipelines and ML infrastructureExperience working within AWS (ideally using Sagemaker) and/or AzureComfort working across our data stack - inc Airflow, SnowflakeExperience with version control and CI/CD practices (ideally using GitHub)Rigorous attention to statistical validity - comfortable challenging assumptions and defending methodologyUnderstanding of best ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

data platform. You'll play a key role in building and optimising scalable data pipelines and analytics solutions, working across AWS, Snowflake, dbt and Airflow to deliver high-quality data products. The Company They are a well-established media organisation with a large and diverse audience, using data … models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline ...

isolation Experience with CI/CD, databases, infra-as-code Comfortable debugging production systems Nice to Have Security/compliance (SOC2, HIPAA) Exposure to AirFlow and MLFlow for scientific workflows MSC/PHD in Machine Learning Biotech/pharma experience Why Helical Live with top pharma High ownership, small ...

scraping pipelines to ensure data accuracy and reliability.Required Technical SkillsStrong proficiency in Python.Experience with monitoring and observability tools.Hands-on experience with workflow orchestration tools - Apache Airflow.Familiarity with data pipelines and ETL processes.Proficiency with containerization and orchestration technologies - Docker, KubernetesStrong understanding of CI/CD pipelines and toolsStrong problem-solving ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

database querying languages Experience in large‐scale data processing, microservices, API design, Kafka, Redis, MemCached, Observability tools (Dynatrace, Splunk, Grafana), and Orchestration tools (Airflow, Temporal) Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test acceleration ...

cluster management and ML training workloads (NVIDIA, CUDA, distributed training). Familiarity with MLOps tooling: Experiment tracking (MLflow, Weights & Biases). Workflow orchestration (Airflow, Prefect, Argo). Data versioning (DVC). Background in scientific computing or HPC environments. Prior experience at a deep tech or computational science company. ...

through to deployment. Strong knowledge of data science libraries such as Polars, Pandas, NumPy, Scikit-learn. Knowledge of MLOps tooling such as Weights & Biases, AirFlow, AWS SageMaker or Docker. Experience working with cloud platforms such as AWS, Azure or GCP. Experience working collaboratively in a multidisciplinary team. Our Hybrid ...

languages and database querying languagesExperience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, Observability tools (Dynatrace, Splunk, Grafana), and Orchestration tools (Airflow, Temporal)Proficiency in automation, continuous delivery methods, and all aspects of the Software Development Life CycleAdvanced understanding of agile methodologies, CI/CD, application ...

data modelling Databricks as our datalake platform Kubernetes for data services Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data ...

improvementRequirementsAdvanced SQL, ETL/ELT frameworks, and cloud data platforms such as Azure, AWS, or GCPData modelling and pipeline orchestration using tools such as Airflow, Data Factory, dbt, or similarIntegrate CDP data sources across digital, CRM, transactional, and behavioural data; support identity resolution, consent handling, audience activation, journey events ...