101 to 125 of 251 Apache Airflow Jobs

business environments is a plusWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureFamiliarity with Generative ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing.Discover more about our Analytics Community ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

platforms)Experience applying DevSecOps practices within cloud or controlled environmentsExposure to large language models or generative AI systemsExperience with data engineering tools (e.g. Spark, Airflow, Databricks)Experience applying AI and automation to operational or productivity-focused use casesExperience working in secure, restricted, or air-gapped environmentsExperience supporting systems subject ...

secureCost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignorePlatform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)AI Native: experienced in agentic coding workflows (context, harness, loop ...

preferred knowledge of AI infrastructure. Proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

knowledge of AI infrastructure. Good proficiency in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

degree in Computer Science, Mathematics, Engineering, Physics or a related discipline from a Russell Group university. Desirable Experience Spark and/or Databricks. Airflow or similar orchestration tools. Distributed computing and large-scale data processing. Defence or National Security experience. Essential Requirements UK national and eligible to obtain ...

experience in data platform architecture and cloud-based data environments Excellent understanding of modern data stack technologies Expertise with Google Cloud Platform, BigQuery, dbt, Airflow, or similar technologies Experience designing data integration patterns and scalable data solutions Knowledge of data governance, security, privacy, and compliance principles Strong understanding ...

technical leadership and solution design responsibility. You'll ideally have: Strong commercial experience with Python & SnowflakeExperience mentoring or technically leading other engineersExperience with dbt, Airflow or similar data engineering toolingA good understanding of data modelling and modern data platform architectureExperience working with AWS (ideally) or another major cloud platformThe ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

will take ownership of the data platform, optimising it for scalability to ensure successful client onboarding. You'll use modern tools (such as Airflow, Prefect, Dagster or AWS Step Functions) for ETL design and orchestration, work on transformation logic to clean, validate and enrich data (including handling missing values ...

pipelines. Proficiency in SQL, Python, AWS, and other modern data engineering technologies. Experience with version control tools (e.g., Git) and workflow orchestration platforms (e.g., Airflow). Proven ability to test, troubleshoot, and improve data platforms, ensuring high levels of reliability and data quality. Strong interpersonal and communication skills with ...

Master• Data Integration & Ingestion – Skill• Data Manipulation and Processing – Skill• GITHUB, Action, Azure DevOps – Skill• Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest – SkillOptional skills:• Experience in project management, running a scrum team.• Experience working with BPC, Planning.• Exposure to working with external ...

native architecturesPractical experience with Azure (Azure OpenAI, Azure AI Foundry, AI Search) and familiarity with AWS or GCPProficiency with containerized deployment, MLOps tooling (MLFlow, Airflow), and infrastructure-as-code practicesBachelor's or Master's degree in Computer Science, Data Science, or related fieldNice to haveExperience with Azure AI Agent ...

translating them into structured data flows.You have several years of strong experience with ETL/ELT processes and APIs with tools such as dbt, Airflow, and Talend.You have strong expertise in SQL & Python for relational database systems such as Redshift/S3 and SQL Server.You are comfortable working with ...

help refine problem statements and figure out how to solve themSome extra skills that are great (but not essential): Experience with MLOps tools: Airflow, MLflow, AWS SageMaker, AWS S3, AWS EMR, CI/CDPrior experience in the fraud domain and a strong understanding of fraud detection techniquesExperience designing ...

problems), advanced AI tools and decision-focused learning. You are comfortable writing production-quality Python code. Experience with MLOps tools, containerisation (Docker), and orchestration (Airflow/Dagster) is essential. Experience with Cloud platforms (GCP/Vertex AI preferred, or AWS/SageMaker) and handling large datasets via SQL/ ...

advanced analytics — ideally in a hybrid PLG/sales-led SaaS environmentStrong Python and SQL; experience with data modelling and pipeline tools (dbt, Airflow, Dagster)Deep expertise in predictive modelling, causal inference, survival analysis, and experimentation (A/B, diff-in-diff, instrumental variables)Experience working directly with product ...

advanced analytics — ideally in a hybrid PLG/sales-led SaaS environmentStrong Python and SQL; experience with data modelling and pipeline tools (dbt, Airflow, Dagster)Deep expertise in predictive modelling, causal inference, survival analysis, and experimentation (A/B, diff-in-diff, instrumental variables)Experience working directly with product ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems ...

/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.)PySparkAWS cloud infrastructure: EMR, ECS, Athena, etcMLOps/DevOps: Terraform, Docker, Airflow, MLFlowMonitoring and evaluation platforms including New Relic and BraintrustMore information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing.Discover more about our Analytics Community ...