51 to 75 of 251 Apache Airflow Jobs

data pipelines, ELT workflows, and contemporary cloud data architectures. Experience with current cloud data platforms such as Snowflake, Azure Data Lake, Azure SQL, dbt, Airflow, or similar tools. Ability to understand and work with SQL queries and data models. Knowledge of data governance, privacy, security, regulatory reporting, and responsible ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow).Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/AI.BenefitsWhat ...

communication skills and experience collaborating with Product, Engineering, and Data teams.Extra awesomeExperience working in a B2B SaaS product company.Experience with orchestration tools such as Airflow or Argo Workflows.Familiarity with SQL, Spark, Snowflake, or other data processing technologies.Experience building systems that combine structured data, unstructured data, and generative AI.Experience with ...

experience collaborating with Product, Engineering, and Data teams. Extra awesomeExperience working in a B2B SaaS product company. Experience with orchestration tools such as Airflow or Argo Workflows. Familiarity with SQL, Spark, Snowflake, or other data processing technologies. Experience building systems that combine structured data, unstructured data, and generative AI.Experience ...

relational databases (e.g., PostgreSQL, DuckDB). Experience with the modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity ...

with a passion for up-skilling junior analysts. Desirable ExperiencePublic sector or consultancy experience is highly desirable.Experience with automated data pipeline tools (e.g., dbt, Airflow).What makes us greatAlongside working with some of the most talented people out there and on-top of a competitive salary, which ...

with a passion for up-skilling junior analysts. Desirable ExperiencePublic sector or consultancy experience is highly desirable.Experience with automated data pipeline tools (e.g., dbt, Airflow).What makes us greatAlongside working with some of the most talented people out there and on-top of a competitive salary, which ...

coupled with rigorous validation of code, data transformations and analytical conclusions. Desirable Skills and Experience Experience with Azure Data Lake, Azure Data Factory, dbt, Airflow, MLflow, Power BI, Microsoft Purview or comparable tools. Experience with search indexes, vector databases, graph databases, knowledge graphs, ontologies or master-data-management techniques. ...

components such as S3, Kinesis, DMS, SNS, Glue, Athena, RDSExperience with implementing Rest APIs.Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practisesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Solid understanding of data modelling ...

data modeling and data integrationSkilled in data manipulation, processing and orchestration frameworksHands-on experience with Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube and PyTestProficiency with GitHub, GitHub Actions and Azure DevOpsStrong communication and collaboration skills to align technical and business stakeholdersProven ability ...

ADLS, and Azure Functions. Proven track record of designing and implementing metadata-driven data pipelines. Deep expertise in orchestration and data workflow automation e.g. Airflow, DBT. Strong understanding of CI/CD practices for data engineering. Experience with infrastructure as code (Terraform, Bicep, or ARM templates) (preferred). Solid ...

pipelines using Python and modern data frameworks.Integrate data from sensors, databases, cloud platforms, and third-party services.Orchestrate data workflows using tools such as Airflow and Prefect.Create IoT and Real-Time Data SolutionsCollect, normalize, and process telemetry from connected devices.Implement edge-to-cloud data flows using MQTT, HTTP, CoAP ...

including ADF, ADLS, and Azure Functions. Proven experience designing and implementing metadata-driven data pipelines. Strong understanding of orchestration and data workflow automation, e.g. Airflow, DBT, Spark Declarative Pipelines. Strong understanding and practical application of CI/CD practices for data engineering. Experience contributing to technical design discussions ...

using Python and modern data frameworks. Integrate data from sensors, databases, cloud platforms, and third-party services. Orchestrate data workflows using tools such as Airflow and Prefect. Create IoT and Real-Time Data SolutionsCollect, normalize, and process telemetry from connected devices. Implement edge-to-cloud data flows using MQTT ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

skills with the ability to work with non-technical stakeholdersNICE TO HAVEHands-on experience with Google Cloud Platform, especially BigQueryExperience with orchestration tools (e.g. Airflow)Knowledge of data governance and data quality frameworksExposure to reverse ETL tools (e.g. Hightouch)Familiarity with AI coding assistants and a habit of integrating ...

novelty. You don't need to know all of it on day one. Data and compute: S3, Athena, Spark on EMR, SQLOrchestration and MLOps: Airflow, MLflow, Docker, ECS, Terraform, GitHub Actions for CI/CDModels: foundation models served through Amazon BedrockAgents: LangGraph for orchestration, deployed and run on Amazon ...

experience with dbt , including models, tests, macros and documentation Strong production experience with BigQuery Experience using Python for data work and automation Experience with Airflow or Cloud Composer Strong understanding of dimensional data modelling Experience with Looker/LookML or a similar BI platform Good understanding ...

concepts (star/snowflake schemas, normalization). Experience with version control systems (e.g., Git) and CI/CD practices. Familiarity with orchestration tools (e.g., Airflow, dbt Cloud, Prefect) is a plus. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition ...

need both skillsets below - strength in one is enough: Hands-on: building AI/ML pipelines or prototypes; vector databases; prompt engineering; orchestration tools (Airflow/Prefect); Git Advisory: designing AI/data architectures; Agentic AI principles (LangGraph, CrewAI, AutoGen); mentoring specialists without formal authority; AI governance (security, privacy ...

pipelines Strong programming skills in Python and SQL or similar Experience with technologies such as Spark, Databricks, AWS, Azure, GCP and/or Airflow Strong focus on data quality, testing and reliability Solid software engineering fundamentals and high engineering standards Strong communication and collaborative skills Comfortable working with ambiguity ...

pipelines Managing AWS cloud data services (Glue, DMS, Athena preferred) Setting up and managing a cloud data warehouse (Redshift preferred) Hands-on working with Airflow or an equivalent orchestrator in production (Dagster, Prefect) Solid understanding of data modelling: dimensional modelling, slowly changing dimensions, and event-sourced schemas Other Desired ...