51 to 75 of 240 Permanent Apache Airflow Jobs

with a passion for up-skilling junior analysts. Desirable ExperiencePublic sector or consultancy experience is highly desirable.Experience with automated data pipeline tools (e.g., dbt, Airflow).What makes us greatAlongside working with some of the most talented people out there and on-top of a competitive salary, which ...

coupled with rigorous validation of code, data transformations and analytical conclusions. Desirable Skills and Experience Experience with Azure Data Lake, Azure Data Factory, dbt, Airflow, MLflow, Power BI, Microsoft Purview or comparable tools. Experience with search indexes, vector databases, graph databases, knowledge graphs, ontologies or master-data-management techniques. ...

components such as S3, Kinesis, DMS, SNS, Glue, Athena, RDSExperience with implementing Rest APIs.Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practisesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Solid understanding of data modelling ...

data modeling and data integrationSkilled in data manipulation, processing and orchestration frameworksHands-on experience with Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube and PyTestProficiency with GitHub, GitHub Actions and Azure DevOpsStrong communication and collaboration skills to align technical and business stakeholdersProven ability ...

ADLS, and Azure Functions. Proven track record of designing and implementing metadata-driven data pipelines. Deep expertise in orchestration and data workflow automation e.g. Airflow, DBT. Strong understanding of CI/CD practices for data engineering. Experience with infrastructure as code (Terraform, Bicep, or ARM templates) (preferred). Solid ...

pipelines using Python and modern data frameworks.Integrate data from sensors, databases, cloud platforms, and third-party services.Orchestrate data workflows using tools such as Airflow and Prefect.Create IoT and Real-Time Data SolutionsCollect, normalize, and process telemetry from connected devices.Implement edge-to-cloud data flows using MQTT, HTTP, CoAP ...

including ADF, ADLS, and Azure Functions. Proven experience designing and implementing metadata-driven data pipelines. Strong understanding of orchestration and data workflow automation, e.g. Airflow, DBT, Spark Declarative Pipelines. Strong understanding and practical application of CI/CD practices for data engineering. Experience contributing to technical design discussions ...

using Python and modern data frameworks. Integrate data from sensors, databases, cloud platforms, and third-party services. Orchestrate data workflows using tools such as Airflow and Prefect. Create IoT and Real-Time Data SolutionsCollect, normalize, and process telemetry from connected devices. Implement edge-to-cloud data flows using MQTT ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

skills with the ability to work with non-technical stakeholdersNICE TO HAVEHands-on experience with Google Cloud Platform, especially BigQueryExperience with orchestration tools (e.g. Airflow)Knowledge of data governance and data quality frameworksExposure to reverse ETL tools (e.g. Hightouch)Familiarity with AI coding assistants and a habit of integrating ...

novelty. You don't need to know all of it on day one. Data and compute: S3, Athena, Spark on EMR, SQLOrchestration and MLOps: Airflow, MLflow, Docker, ECS, Terraform, GitHub Actions for CI/CDModels: foundation models served through Amazon BedrockAgents: LangGraph for orchestration, deployed and run on Amazon ...

experience with dbt , including models, tests, macros and documentation Strong production experience with BigQuery Experience using Python for data work and automation Experience with Airflow or Cloud Composer Strong understanding of dimensional data modelling Experience with Looker/LookML or a similar BI platform Good understanding ...

concepts (star/snowflake schemas, normalization). Experience with version control systems (e.g., Git) and CI/CD practices. Familiarity with orchestration tools (e.g., Airflow, dbt Cloud, Prefect) is a plus. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition ...

need both skillsets below - strength in one is enough: Hands-on: building AI/ML pipelines or prototypes; vector databases; prompt engineering; orchestration tools (Airflow/Prefect); Git Advisory: designing AI/data architectures; Agentic AI principles (LangGraph, CrewAI, AutoGen); mentoring specialists without formal authority; AI governance (security, privacy ...

pipelines Strong programming skills in Python and SQL or similar Experience with technologies such as Spark, Databricks, AWS, Azure, GCP and/or Airflow Strong focus on data quality, testing and reliability Solid software engineering fundamentals and high engineering standards Strong communication and collaborative skills Comfortable working with ambiguity ...

pipelines Managing AWS cloud data services (Glue, DMS, Athena preferred) Setting up and managing a cloud data warehouse (Redshift preferred) Hands-on working with Airflow or an equivalent orchestrator in production (Dagster, Prefect) Solid understanding of data modelling: dimensional modelling, slowly changing dimensions, and event-sourced schemas Other Desired ...

Data Vault, dimensional modelling or complex entity resolution. Building data foundations for machine learning, anomaly detection or GenAI applications. Experience with Azure Data Factory, Airflow or similar orchestration technologies. Why consider it? This is an opportunity to take genuine technical ownership of an important data engineering capability, rather than ...

every day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship modelling and Dimensional modellingExperience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery) and Cloud ...

hands-on experience in at least one data area and one platform area below. Data experience Orchestration: operating pipelines with tools such as Airflow, Dagster or Prefect, including dependencies, retries, backfills, reruns or schema changes. Storage and modelling: designing schemas and storage for relational databases, warehouses or lakehouses ...

with Pandas or NumPy, scikit-learn, and model evaluation or tuningExperience building ETL pipelines and querying APIs using tools such as Airflow or dbt, and Git-based version controlExperiencedeveloping predictive models and quantitative analyses that support intelligence assessments and operational decision-makingExperience applying NLP or text mining to reporting ...

with text, audio, or video in AI applications.Experience evolving shared AI infrastructure or platforms used by multiple product teams.Experience with orchestration tools such as Airflow or Argo Workflows.Familiarity with SQL, Spark, Snowflake, or other data processing technologies.Experience with AI security, privacy, responsible AI, prompt injection protection, or data leakage ...

audio, or video in AI applications. Experience evolving shared AI infrastructure or platforms used by multiple product teams. Experience with orchestration tools such as Airflow or Argo Workflows. Familiarity with SQL, Spark, Snowflake, or other data processing technologies. Experience with AI security, privacy, responsible AI, prompt injection protection ...