101 to 125 of 928 Permanent Apache Airflow Jobs

engineering: building and maintaining data pipelines, ETL/ELT, SQL, cloud data platforms (for example Azure, AWS or GCP), tools such as dbt, Databricks, Airflow, Snowflake or BigQuery. Machine learning: designing, training, deploying and monitoring models in production, Python, and the ML lifecycle including MLOps. Applied AI: modern ...

Azure) and infrastructure-as-code tools (e.g., Terraform) Familiarity with data processing frameworks (e.g., Spark, Pandas) and workflow orchestration tools (e.g., Airflow) Strong understanding of API design, microservices architecture, and system integration Proficiency with version control (Git) and CI/CD practices Project & Cross-Functional Skills Demonstrated experience leading ...

maintaining scalable data platforms. Strong SQL and data modelling skills, with experience designing analytical datasets. Experience with modern data stack tools (e.g. dbt, Spark, Airflow, or similar orchestration and transformation frameworks). Experience with cloud‐based data platforms, ideally AWS. Understanding of medallion architecture or similar data layering approaches. ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

structure prediction, surrogate modeling, Bayesian optimization). Hands-on work with multi-agent systems, knowledge graphs (Neo4j), or agent frameworks/SDKs. Familiarity with Airflow or Prefect, PyTorch, and related ML/LLM tooling. Direct experience with commercial LIMS/ELN or analytical platforms such as Genedata, CDD Vault ...

structure prediction, surrogate modeling, Bayesian optimization). Hands-on work with multi-agent systems, knowledge graphs (Neo4j), or agent frameworks/SDKs. Familiarity with Airflow or Prefect, PyTorch, and related ML/LLM tooling. Direct experience with commercial LIMS/ELN or analytical platforms such as Genedata, CDD Vault ...

consulting experience within BFSI, leading complex enterprise transformations. Platform Expertise: Deep technical proficiency in modern data platforms (Databricks, Snowflake) and toolsets (dbt, Airflow, Kafka). Advanced Architecture & AI: Mastery of Lakehouse, Data Mesh, Streaming, MLOps, and GenAI integration to drive continuous innovation. Governance & Compliance: Expertise in zero-trust security ...

just how fast. We expect you to enjoy leveraging AI and agentic systems to operate more effectively. Tech environmentScala | Spark | Databricks | AWS | Claude | Airflow | Kubernetes | Terraform | Functional ProgrammingYou will thrive here if youEnjoy end-to-end ownership, from writing the code to architecture design to coaching others through delivery. ...

experienced with modern cloud data platforms such as BigQuery, Snowflake, or Databricks. You've worked with data orchestration tools such as Dagster, Airflow, or Prefect. We use Dagster. You're comfortable owning cloud infrastructure. You can work with infrastructure-as-code such as Terraform and own the infrastructure your ...

real-time and batch data processing pipelines using modern frameworks like Spark, Flink, Spark Streaming, Kafka, Debezium, etc. Experience with orchestration tools such as Airflow, dbt, or similar. Exceptional problem-solving skills paired with a proactive, innovative mindset focused on continuous improvement. Strong communication and cross-functional collaboration skills ...

hands-on familiarity with vessel-tracking data, its structure, quirks, and limitations. SQL— strong proficiency, including query optimization on large datasets. Nice-to-Haves Apache Airflow (or equivalent orchestration) for scheduling and managing pipelines. AWS experience (Azure or GCP equally welcome). Machine Learning experience, particularly applied ...

infrastructure. Proficiency in Python and experience with ML frameworks (e.g., TensorFlow, PyTorch). Strong experience with data engineering tools (e.g., Pandas, SQL, Apache Airflow, Spark). Familiarity with cloud platforms (AWS, GCP, or Azure) and containerization (Docker, Kubernetes). Experience working with massive amounts of real-world training ...

Spark & distributed computing dbt Terraform Data Warehousing (we use Databricks but experience with other platforms is fine) MLOps implementation (desirable but not required) Airflow (or other orchestration tooling) #J-18808-Ljbffr ...

automation or cybersecurity tools Knowledge of AI/ML pipelines and data preparation Experience with Databricks and Databricks AI Experience with workflow orchestration tools (Airflow, Prefect) Background in cybersecurity or threat intelligence automation Experience with containerization and orchestration (Docker, Kubernetes) Knowledge of streaming data platforms (Kafka, Kinesis) Familiarity with ...

day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processing Extensive experience in dbt and ideally Airflow (or other orchestration tools) Extensive experience in Entity-Relationship modelling and Dimensional modelling Experience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

/or Genetics datasets Proficiency in cloud computing, data/scripting programming languages and workflow orchestration technologies (e.g. Python, R, Bash, SQL, Nextflow, Airflow, Databricks , etc.) Demonstrated expertise in ETL processes, data pipelines, metadata harmonisation, data lifecycle management, data product release processes and operational support Experience operationalising analytical workflows ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture design to coaching ...

translate requirements into scalable data solutions Experience mentoring, coaching or supporting the development of other analytics engineers Knowledge of orchestration tools such as Airflow or Composer Familiarity with cloud platforms, particularly Google Cloud Platform (GCP) And it would be great if you had any of the following... Experience with ...

Application Resiliency, and Security. Preferred qualifications, capabilities, and skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Strong working knowledge of both supervised and unsupervised techniques, including tree-based and kernel methods (gradient boosting, random forest, SVM) and anomaly-detection ...

/or Genetics datasets Proficiency in cloud computing, data/scripting programming languages and workflow orchestration technologies (e.g. Python, R, Bash, SQL, Nextflow, Airflow, Databricks, etc.) Demonstrated expertise in ETL processes, data pipelines, metadata harmonisation, data lifecycle management, data product release processes and operational support Experience operationalising analytical workflows ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricks* Ability to think strategically and relate architectural decisions ...