101 to 125 of 947 Apache Airflow Jobs

implementing enterprise-scale data solutions. The ideal candidate will possess strong expertise in Snowflake and hands‐on experience with Python, PySpark, AWS, and Airflow for building scalable, high-performance data platforms. The role demands strong technical leadership, problem‐solving capabilities, and the ability to collaborate effectively with cross‐functional ...

components such as S3, Kinesis, DMS, SNS, Glue, Athena, RDSExperience with implementing Rest APIs. Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practisesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

like Snowflake/Redshift/BigQueryExperience with Python/PySparkExperience with implementing Rest APIs. Business Intelligence (BI) knowledge/experienceKnowledge on tools like Terraform, Airflow, DBT and HarnessModern data engineering practisesThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Solid understanding of data modelling ...

automation or cybersecurity tools Knowledge of AI/ML pipelines and data preparation Experience with Databricks and Databricks AI Experience with workflow orchestration tools (Airflow, Prefect) Background in cybersecurity or threat intelligence automation Experience with containerization and orchestration (Docker, Kubernetes) Knowledge of streaming data platforms (Kafka, Kinesis) Familiarity with ...

engineering: building and maintaining data pipelines, ETL/ELT, SQL, cloud data platforms (for example Azure, AWS or GCP), tools such as dbt, Databricks, Airflow, Snowflake or BigQuery. Machine learning: designing, training, deploying and monitoring models in production, Python, and the ML lifecycle including MLOps. Applied AI: modern ...

Azure) and infrastructure-as-code tools (e.g., Terraform) Familiarity with data processing frameworks (e.g., Spark, Pandas) and workflow orchestration tools (e.g., Airflow) Strong understanding of API design, microservices architecture, and system integration Proficiency with version control (Git) and CI/CD practices Project & Cross-Functional Skills Demonstrated experience leading ...

maintaining scalable data platforms. Strong SQL and data modelling skills, with experience designing analytical datasets. Experience with modern data stack tools (e.g. dbt, Spark, Airflow, or similar orchestration and transformation frameworks). Experience with cloud‐based data platforms, ideally AWS. Understanding of medallion architecture or similar data layering approaches. ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

structure prediction, surrogate modeling, Bayesian optimization). Hands-on work with multi-agent systems, knowledge graphs (Neo4j), or agent frameworks/SDKs. Familiarity with Airflow or Prefect, PyTorch, and related ML/LLM tooling. Direct experience with commercial LIMS/ELN or analytical platforms such as Genedata, CDD Vault ...

structure prediction, surrogate modeling, Bayesian optimization). Hands-on work with multi-agent systems, knowledge graphs (Neo4j), or agent frameworks/SDKs. Familiarity with Airflow or Prefect, PyTorch, and related ML/LLM tooling. Direct experience with commercial LIMS/ELN or analytical platforms such as Genedata, CDD Vault ...

consulting experience within BFSI, leading complex enterprise transformations. Platform Expertise: Deep technical proficiency in modern data platforms (Databricks, Snowflake) and toolsets (dbt, Airflow, Kafka). Advanced Architecture & AI: Mastery of Lakehouse, Data Mesh, Streaming, MLOps, and GenAI integration to drive continuous innovation. Governance & Compliance: Expertise in zero-trust security ...

just how fast. We expect you to enjoy leveraging AI and agentic systems to operate more effectively. Tech environmentScala | Spark | Databricks | AWS | Claude | Airflow | Kubernetes | Terraform | Functional ProgrammingYou will thrive here if youEnjoy end-to-end ownership, from writing the code to architecture design to coaching others through delivery. ...

experienced with modern cloud data platforms such as BigQuery, Snowflake, or Databricks. You've worked with data orchestration tools such as Dagster, Airflow, or Prefect. We use Dagster. You're comfortable owning cloud infrastructure. You can work with infrastructure-as-code such as Terraform and own the infrastructure your ...

real-time and batch data processing pipelines using modern frameworks like Spark, Flink, Spark Streaming, Kafka, Debezium, etc. Experience with orchestration tools such as Airflow, dbt, or similar. Exceptional problem-solving skills paired with a proactive, innovative mindset focused on continuous improvement. Strong communication and cross-functional collaboration skills ...

Google Cloud. Key skills: Strong Python & SQL experience Hands-on Azure Databricks, Delta Lake & Unity Catalog Strong GCP/BigQuery/GCS experience Apache Airflow/Cloud Composer for pipeline orchestration Experience building ELT/data pipelines and cross-cloud data transfers Understanding of IAM, RBAC, secrets management ...

with CI/CD and automated deployment practices. Experience supporting and maintaining production data platforms. Desirable Skills Scikit-Learn or Spark MLlib AWS EMR Apache Airflow Jenkins Hudi or other open table formats Kibana Grafana Understanding of Data Science principles If you receive suspicious outreach claiming ...

hands-on familiarity with vessel-tracking data, its structure, quirks, and limitations. SQL— strong proficiency, including query optimization on large datasets. Nice-to-Haves Apache Airflow (or equivalent orchestration) for scheduling and managing pipelines. AWS experience (Azure or GCP equally welcome). Machine Learning experience, particularly applied ...

infrastructure. Proficiency in Python and experience with ML frameworks (e.g., TensorFlow, PyTorch). Strong experience with data engineering tools (e.g., Pandas, SQL, Apache Airflow, Spark). Familiarity with cloud platforms (AWS, GCP, or Azure) and containerization (Docker, Kubernetes). Experience working with massive amounts of real-world training ...

Spark & distributed computing dbt Terraform Data Warehousing (we use Databricks but experience with other platforms is fine) MLOps implementation (desirable but not required) Airflow (or other orchestration tooling) #J-18808-Ljbffr ...

Optimise Python performance through effective use of loops , vectorization , threading , and understanding of the Global Interpreter Lock (GIL) . Desirable Experience Experience with Kubeflow , Airflow , or dbt . Hands-on experience with Google Cloud Platform (GCP) . Knowledge of containerisation (Docker, Kubernetes) and CI/CD practices. Familiarity with ...

automation or cybersecurity tools Knowledge of AI/ML pipelines and data preparation Experience with Databricks and Databricks AI Experience with workflow orchestration tools (Airflow, Prefect) Background in cybersecurity or threat intelligence automation Experience with containerization and orchestration (Docker, Kubernetes) Knowledge of streaming data platforms (Kafka, Kinesis) Familiarity with ...

day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processing Extensive experience in dbt and ideally Airflow (or other orchestration tools) Extensive experience in Entity-Relationship modelling and Dimensional modelling Experience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...