226 to 250 of 250 Apache Airflow Jobs

integration failures, data inconsistencies, and production issues Key Skills • Python & SQL (production-grade pipeline development) • REST APIs, webhooks, OAuth, event-driven architecture • Orchestration tools: Airflow, Prefect, or Dagster • Cloud platforms: AWS, GCP, or Azure • Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills • Vector databases and embedding pipelines • Real-time … robust, production-grade data pipelines from scratch. • Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. • Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). • Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

sequenced engineering plan, and keep the team aligned to it. Own the technical quality of the platform: Snowflake infrastructure and RBAC, ingestion patterns, orchestration (Airflow), CI/CD, observability and cost controls. Make and record technical decisions. Run the decision-document process. Make the final call when the team … enterprise-grade dbt model development. Experience designing end-to-end data pipelines. Combining batch, file-based, and streaming ingestion (Kafka/CDC) managed with Airflow orchestration. A good understanding of data security and governance, including PII handling and access control. Strong infrastructure-as-code skills and CI/ ...

with cloud platforms (AWS, Azure, GCP)Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow)Practical experience with NLP and/or Search techniquesPrompt engineering, and LLMs for enterprise-scale applications. You have team lead experienceStrong ...

with cloud platforms (AWS, Azure, GCP) Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

with cloud platforms (AWS, Azure, GCP) Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

Why Sony Interactive Entertainment? Sony Interactive Entertainment isn't just the Best Place to Play — it's also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking … experience with TerraformExperience in creating workflows for Apache Airflow#LI-GL1What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned ...

engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Managed Airflow (Composer) to orchestrate ETL pipelines and enables reliable movement of data from source systems through the platform into downstream vendor systems. You'll guide … contribution where needed (critical ETLs, framework components, design spikes, performance fixes).ETL orchestration & data movement Design and develop robust ETL pipelines orchestrated in Airflow, including scheduling strategies, dependency management, retries, backfills, and processing. Establish patterns for onboarding new source systems and delivering data to downstream vendor systems with strong ...

deliver new capabilities Build and maintain robust MLOps pipelines to support scalable, reproducible, and automated model development, deployment, and monitoring Leverage tools such as Airflow for workflow orchestration and MLflow for experiment tracking, model registry, and lifecycle management, ensuring strong CI/CD practices and model governance Essential Skills … development of machine learning models and/or deep learning to solve complex science and engineering problems Experience with MLOps tools and practices, including Airflow, MLflow, and containerization (e.g., Docker) A passion for gaining insight into real-world datasets and clearly communicating through data visualization techniques Interest in material ...

deliver new capabilities Build and maintain robust MLOps pipelines to support scalable, reproducible, and automated model development, deployment, and monitoring Leverage tools such as Airflow for workflow orchestration and MLflow for experiment tracking, model registry, and lifecycle management, ensuring strong CI/CD practices and model governance Essential Skills … development of machine learning models and/or deep learning to solve complex science and engineering problems Experience with MLOps tools and practices, including Airflow, MLflow, and containerization (e.g., Docker) A passion for gaining insight into real-world datasets and clearly communicating through data visualization techniques Interest in material ...

Why Sony Interactive Entertainment Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As ...

role). Cloud Infrastructure & Data Pipelines: Build with AWS and Azure services to construct ETL workflows and dynamic systems using orchestration platforms like Apache Airflow. Production Operations: Support production applications, maintain SLAs, and create continuous evaluation metrics for ML solution performance. Qualifications & Skills 2-5 years' hands-on software … Hands-on experience building GenAI applications, LLM workflows, and data pipelines. Cloud experience across AWS or Azure ecosystems along with data orchestration tools (e.g., Airflow). Practical understanding of AI safety, prompt injection handling, and PII/MNPI compliance guardrails. If you have 2-5 years' experience ...

programme owners to resolve gaps and improve data qualityBuild and Optimise Pipelines: Develop, optimise, and maintain data pipelines using Python, REST APIs, DBT, and Airflow to ensure seamless data flow for compliance and risk management. People Management: Manage and support one or two analysts working across the security … with data visualization tools such as Looker, Superset, PowerBI, or TableauExperience in automation and building data pipelines and working with tools like DBT and Airflow for ETL processes. Skilled in using REST APIs to develop and manage data integrationsBonus: Familiarity with security frameworks (e.g., SOC 2, ISO 27001, NIST ...

streaming pipelines and deliver scalable Databricks solutions using Delta Live Tables, Workflows, Unity Catalog, Python, PySpark, Terraform and Databricks Asset Bundles. Key requirements: Advanced Apache Spark and Spark SQL optimisation Production Spark Structured Streaming experience Deep Databricks platform knowledge Strong Python and PySpark engineering capability Delta Live Tables, Workflows … Databricks Asset Bundles CI/CD and GitHub Actions Ability to work autonomously in a fast-moving environment Experience with Databricks Apps, Lakebase, Airflow, Azure Data Factory, monitoring platforms or financial services would be beneficial. Location: London Rate: £800£900 per day IR35: Inside IR35 ...

SDLC) Deep expertise in object oriented programming using Java Proven hands on experience building cloud based solutions on AWS, including services like EKS, S3, Airflow, IAM, and Lambda Experience with cloud database platforms such as Snowflake, Postgres, Athena, and SQL Server Experience with industry standard AI frameworks/tools … Agile development and CI/CD pipelines, with exposure to testing/automation tools (JUnit, Selenium, JMeter) and orchestration/scheduling services (UAC, Airflow, Autosys); experience with PowerShell, Python, and Kubectl is a plus Bachelor's degree in Computer Science , MIS, Computer Engineering, or related field (or equivalent experience ...

scalable APIs (FastAPI, etc.) for model deployment, implement MLOps pipelines, leverage cloud platforms (AWS/Azure) to optimize AI infrastructure, and use orchestration tools (Airflow) for complex ML workflows Champion ML governance and operational excellence – Ensure adherence to AI/ML governance guidelines, monitor model performance and SLAs, optimize … excellence – Strong understanding of MLOps, containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure including Bedrock, SageMaker, Azure AI Search), workflow orchestration (Airflow), and API development for ML systems Software development fundamentals – Deep understanding of automated testing, source version control, code optimization, software architecture, and building scalable ...

store of value, and/or FICC options tradingExperience with Linux-based, concurrent, high-throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree ...

migrate data sets into their AWS Snowflake platform. Designing and implementing dbt models that power clean, consistent metrics. Buidl out orchestartion pipelines using Airflow Creating clear documentation for dashboards, metrics, and models. Ensuring optimal performance and cost efficiency across Snowflake Supporting a stakeholder-first approach to data usability … access. KEY SKILLS AND REQUIREMENTS Extensive experience operating within Snowflake environments Airflow for data orchestration. Proficiency in SQL and dbt, with a strong understanding of data modeling best practices. Strong communication skills with the ability to translate complex needs into elegant data solutions. Focus on usability, documentation, and data ...

Company DescriptionWise is a global technology company, building the best way to move and manage the world's money. Min fees. Max ease. Full speed. Whether people and businesses are sending money to another country ...

Data & Insight Engineer Salary: £55,000-£60,000 + Share Options + Biannual Bonus Location: Central London (3 days in office) About the Company/Role Fast-growing food-tech scale-up founded by leaders ...

Job ID: 8603Location: London, England, GBType: PermanentSalary: USD excellent salary, bonus, full bens per yearPosted: 2023-11-24Data Engineer to collaborate with data scientists to maintain strict control on data cleaning workflows to power ...

Hiring: Big Data PySpark Developer Location: Irving, TX (Hybrid 3 Days Onsite) Experience: 7+ Years Job Type: Full-Time Required Skills: PySpark, Python, Apache Spark, SparkSQL, Hadoop, SQL, Unix/Linux, Shell Scripting, and Apache Airflow. Interested candidates, please DM me or share your updated resume. VITS Consulting ...

product design solution.Design mechanical components in CREO Parametric 3D - Pro/E CAD software.Conduct or direct CFD and FEA analyses to optimize Semiconductor Equipment airflow requirements, and mechanical design.Reduce 3D mechanical designs to 2D drawing and proper application of mechanical tolerancing/GD&T to achieve desired … machining methods.Experience with 2D drafting and tolerance analysis.Experience in several of these areas is desirable: CREO Parametric 3D MCAD - Pro/E software, CFD airflow analysis; FEA analysis; Design for manufacture (DFM); Design of welded sheet metal structures and frames; Measurement instrumentation, testing and data acquisition; Programming in Python ...

solution. Design mechanical components in CREO Parametric 3D - Pro/E CAD software. Conduct or direct CFD and FEA analyses to optimize Semiconductor Equipment airflow requirements, and mechanical design. Reduce 3D mechanical designs to 2D drawing and proper application of mechanical tolerancing/GD&T to achieve desired … Experience with 2D drafting and tolerance analysis. Experience in several of these areas is desirable: CREO Parametric 3D MCAD - Pro/E software, CFD airflow analysis; FEA analysis; Design for manufacture (DFM); Design of welded sheet metal structures and frames; Measurement instrumentation, testing and data acquisition; Programming in Python ...