376 to 400 of 468 Apache Airflow Jobs in England

harnessing, mounting and internal assembly, defining the interface requirements and constraints that keep electrical and mechanical designs coherent. Validate mechanical designs for electrical fit, airflow, thermal performance, manufacturability and secure-by-design alignment, and represent electrical, integration and security requirements in mechanical design reviews. Lead EMC design, EMC testing ...

Learning, or AI workloads. Experience with MLOps tooling such as MLflow, Kubeflow, Vertex AI, or similar technologies. Knowledge of workflow orchestration platforms such as Apache Airflow. Experience with data processing technologies such as Pandas, Apache Beam, Spark, or similar frameworks. Strong SQL skills and experience with analytical databases ...

Databricks REST APIs, Prometheus, or Azure Monitor. Maintain best practices for data engineering performance, testing, and documentation. Technical Skills Required Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake. Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies. Experience tuning job configurations (e.g. … memory management, parallelism, caching, adaptive query execution). Familiarity with orchestration tools: Databricks Workflows, Airflow, or Azure Data Factory. Proficiency in Python or Scala for data engineering and pipeline development. Hands-on experience with Azure, AWS, or multi-cloud Databricks deployments. Knowledge of data storage layers (Azure Data Lake ...

cost efficiency, scalability, and reliabilityContribute to the target architecture and long-term platform directionData Engineering & DeliveryDesign and build robust, scalable data pipelines (Snowflake, dbt, Airflow, ADF, Python)Apply strong practices in test-driven development, monitoring, logging, and observabilityOwn more complex technical solutions and problem-solvingSupport production issues and drive … Python, with hands-on experience building data pipelines and transformations at scaleHands-on experience with a modern cloud data stack, including Snowflake, dbt, Airflow, and Databricks, on a cloud platform (Azure preferred)Demonstrated experience delivering data lineage and traceability within a regulated environment (e.g. SOX, financial services, healthcare ...

develop. On the data pipeline side, we value awareness across both established tools such as SSIS/SSRS and newer approaches such as dbt, Airflow, or AWS Glue — dbt experience is essential, with the others a strong plus. Previous experience in working in cloud environments (AWS preferred … definite plus. Familiarity with legacy SSIS/SSRS, useful for maintaining existing packages, as well as newer orchestration/ingestion tools such as Airflow or AWS Glue. Exposure to cloud data warehouse/data fabric platforms such as Snowflake or Databricks. A good understanding of financial markets. Background ...

provenance. Hands‐on experience with data lineage/metadata tooling in enterprise environments (e.g., Collibra, Alation, Informatica EDC/IDMC, IBM Infosphere, Microsoft Purview, Apache Atlas, Amundsen, DataHub or similar). Proven ability to build lineage for complex platforms: data lakes, warehouses, marts, and distributed processing (Spark‐based pipelines … production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys & Apache Airflow. Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift). Familiarity with AWS S3 for large-scale data storage. ...

critical dashboards, enable self-service analytics, and fuel advanced machine learning models and real-time data products. Working with tools like DBT, Spark, and Airflow, you'll transform high volume raw event data into user-friendly, high impact datasets that support machine learning use cases across the business. … Comfort with data modelling and building efficient data marts and warehouses in the cloud. Experience building data pipelines using tools such as Spark and Airflow, or similar technologies, within a cloud environment such as AWS.Familiarity with both real-time and batch data workloads, along with modern data transformation ...

lifecycle. What you’ll be doing Build and maintain ML pipelines for training, evaluation, and deployment using tools like Databricks, MLFlow, Airflow, DBT, Sagemaker, Tecton Support AI scientist creating reproducible, containerized model training environments (on-demand and scheduled), and manage compute at scale (e.g., spot/GPU autoscaling) Define … looking for Proven experience designing and deploying ML systems in production (5+ years in relevant roles) Proficiency in Python and SQL, and orchestration tools (Airflow, Kubeflow, Dagster, etc.) Experience with modern cloud platforms (preferably GCP or AWS), Kubernetes, and CI/CD workflows Understanding of ML model lifecycles: training ...

product and the company. Your Contribution Will Be The stack is modern: StarRocks as our primary analytical layer, Flink for processing, DBT for transformation, Airflow for orchestration and various tooling for surfacing insights. The hard work of making it genuinely reliable is still in front of us — and that …/lakehouse architecture patterns. Verbal and written fluency in English is required — we're a distributed team and written communication is important. Preferred Qualifications Airflow for orchestration. Experience with Apache Hudi. Experience with financial, transactional, or payment data. What We Offer at Yuno Competitive Compensation. Remote Work ...

bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than … Salesforce) and sales enablement platforms. Bonus Points If You Have: Experience selling to data teams, developers, or technical buyers. Background in data orchestration or Airflow-related technologies. Prior success in a startup or high-growth environment. The estimated OTE is up to £225,000 uncapped, along with a generous ...

operate pipelines feeding daily reporting, attribution, and LTV, ingesting data from ads, payments, and product backends into our lakehouse. You will orchestrate jobs with Apache Airflow, maintain silver/gold tables, and work with Analytics, Marketing, and Data Science to impact products and processes. This 3-month contract ...

Sign in to set job alerts for "Senior Data Engineer" roles. Senior Data Engineer - AWS Data Platform (Redshift, Glue, and Airflow) Senior Data Engineer - Healthcare Data & Audience Applications Senior Data Engineer I - Amex Ads, Offers, and Dining Technology New York City Metropolitan Area 5 months ago Senior Data Engineer ...

related activities for datasets used by Quants and Traders. The role requires strong SQL and Python skills (PySpark, Pandas, Jupyter Notebooks), plus experience with Airflow, Linux/Bash, and data-quality practices. You will collaborate with stakeholders and help improve data pipelines. #J-18808-Ljbffr ...

/batch pipelines, and partner with marketing, analytics and engineering to deliver a scalable data platform. You will drive data resilience, implement dbt and Airflow workflows, and mentor engineers while shaping the CDP and MarTech data flows for global growth. #J-18808-Ljbffr ...

data platform, blending hands-on delivery with leadership and mentorship responsibilities. The role requires strong GCP experience (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow), advanced Python and SQL, and a track record of building scalable batch and real-time pipelines. #J-18808-Ljbffr ...

monitor models, build cloud-native infra with Kubernetes, and drive CI/CD and observability to scale AI services. You will work with Dagster, Airflow, or Prefect, use Terraform or Bicep, and collaborate with data scientists #J-18808-Ljbffr ...

understand and outperform the markets. Data science and machine learning skills are a strong plus. Exposure to modern development and operations tools, such as Airflow, Kubernetes, or similar technologies, is a plus. Prior fixed-income trading or portfolio management experience is not required. London, UKSummaryLocation: London, UKType: Full time ...

data analysis packages such as Numpy, Pandas, etc. Experience with SQL and/or other database technologies Knowledge of modern technologies such as AWS, Airflow, Docker, GitLab and Redis Ideally working with data visualization technologies such as Dash, Plotly, You will be joining a progressive and exciting company committed ...

understand and outperform the markets.* Data science and machine learning skills are a strong plus.* Exposure to modern development and operations tools, such as Airflow, Kubernetes, or similar technologies, is a plus. Prior fixed-income trading or portfolio management experience is not required.London, UK### About UsWe are a leading ...

taken a model through its full lifecycle before, from training through to serving it in production. You work with Python, PyTorch, SageMaker, and Airflow day to day. You default to owning problems end to end rather than waiting to be told what's next, and you're comfortable when ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

economy. Familiarity with Supabase specifically, or with similar BaaS platforms. BigQuery experience. Familiarity with n8n. Equivalent experience with code‐first orchestrators such as Temporal, Airflow or Prefect is welcome. Design sensibility. You care how things look and feel, not just how they function. Experience replacing legacy or external vendor ...

Spark for building and optimizing large-scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure ...

technical design reviews. Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… Avro). Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow). Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python ...

FastAPI) a huge plus; Experience in Kubernetes, Docker, Spark and related monitoring tools (e.g. DataDog, Grafana, Prometheus) for DataOps a huge plus; Experience with Airflow a huge plus; Experience with dbt for pipeline modelling also beneficial; Skilled at shaping needs into a solid set of requirements and designing scalable … main programming language Databricks Spark for data processing Databricks/dbt/SQL for data modelling and analytics Streamlit for data applications and visualisations Airflow for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Terraform for our infrastructure ...