76 to 100 of 283 Apache Airflow Jobs in England

maintaining scalable data platforms. Strong SQL and data modelling skills, with experience designing analytical datasets. Experience with modern data stack tools (e.g. dbt, Spark, Airflow, or similar orchestration and transformation frameworks). Experience with cloud‐based data platforms, ideally AWS. Understanding of medallion architecture or similar data layering approaches. ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

consulting experience within BFSI, leading complex enterprise transformations. Platform Expertise: Deep technical proficiency in modern data platforms (Databricks, Snowflake) and toolsets (dbt, Airflow, Kafka). Advanced Architecture & AI: Mastery of Lakehouse, Data Mesh, Streaming, MLOps, and GenAI integration to drive continuous innovation. Governance & Compliance: Expertise in zero-trust security ...

just how fast. We expect you to enjoy leveraging AI and agentic systems to operate more effectively. Tech environmentScala | Spark | Databricks | AWS | Claude | Airflow | Kubernetes | Terraform | Functional ProgrammingYou will thrive here if youEnjoy end-to-end ownership, from writing the code to architecture design to coaching others through delivery. ...

experienced with modern cloud data platforms such as BigQuery, Snowflake, or Databricks. You've worked with data orchestration tools such as Dagster, Airflow, or Prefect. We use Dagster. You're comfortable owning cloud infrastructure. You can work with infrastructure-as-code such as Terraform and own the infrastructure your ...

real-time and batch data processing pipelines using modern frameworks like Spark, Flink, Spark Streaming, Kafka, Debezium, etc. Experience with orchestration tools such as Airflow, dbt, or similar. Exceptional problem-solving skills paired with a proactive, innovative mindset focused on continuous improvement. Strong communication and cross-functional collaboration skills ...

Google Cloud. Key skills: Strong Python & SQL experience Hands-on Azure Databricks, Delta Lake & Unity Catalog Strong GCP/BigQuery/GCS experience Apache Airflow/Cloud Composer for pipeline orchestration Experience building ELT/data pipelines and cross-cloud data transfers Understanding of IAM, RBAC, secrets management ...

hands-on familiarity with vessel-tracking data, its structure, quirks, and limitations. SQL— strong proficiency, including query optimization on large datasets. Nice-to-Haves Apache Airflow (or equivalent orchestration) for scheduling and managing pipelines. AWS experience (Azure or GCP equally welcome). Machine Learning experience, particularly applied ...

Spark & distributed computing dbt Terraform Data Warehousing (we use Databricks but experience with other platforms is fine) MLOps implementation (desirable but not required) Airflow (or other orchestration tooling) #J-18808-Ljbffr ...

Optimise Python performance through effective use of loops , vectorization , threading , and understanding of the Global Interpreter Lock (GIL) . Desirable Experience Experience with Kubeflow , Airflow , or dbt . Hands-on experience with Google Cloud Platform (GCP) . Knowledge of containerisation (Docker, Kubernetes) and CI/CD practices. Familiarity with ...

marketing teams to align data solutions with business needs Requirements Strong SQL skills and experience with ETL/ELT and pipeline orchestration tools (e.g., Airflow, dbt, Data Factory) Experience with cloud data platforms (e.g., Azure, AWS, or GCP) Proficiency in data modelling (dimensional, entity-relationship, event-driven) Understanding ...

day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processing Extensive experience in dbt and ideally Airflow (or other orchestration tools) Extensive experience in Entity-Relationship modelling and Dimensional modelling Experience with Cloud Data Warehousing (eg Snowflake, Databricks or BigQuery ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

/or Genetics datasets Proficiency in cloud computing, data/scripting programming languages and workflow orchestration technologies (e.g. Python, R, Bash, SQL, Nextflow, Airflow, Databricks , etc.) Demonstrated expertise in ETL processes, data pipelines, metadata harmonisation, data lifecycle management, data product release processes and operational support Experience operationalising analytical workflows ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture design to coaching ...

translate requirements into scalable data solutions Experience mentoring, coaching or supporting the development of other analytics engineers Knowledge of orchestration tools such as Airflow or Composer Familiarity with cloud platforms, particularly Google Cloud Platform (GCP) And it would be great if you had any of the following... Experience with ...

/or Genetics datasets Proficiency in cloud computing, data/scripting programming languages and workflow orchestration technologies (e.g. Python, R, Bash, SQL, Nextflow, Airflow, Databricks, etc.) Demonstrated expertise in ETL processes, data pipelines, metadata harmonisation, data lifecycle management, data product release processes and operational support Experience operationalising analytical workflows ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricks* Ability to think strategically and relate architectural decisions ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing Vector Databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation and optimisation ...

required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation and optimisation ...

wider stakeholders. Technical Environment Experience with some of the following technologies would be beneficial: Python SQL Cloud platforms – AWS, Azure or GCP Databricks Apache Spark/PySpark Kafka/event-driven architectures Airflow dbt Git/GitHub CI/CD Infrastructure as Code Modern data platforms and distributed ...

Data Engineer within GCP environments Hands-on experience with BigQuery, Dataflow and Spanner Strong programming capability in Python, Java or Scala Experience with Apache Beam/Dataflow and distributed processing frameworks Experience designing and managing ETL/ELT pipelines Solid understanding of data modelling and database design Experience with … workflow orchestration tools such as Apache Airflow Strong understanding of data governance, security and scalability Desirable Skills for the GCP Data Engineer: Experience with Pub/Sub, Cloud Composer or Cloud Data Fusion Exposure to CI/CD, DevOps and Infrastructure as Code Experience with real-time streaming ...