76 to 100 of 149 Apache Airflow Jobs in the UK excluding London

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We are looking … relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL.* **Orchestration** Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect.* **Data Quality and Lineage** Familiarity with tools and techniques for data testing, documentation and lineage. Examples include Great Expectations or dbt. ...

data platform. You'll play a key role in building and optimising scalable data pipelines and analytics solutions, working across AWS, Snowflake, dbt and Airflow to deliver high-quality data products. The Company They are a well-established media organisation with a large and diverse audience, using data … models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline ...

code Requirements: Must have strong, commercial AWS (contract) Data Engineering experience Must bring solid knowledge of data warehousing and modelling Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial Must have experience with Redshift Nice to have: Experience with dbt/lake formation ...

code Requirements: - Must have strong, commercial AWS (contract) Data Engineering experience - Must bring solid knowledge of data warehousing and modelling - Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial - Must have experience with Redshift Nice to have: - Experience with dbt/lake formation ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

data engineering. You'll likely have experience with: Python SQL AWS, Azure or GCP Building production data pipelines Data modelling and data platforms Git Airflow or similar orchestration technologies Testing, debugging and improving data reliability We're particularly interested in engineers who have worked in consultancy, professional services ...

Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing. Discover more about our Analytics Community ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systemsWhat ...

Cloud Run), AWS (Bedrock, SageMaker) Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar. Ability to design ...

timely delivery while collaborating with engineers, analysts and compliance teams. The role requires strong Python/Java, SQL and AWS experience, familiarity with Airflow, dbt and Spark, and a practical understanding of CI/CD and Agile delivery in a fast-paced financial services environment. #J-18808-Ljbffr ...

team mentorship. Key Skills: Strong experience in a Lead/Senior Data Engineering role Expertise with GCP (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow) Advanced Python and SQL Experience building scalable batch and real-time data pipelines Strong data modelling, CI/CD, Git, and Terraform knowledge Experience ...

Python NLP models deployed on AWS (Lambda, ECS, SageMaker, SQS, SNS) Train, evaluate, and monitor ML models using orchestration tools such as Flyte or Airflow Manage ML pipelines on AWS with containerised services and CI/CD via GitHub Actions Implement streaming data processing with Kafka for real-time ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

CycleSolid understanding of agile methodologies such as CI/CD, Applicant Resiliency, and Security, Jules, Jenkins, Git/StashPreferred qualifications, capabilities, and skillsKnowledge of Airflow, Data Mesh architectureExperience with data analysisJ.P. Morgan is a global leader in financial services, providing strategic advice and products to the world's most ...

advanced analytics - ideally in a hybrid PLG/sales-led SaaS environment Strong Python and SQL; experience with data modelling and pipeline tools (dbt, Airflow, Dagster) Deep expertise in predictive modelling, causal inference, survival analysis, and experimentation (A/B, diff-in-diff, instrumental variables) Experience working directly with ...

Cloud Run), AWS (Bedrock, SageMaker)* Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK* Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar.* Ability to design ...

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

Engineer delivering data engineering or integration platforms in cloud environments. Strong hands-on experience building production-grade ETL/ELT pipelines and orchestration with Apache Airflow . Solid experience on Google Cloud Platform , such as: Data services (e.g., BigQuery, Cloud Storage, Pub/Sub, Dataflow) and/ ...

within financial markets/securities . Experience working with GCP BigQuery . Experience with GCP Cloud SQL . Experience with Terraform . Experience with Apache Airflow . Understanding of security reference data, pricing data or corporate actions. The Project The wider project is focused on creating a multi ...

Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added advantage. ...

tools (Git), Fluent English – for international communication and collaboration. Nice to have: Experience with LLMs, or Time Series, Familiarity with tools like Docker, MLflow, Airflow, AWS/GCP/Azure, Background in research or R&D, Experience working in cross-functional or international teams. Soft Skills: Curiosity and problem ...

agile methodologies such as CI/CD, Applicant Resiliency, and Security, Jules, Jenkins, Git/Stash Preferred qualifications, capabilities, and skills Knowledge of Airflow, Data Mesh architecture Experience with data analysis #J-18808-Ljbffr ...