76 to 100 of 110 Apache Airflow Jobs in England

Scala experience Experience delivering and maintaining distributed data processing systems using Big Data tools, lakehouse platforms, and workflow orchestration (experience with Spark, Databricks, or Airflow is a plus) Experience building with cloud platforms such as AWS, Azure, or GCP Experience working with AI tools or platforms (LLMs, predictive models ...

serving, scaling and monitoring workflows (e.g., BentoML, MLflow, Vertex AI, AWS Sagemaker) Experience building streaming + batch data ingestion and transformation pipelines (Spark/Airflow/Beam) Mobile and front-end web application development experience What We Offer At Newpage, we’re building a company that works smart ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8 10 years ' experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop ...

data platforms. Hands on experience with ETL development, data modelling and data normalization. Background in AWS environments and modern orchestration tools such as Airflow or Dagster. Confidence leading client conversations, running workshops and defining solutions Experience within the music industry is a significant advantage. BENEFITS The successful Head ...

data platforms. Hands on experience with ETL development, data modelling and data normalization. Background in AWS environments and modern orchestration tools such as Airflow or Dagster. Confidence leading client conversations, running workshops and defining solutions Experience within the music industry is a significant advantage. BENEFITS The successful Head ...

data platforms. Hands on experience with ETL development, data modelling and data normalization. Background in AWS environments and modern orchestration tools such as Airflow or Dagster. Confidence leading client conversations, running workshops and defining solutions Experience within the music industry is a significant advantage. BENEFITS The successful Head ...

data platforms. Hands on experience with ETL development, data modelling and data normalization. Background in AWS environments and modern orchestration tools such as Airflow or Dagster. Confidence leading client conversations, running workshops and defining solutions Experience within the music industry is a significant advantage. BENEFITS The successful Head ...

data platforms. Hands on experience with ETL development, data modelling and data normalization. Background in AWS environments and modern orchestration tools such as Airflow or Dagster. Confidence leading client conversations, running workshops and defining solutions Experience within the music industry is a significant advantage. BENEFITS The successful Head ...

data platforms. Hands on experience with ETL development, data modelling and data normalization. Background in AWS environments and modern orchestration tools such as Airflow or Dagster. Confidence leading client conversations, running workshops and defining solutions Experience within the music industry is a significant advantage. BENEFITS The successful Head ...

engineering and product teams to support evolving data requirements Requirements: SQL Server T-SQL DBA Skills Power BI ETL (SSIS or other, such as Airflow) MySQL would be useful but certainly not essential. Apply Now! If you have a range of experience in Data Engineering and you are looking ...

/Gas), who is technically well versed in systematic trading development, and analysis data-sets (time series, market-data). Stack: Python, AWS, ETL, Airflow The team has a flat structure, where developers are interfaced with PMs and traders, to ensure the software systems created are bespoke. ...

Senior Data Engineer (AWS, Airflow, Python) Based at client locations, working remotely, or based in our Godalming or Milton Keynes offices. Salary up to £65k plus company benefits. About Us Triad Group Plc is an award-winning digital, data, and solutions consultancy with over 35 years' experience primarily serving … maintain cloud data solutions. Maintain modular Terraform based IaC for reliable provisioning of AWS infrastructure. Develop, optimise and maintain robust data pipelines using Apache Airflow. Implement data transformation processes using Python to clean, preprocess, and enrich data for analytical use. Collaborate with data analysts, data scientists, developers, and other ...

Senior Data Engineer (AWS, Airflow, Python) Based at client locations, working remotely, or based in our Godalming or Milton Keynes offices. Salary up to £65k plus company benefits. About Us Triad Group Plc is an award-winning digital, data, and solutions consultancy with over 35 years' experience primarily serving … maintain cloud data solutions. Maintain modular Terraform based IaC for reliable provisioning of AWS infrastructure. Develop, optimise and maintain robust data pipelines using Apache Airflow. Implement data transformation processes using Python to clean, preprocess, and enrich data for analytical use. Collaborate with data analysts, data scientists, developers, and other ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

Linux environment. - Experience in building automation tools and managing system configurations. - Knowledge of C++, KDB/q, and experience with technologies like Slurm, Airflow, Kafka, or AMPS. - Background in enhancing system stability, scalability, and performance while conducting root cause analyses to resolve incidents efficiently. - Observability skillset, monitoring and analysis ...

AI. Comfortable working in a fast-paced, agile environment. Excellent communication and collaboration skills. Nice to Have Experience with workflow orchestration tools (e.g., Temporal, Airflow). Familiarity with vector databases or semantic search tools. Experience building internal tooling or developer productivity enhancements using AI. Exposure to customer support ...

deliver reliable, high-performance data solutions using Databricks and modern cloud technologies. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake Build and optimize ETL/ELT workflows for batch and streaming data Develop data models to support analytics, reporting, and machine … data pipelines, architectures, and best practices Required Qualifications 3+ years of experience in data engineering or a similar role Strong experience with Databricks and Apache Spark Proficiency in Python and SQL Experience with Delta Lake, data modeling, and performance tuning Hands-on experience with cloud platforms (AWS, Azure ...

looking for Extensive experience in data engineering, including time in a lead role Strong Python and SQL skills Experience with orchestration tools (Airflow, Dagster, Prefect) Hands-on dbt , AWS, and modern data warehouses Strong data modelling and architectural capability Background scaling data platforms in high-growth environments Nice … have Snowflake or Apache Iceberg Infrastructure-as-Code (Terraform) Observability-driven data systems Package & Benefits Salary up to £120,000 Fully remote working Home-office setup Private healthcare and income protection 25 days holiday + enhanced parental leave Strong learning and development budget ...

objects (databases, schemas, tables, views, tasks, streams, resource monitors). ETL/ELT Pipelines: Develop and maintain robust data pipelines using tools like dbt, Airflow, Azure Data Factory, or similar. Data Modelling: Implement dimensional models (star/snowflake schemas), handle SCDs, and design efficient structures for analytics. Performance Tuning … Degree in IT Strong SQL skills (complex queries, performance tuning) and proficiency in Python for data processing. Experience with ETL/ELT tools (dbt, Airflow, ADF, Informatica, Matillion). Solid understanding of data warehousing concepts (Kimball, Data Vault, normalization). Familiarity with cloud platforms (Azure preferred; AWS/ ...

What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs … principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join ...

What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs … principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join ...

with engineers, architects, and product managers on data solutions. Support deployment, testing, and documentation of data products. Work with cloud-based tools, including BigQuery, Airflow, Terraform, DBT, and GIT. Continuously learn new tools and technologies, contributing to team knowledge and best practices. Key Skills & Experience: 1–3 years’ experience … roles. Strong SQL and Python skills. Exposure to cloud platforms (GCP preferred; Azure/AWS acceptable) . Familiarity with data tooling: DBT, GIT, Terraform, Airflow, Docker (nice to have). Good problem-solving skills, attention to detail, and collaborative mindset. Interested in Agile practices, Test-Driven Design, and modern ...

skills with the ability to work cross-functionally with stakeholders, engineers, and analysts. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT, Docker Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable skills/knowledge/… Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. Familiarity with orchestration tools like Airflow or Azure Data Factory. ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...