3 of 3 Apache Airflow Jobs in Staffordshire

building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement robust ETL/… solutions for both structured and unstructured data. Automate workflows and orchestrate jobs using Airflow and GitHub Actions. Integrate data with third-party APIs to support real-time marketing insights. Collaborate closely with cross-functional teams including Data Science, Software Engineering and Product. Champion best practices in data governance, observability ...

services and tools, such as Salesforce, Segment, HubSpot, Stripe is beneficial Experience with SQL and data warehousing/pipelining tools such as BigQuery, Airflow, etc Proven experience in navigating and transforming legacy codebases, as well as leading greenfield projects ...

ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to define what "business-ready" means, and confidently sign off datasets … Experience Required Extensive hands-on experience with PySpark, including performance optimisation Deep working knowledge of Databricks (development, architecture, and operations) Proven experience working with Airflow for orchestration Proven track record in managing and securing PII data, with GDPR compliance in mind Experience in data governance processes; Alation experience preferred ...