6 of 6 Apache Airflow Jobs in the Midlands

relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL. Orchestration Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect. Data Quality and Lineage Familiarity with tools and techniques for data testing, documentation and lineage. Examples include Great Expectations or dbt. ...

complex data initiatives, providing guidance on solution design, engineering standards and implementation approaches Design, build and optimise distributed data processing pipelines using PySpark, Python, Airflow and AWS services Drive engineering excellence through code reviews, technical coaching, design reviews and adoption of software engineering best practices Partner closely with Product … Engineer, Principal Data Engineer or Technical Lead within a large-scale, data-driven organisation Strong hands-on software engineering expertise with Python, PySpark and Apache Airflow Deep experience designing and building distributed data processing systems and large-scale data platforms on AWS Strong knowledge of modern AWS data ...

analytics and reporting, and contributing to the adoption of AI-powered workflows and automation. Key Responsibilities & Skills: Build and maintain scalable data pipelines using Apache Airflow or similar orchestration tools. Develop integrations with internal and third-party APIs, ensuring reliable and efficient data flows. Design and optimise ...

Architectural Leaders hipDefine and evolve the enterprise data architecture strategy, with a primary focus on Cloud-scale Distributed Data Processing using Apache Spark and PySpark at the core of the platf ormEstablish and maintain architectural artefacts, reference architectures, Architecture Decision Records (ADRs), technical standards, and architecture roadmaps … Architect in a large, complex, data-led organisation, operating at an influence and strategy level rather than hands-on de liveryExpert-level understanding of Apache Spark and PySpark as the foundation of distributed data processing architectures, including tuning, optimisation, and enterprise-scale deployment pa tternsDeep expertise in designing ...

/SD/PP etc.) required for December cutover. Build scalable ingestion pipelines using Delta Lake/Parquet and modern orchestration tools (ADF, dbx, Airflow, etc.). Collaborate with SAP functional teams to validate business logic, deltas, and data quality. Implement CI/CD and DevOps practices for data ...

your initiative and get on with resolving it! From a tech perspective, it would be most excellent if you've come from an AWS, Apache AirFlow, Python, Spark type of environment and understand these technologies (or alternatives) and what they do. You'll still be considered ...