6 of 6 Apache Jobs in the North East

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. This role combines hands-on technical leadership with test strategy ownership . … reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis for production data issues and platform defects Risk Management & Reporting Identify, assess, and communicate quality risks across releases ...

Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena ...

Will Work With Designing, building and maintaining scalable data pipelines. Developing ETL/ELT processes to ingest and transform data from multiple sources. Apache Airflow for workflow orchestration, scheduling and automation. Azure SQL and cloud-based data platforms. Data modelling, warehousing and optimisation activities. Data quality, governance and validation … optimising data pipelines. Knowledge of ETL/ELT development and data integration. Python scripting for automation, transformation and data engineering tasks. Experience with Apache Airflow or similar orchestration tools. Understanding of data warehousing and data modelling principles, including dimensional modelling. Experience working with cloud-based data platforms. Familiarity with ...

involved in a modern data transformation programme, including a planned migration to Microsoft Fabric. Key Responsibilities Develop and maintain data pipelines using Apache Airflow. Design and optimise SQL-based data solutions. Build and enhance Power BI reports and dashboards. Work closely with stakeholders to gather requirements and deliver insights. … Support ongoing improvements to the data platform and reporting environment. Contribute to the organisation's future migration to Microsoft Fabric. Required Skills Experience with Apache Airflow. Strong Azure SQL skills. Power BI development and data modelling experience. Good understanding of ETL/ELT processes and data warehousing concepts. Strong ...