1 to 25 of 35 Apache Jobs in the North of England

will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We are looking ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

engineering teams. Strong Python and SQL skills, with experience working with large-scale datasets and distributed data processing environments. Experience with technologies such as Apache Spark and modern data engineering frameworks. Experience with cloud platforms and data services, ideally AWS, including technologies such as Redshift and S3. Experience working ...

Computer Science or equivalent practical experience Experience developing finance applications is a plus Working experience with Terraform is a plus Experience creating workflows for Apache Airflow is a plus What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work ...

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

Software Engineer on large software applications. Proficient in technologies including Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, and Apache Spark. Experience working with one or more of the following database systems: DynamoDB, DocumentDB, MongoDB. Demonstrated expertise in unit testing and tools such as JUnit ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability – Experience ...

design, build, and support scalable data pipelines powering Moody’s next-generation digital content platform. You will develop data workflows using Databricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS to turn diverse data sources into reliable analytics. You will collaborate with cross-functional teams and mentor peers ...

Hybrid : London or Manchester - 2 days per week. This role requires the ability to hold an active security clearance. Skills : Python and SQL . Apache Airflow for DAG orchestration and monitoring. AWS data services: Redshift , OpenSearch , Lambda , Glue , Step Functions , Batch . CI/CD pipelines and YAML-based ...

Engineer delivering data engineering or integration platforms in cloud environments. Strong hands-on experience building production-grade ETL/ELT pipelines and orchestration with Apache Airflow . Solid experience on Google Cloud Platform , such as: Data services (e.g., BigQuery, Cloud Storage, Pub/Sub, Dataflow) and/or container ...

data quality, validation, reconciliation, and audit controls across betting, trading, and customer data domains. Develop streaming and micro-batch data solutions using AWS Kinesis, Apache Kafka, or Amazon MSK to support real-time sportsbook and in-play data feeds. Build and maintain data solutions using AWS services including … essential Tech snapshot Primary cloud AWS — Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, Lake Formation, Step Functions, MWAA Streaming AWS Kinesis, Apache Kafka/MSK, Apache Flink, Spark Streaming Languages Python, SQL, Spark Pipeline tools dbt, Airflow/AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift ...

deliver secure, reliable, and scalable services that support critical public sector operations. Key Responsibilities Design, develop, and maintain data pipelines and processing workflows using Apache Spark and Python Build, schedule, and monitor workflow orchestration using Apache Airflow Develop and maintain infrastructure-as-code using Terraform across AWS environments … sprint planning, refinement, and other agile ceremonies Required Skills & Qualifications Experience with AWS services Experience with Terraform for infrastructure as code Experience with Apache Spark for data processing Experience with Apache Airflow for workflow orchestration Experience with Docker and containerisation Experience building and maintaining CI/CD pipelines ...

performance database environments Excellent communication and cross-functional collaboration skills A passion for continuous learning and innovation Desirable skill: Databricks Azure Fabric Confluent/Apache Flink Knowledge Grow with Acorn At Acorn Insurance, we're proud of our Liverpool roots - and even prouder of how far we've come. ...

test, and deploy ETL pipelines to source, cleanse, and optimise data at scale. Automate and orchestrate data workflows: Develop and maintain orchestration solutions (e.g. Apache Airflow), implementing testing, alerting, monitoring, and data quality checks as first‐class concerns. Ensure data quality and observability: Embed data testing and validation processes … event streaming platforms including Kafka and its core concepts on topics, messages, and consumers. Experience building and operating automated data workflows using tools like Apache Airflow. Experience implementing data quality and validation checks throughout pipelines to ensure accuracy, completeness, consistency, and reliability. Practical experience implementing monitoring, observability and alerting ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java KEY SKILLS: DATA ENGINEER/ ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

Consulting Global is seeking a Technical Lead to steer the development of a scalable, multi-tenant data integration platform on Google Cloud Platform using Apache Airflow. You will shape the technical direction, drive reliable ETLs, and raise the bar on automation, governance and performance across environments. You will mentor ...

cross-functional teams to meet audit requirements, work with microservices and cloud-native tech, and contribute to IaC/tooling improvements across Terraform and Apache #J-18808-Ljbffr ...

Lead the engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Apache Airflow to orchestrate ETL pipelines and enables reliable movement of data from source systems through the platform into downstr click apply for full ...

seeking a Secure Data Engineer in Newcastle. This role focuses on building scalable and secure data platforms for Defence clients using Azure, Databricks, and Apache Spark. The successful candidate will engage in developing robust data applications, ensuring high security and compliance standards. Opportunities for professional growth and a flexible ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/ ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern observability ...