1 to 25 of 40 Apache Jobs in the North of England

Automated deployments across environments. Monitoring and logging. Release and configuration management. Production support and incident troubleshooting. Python or another scripting language. AWS certification. Terraform. Apache Airflow/MWAA. Databricks or Spark. Kafka/Kinesis. dbt. Experience working in Agile/Scrum environments. Experience 4+ years of experience in Data ...

will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We are looking ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

clean, maintainable, and efficient code, ensuring adherence to best practices through code reviewsBig Data EngineeringDesign, develop, and maintain data pipelines and ETL workflows using Apache Spark and Apache AirflowOptimise data storage, retrieval, and processing systems to ensure reliability, scalability, and performanceDevelop and fine-tune complex queries using Apache … related field (or equivalent experience)8+ years of experience in software and/or data engineering with expertise in big data technologies such as Apache Spark and Apache AirflowAdvanced SQL skills, with expertise in query optimisation for large datasetsExpertise in developing APIs and robust backend services, with strong ...

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

stakeholder/data consumer management. Strong programming skills in languages such as Python, familiarity with large scale, distributed data processing engines such as Apache Spark, with a solid understanding of data structures and algorithms. Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL) or NoSQL databases (e.g. ...

engineering teams. Strong Python and SQL skills, with experience working with large-scale datasets and distributed data processing environments. Experience with technologies such as Apache Spark and modern data engineering frameworks. Experience with cloud platforms and data services, ideally AWS, including technologies such as Redshift and S3. Experience working ...

degree in Computer Science or equivalent practical experienceExperience developing finance applications is a plusWorking experience with Terraform is a plusExperience creating workflows for Apache Airflow is a plus#LI-JB6What's Roku's approach to hybrid working Roku fosters an inclusive and collaborative environment where teams generally work ...

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

Software Engineer on large software applications. Proficient in technologies including Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, and Apache Spark. Experience working with one or more of the following database systems: DynamoDB, DocumentDB, MongoDB. Demonstrated expertise in unit testing and tools such as JUnit ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability – Experience ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability – Experience ...

design, build, and support scalable data pipelines powering Moody’s next-generation digital content platform. You will develop data workflows using Databricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS to turn diverse data sources into reliable analytics. You will collaborate with cross-functional teams and mentor peers ...

strong understanding of modern data engineering best practicesYou have experience with Databricks, ideally also AzureYou have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and PandasYou're excited to join a start-up in a role that you can shape and influenceYou ...

knowledge Demonstrable experience designing, deploying, and operating big data platforms at scale. Hands‐on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies). Knowledge of cloud platforms and traditional on‐premise hosting; hands‐on experience with Azure, AWS and/ ...

Hybrid : London or Manchester - 2 days per week. This role requires the ability to hold an active security clearance. Skills : Python and SQL . Apache Airflow for DAG orchestration and monitoring. AWS data services: Redshift , OpenSearch , Lambda , Glue , Step Functions , Batch . CI/CD pipelines and YAML-based ...

Engineer delivering data engineering or integration platforms in cloud environments. Strong hands-on experience building production-grade ETL/ELT pipelines and orchestration with Apache Airflow . Solid experience on Google Cloud Platform , such as: Data services (e.g., BigQuery, Cloud Storage, Pub/Sub, Dataflow) and/or container ...

deliver secure, reliable, and scalable services that support critical public sector operations. Key Responsibilities Design, develop, and maintain data pipelines and processing workflows using Apache Spark and Python Build, schedule, and monitor workflow orchestration using Apache Airflow Develop and maintain infrastructure-as-code using Terraform across AWS environments … sprint planning, refinement, and other agile ceremonies Required Skills & Qualifications Experience with AWS services Experience with Terraform for infrastructure as code Experience with Apache Spark for data processing Experience with Apache Airflow for workflow orchestration Experience with Docker and containerisation Experience building and maintaining CI/CD pipelines ...

performance database environments Excellent communication and cross-functional collaboration skills A passion for continuous learning and innovation Desirable skill: Databricks Azure Fabric Confluent/Apache Flink Knowledge Grow with Acorn At Acorn Insurance, we're proud of our Liverpool roots - and even prouder of how far we've come. ...

Upgrades and Support Package Stack patchingSystem copies & refreshesLog and trace file analysisSolution ManagerHANA Database AdministrationDesirable but not essential: Installation Performance analysisWeb Dispatcher, SAP Router, Apache administration skillsSSL/TLS knowledgeCloud Connector install and configSSO/IAMKnowledge of SAP/HANA/Oracle HA and DR set upBackup & RecoverySAP Cloud ...

test, and deploy ETL pipelines to source, cleanse, and optimise data at scale. Automate and orchestrate data workflows: Develop and maintain orchestration solutions (e.g. Apache Airflow), implementing testing, alerting, monitoring, and data quality checks as first‐class concerns. Ensure data quality and observability: Embed data testing and validation processes … event streaming platforms including Kafka and its core concepts on topics, messages, and consumers. Experience building and operating automated data workflows using tools like Apache Airflow. Experience implementing data quality and validation checks throughout pipelines to ensure accuracy, completeness, consistency, and reliability. Practical experience implementing monitoring, observability and alerting ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java KEY SKILLS: DATA ENGINEER/ ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

Consulting Global is seeking a Technical Lead to steer the development of a scalable, multi-tenant data integration platform on Google Cloud Platform using Apache Airflow. You will shape the technical direction, drive reliable ETLs, and raise the bar on automation, governance and performance across environments. You will mentor ...

cross-functional teams to meet audit requirements, work with microservices and cloud-native tech, and contribute to IaC/tooling improvements across Terraform and Apache #J-18808-Ljbffr ...