14 of 14 Apache Jobs in the North East

Python, Java or Scala, and SQL. Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms ...

data estate. Deep expertise in Microsoft Azure cloud services — ADF, ADLS, Synapse, Purview. Databricks — Delta Lake architecture, optimisation and advanced data processing. Apache Spark for large‐scale distributed computing and performance tuning. Microsoft Fabric — OneLake and DirectLake integration. Azure Synapse Analytics for enterprise‐scale data warehousing. Infrastructure‐as‐Code ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering workflows - code generation, code review, pipeline debugging, or documentation Strong coding experience ...

professionals with strong Python development expertise, cloud engineering experience, and exposure to modern DevOps and data engineering practices. Must Have: AWS services; Terraform; Apache Spark; Python; Apache Airflow; Docker; GitLab; Security scanning agent; Jupyter; Notebooks; Key Responsibilities Design, develop, and maintain scalable Python applications and services Build … optimize cloud-native data processing pipelines Develop and manage workflows using orchestration tools such as Airflow Work with large-scale data processing frameworks including Apache Spark Deploy and maintain containerized applications using Docker Implement infrastructure automation using Terraform Integrate solutions with enterprise data and secure file transfer systems Collaborate ...

design, including data quality testing, cleansing, and monitoring, and data warehousing and data modelling capabilities Experience of cloud technologies including AWS, Snowflake, SQL, and Apache Airflow, SQL and programming languages such as PL/SQL and Python, Spark, and Pyspark Good knowledge of modern code development practices Strong communication ...

design, including data quality testing, cleansing, and monitoring, and data warehousing and data modelling capabilities Experience of cloud technologies including AWS, Snowflake, SQL, and Apache Airflow, SQL and programming languages such as PL/SQL and Python, Spark, and Pyspark Good knowledge of modern code development practices Strong communication ...

MySQL, PostgreSQL, MongoDB) Experience with software delivery and CI/CD Experience with software service metrics and monitoring tools such as Grafana, Plausible and Apache Superset. Familiarity with Linux server administration and networking Experience directly interacting with end users, replying professionally to queries and questions Enthusiasm for growing ...

Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena ...

data validation and application testing processes across distributed systems Investigate production issues using data analysis to identify root causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats … additional programming language (e.g., Java) Experience with scripting languages such as Bash Strong understanding of automated testing frameworks and methodologies Hands-on experience with Apache Airflow and log analysis Familiarity with data processing tools such as Apache Spark Experience working with cloud platforms and services, including: Monitoring ...

Design, build, and maintain cloud-based infrastructure using AWS Develop and manage Infrastructure as Code using Terraform Support and optimise data processing pipelines using Apache Spark and Airflow Collaborate with cross-functional teams to deliver secure and scalable solutions Implement CI/CD pipelines using GitLab Containerise applications … Notebooks for data exploration and development Key Skills & Experience Strong experience with AWS services Expertise in Terraform (Infrastructure as Code) Hands-on experience with: Apache Spark Python Apache Airflow Docker GitLab CI/CD Experience with security scanning tools such as: Trivy Trend Micro Wiz (or similar) Knowledge ...

Role – Confluent Consulting Engineer Technology – Apache Kafka, Confluent Platform, Stream Processing Location –(Remote) UK Job Description Your role: As a Confluent Consulting Engineer, you will be responsible for designing, developing, and maintaining scalable real-time data pipelines and integrations using Kafka and Confluent components. You will collaborate with data … engineers, architects, and DevOps teams to deliver robust streaming solutions. Required: · 5+ years of hands-on experience with Apache Kafka (any distribution: open-source, Confluent, Cloudera, AWS MSK, etc.) · Strong proficiency in Java, Python, or Scala · Solid understanding of event-driven architecture and data streaming patterns · Experience deploying Kafka ...

millions of developers, data practitioners, and AI agents worldwide from high-level data visualization plugins (Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tooling. Our work directly shapes how companies process massive datasets: real-time analytics platforms ingesting … should build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse ...

DevOps Engineer/SysAdmin - IBM Cloud, AWS, Oracle Cloud, Linux, RHEL, Apache, Tomcat, Docker, Kubernetes, Java, Salt, Puppet, Chef, Terraform, MySQL, Oracle, Shell Scripting; ITIL. Contract, 12 Months+, Inside IR35, Bedfordshire/Fully Remote/UK Wide. £425 - £540 (Contract Rate) Well established business consultancy seeks Devops Engineer/… Chef, Terraform Manage AWS and IBM Cloud environments using Shell Scripting techniques and core Java where applicable Configure web servers and containerisation tech inc Apache, Tomcat, Docker, Kubernetes We are open to DevOps Engineer/Systems Administrators/SysAdmins from all industries to assist with the workload during ...