226 to 250 of 422 Apache Jobs in the UK

with external clients across a variety of industry markets. Minimum of 5+ years of customer‐facing experience would be preferred Preferred: Experience working with Apache Spark (TM) to process large‐scale distributed datasets Communication: Proven ability to communicate and teach complex technical concepts to both technical and non‐technical ...

backend systems and data-intensive applications. You are experienced with Scala or Java and at least one distributed data processing framework such as Scio, Apache Beam, Spark, or Flink. You know how to design and build reliable backend services and scalable data pipelines that support business-critical workflows. ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

performance database environments Excellent communication and cross-functional collaboration skills A passion for continuous learning and innovation Desirable skill: Databricks Azure Fabric Confluent/Apache Flink Knowledge Grow with Acorn At Acorn Insurance, we're proud of our Liverpool roots - and even prouder of how far we've come. ...

application components, microservices, and data integration layers to support high-throughput regulatory reporting. Big Data Processing: Develop and optimize distributed data processing jobs using Apache Spark, Hadoop, and Hive to aggregate and transform large-scale financial datasets. Real-Time Streaming: Build and maintain real-time data ingestion and streaming … pipelines using Apache Kafka. Database Management: Design and optimize data models in MongoDB and relational databases, ensuring high performance for complex queries and data storage. Agile Execution: Drive sprint planning, task estimation, and daily progress within an Agile/Scrum framework to ensure timely delivery of project milestones. Requirement ...

platforms capable of processing billions of events per day and managing terabytes scale datasets. Develop modern Data Lakehouse platforms leveraging S3-compatible object storage, Apache Iceberg, Spark, and Trino. Build real-time streaming integration using Apache Kafka and Apache Flink for low-latency event processing, enrichment … OpenShift, Terraform, CI/CD, GitOps, and Infrastructure as Code. Lead metadata, lineage, and data discovery capabilities using technologies such as DataHub, OpenMetadata, or Apache Atlas to improve platform transparency and usability. Ensure platform security and regulatory compliance through fine-grained access controls, encryption, secrets management, auditing, and secure ...

have Programming: Proficiency in Python, Java, Scala, or similar languages for data processing. Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop. Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Database Management: Expertise in relational databases … e.g. Postgres, SQL Server). Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines ...

data engineering, with demonstrated expertise in designing and implementing scalable, flexible modern data architectures Experience building modern data lake/lakehouse architectures (e.g., Apache Iceberg) in cloud environments Strong knowledge of AWS (S3, Redshift) or Azure data services Experience with ETL/ELT processes and data ingestion Proven expertise … using transformation tools such as dbt Experience with data pipeline orchestration tools (such as Apache Airflow) Experience with best practices applied to data (e.g., testing, CI/CD) Advantageous Experience developing dashboards and reporting solutions using data visualization tools such as Power BI or Amazon QuickSight Experience with generative ...

skills Hands-on experience with Databricks for scalable data engineering, analytics, and collaborative development, as well as proficiency in managing large datasets using Apache Iceberg for efficient, reliable table management and data lake optimization Experience architecting and implementing enterprise-scale data warehouses on cloud platforms (e.g., AWS, Azure … data pipeline deployment and monitoring using CI/CD tools and infrastructure-as-code practices Hands-on expertise with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and real-time data streaming technologies (e.g., Kafka, Kinesis, Spark) Track record of collaborating with cross-functional teams (engineering, analytics, business stakeholders ...

test, and deploy ETL pipelines to source, cleanse, and optimise data at scale. Automate and orchestrate data workflows: Develop and maintain orchestration solutions (e.g. Apache Airflow), implementing testing, alerting, monitoring, and data quality checks as first‐class concerns. Ensure data quality and observability: Embed data testing and validation processes … event streaming platforms including Kafka and its core concepts on topics, messages, and consumers. Experience building and operating automated data workflows using tools like Apache Airflow. Experience implementing data quality and validation checks throughout pipelines to ensure accuracy, completeness, consistency, and reliability. Practical experience implementing monitoring, observability and alerting ...

provenance. Hands‐on experience with data lineage/metadata tooling in enterprise environments (e.g., Collibra, Alation, Informatica EDC/IDMC, IBM Infosphere, Microsoft Purview, Apache Atlas, Amundsen, DataHub or similar). Proven ability to build lineage for complex platforms: data lakes, warehouses, marts, and distributed processing (Spark‐based pipelines … production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys & Apache Airflow. Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift). Familiarity with AWS S3 for large-scale data storage. ...

About the Job you are considering: We are seeking a highly skilled Data Engineer with strong expertise in Apache Spark AWS Scala TABLEAU and JAVA Microservices Architecture to design and build scalable cloud native data platforms and applications. The ideal candidate will have experience in developing high performance data … other BI tools. Participate in architecture discussions code reviews and technical mentoring. Your Skills: Strong hands on experience in Tableau development. Expertise in Apache Spark for largescale data processing. Strong programming experience in Scala. Experience with AWS Cloud services and cloud native architecture. Deep understanding of Microservices Architecture REST ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java KEY SKILLS: DATA ENGINEER/ ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full time on site in Gloucester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Gloucester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

fields of data analytics and software Proficiency in Python programming language and common data science packages. Experience with data engineering tools (e.g. Apache Spark, AWS Glue, Kinesis, Snowflake, Apache Airflow) Experience in building data pipelines and data streaming applications. Experience with tools to support the software and analytics ...

role. Managing multi‐site Digital Signage platforms (AMP, Scala, Broadsign, Bightsign) is preferred. Server Technology Proficiency in configuring and managing web server software like Apache and Nginx. Databases & Caching Expertise in setting up and maintaining scalable MySQL databases and Redis cache clusters. Platforms & OS Experience managing multi‐site digital ...

root‐cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, Kafka Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment: CI/CD tooling, containerised ...

tools such as Jenkins, and automation testing tools such as Selenium, Cucumber, Flyway, or Concordion Knowledge of Linux, shell scripting, and application services including Apache, Tomcat, and Spring Boot Experience with user interface technologies (Angular, TypeScript, React) and user interface testing tools (Jasmine, Karma, Jest) Working experience with cloud ...

leading Data Management consultancy in London seeks a skilled professional experienced in PySpark and Apache Spark for a hands-on role. You will work closely with clients and play a crucial part in developing innovative solutions using cloud technologies like AWS, Azure, and GCP. Excellent communication and ability ...

streaming technologies such as Kafka and FlinkWorked in cloud Native environmentsDeployed to the cloud themselves using Docker/KubernetesPrevious use of technologies such as Apache DruidBenefitsExceptional bonus structureFlexible workingA team that will truly listen to any ideas and influence you could havePlease email Lily. Wolahan@stanfordblack.comor call ...

services Apply appropriate patterns using proven and emerging technologies Skills and ExperiencePython Linux Bash scripting Git Docker Messaging or streaming technologies (e.g. Kafka or Apache Pulsar) Experience working in Agile/Scrum teams Tooling such as Atlassian or GitLab Exposure to CI/CD pipelines ClearanceThis role requires candidates ...

service and critical incident reports. Leadership: Mentor support engineers, maintain strong client relationships, and coordinate UK datacentre visits when required. Requirements Essential Technical: Linux, Apache NiFi. Knowledge: ITIL 4 Foundation. Traits: Proactive, independent, adaptable, strong problem-solver with excellent interpersonal skills. Desirable Tech Stack Active Directory (ADDS), DNS, Network ...

Mockito Experience with clean architecture, DI, modular design Comfortable owning CI/CD, code quality, observability Familiarity with Docker, Maven, Jenkins ⭐ Nice to Have Apache Calcite Starburst or federated query engines JVM performance tuning High-throughput service interfaces (REST/gRPC) Data governance or policy enforcement layers 👥 This Role ...