21 of 21 Apache Avro Jobs in London

Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps & Platform Engineering Experience implementing CI/CD pipelines and DevOps practices. Experience with infrastructure automation and environment management. Familiarity with: Docker ...

integration components and services, including Java-based microservices where required Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build ...

integration components and services, including Java-based microservices where required Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build ...

experience designing schemas for relational databases, messaging, data lakes/lakehouses (Databricks, Snowflake, or equivalent), and API/message formats (JSON Schema, OpenAPI, Avro, Protobuf). Strong understanding of metadata and lineage management.• 5+ years of experience, or demonstrated delivery against requirements in data modelling, data architecture ...

Columnar Databases, and/or Data Integration (ETL) or API development.Knowledge of some Data Formats such as JSON, XML, and binary formats such as Avro or Google Protocol Buffers.Experience collaborating with business and technical teams to understand, translate, review, and playback requirements and collaborate to develop Model Engineering solutions.Exposure ...

with data management technologies such as relational or columnar databases, data integration (ETL), or API development• Familiarity with data formats such as JSON, XML, Avro, or Google Protocol Buffers• Comfort engaging with the algorithms and mathematics behind languages, tooling, and data processing• Academic background paired with hands-on implementation ...

reconciliation and root-cause analysis.Experience building or supporting production data pipelines in a collaborative engineering environment.Experience working with distributed data processing frameworks such as Apache Spark.Working knowledge of common data formats such as JSON, Avro and Parquet.Technology EnvironmentThe role will involve working with a modern and evolving data … should bring relevant experience and the ability to work across comparable technologies.Examples of technologies in scope include:Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/CD tooling ...

root-cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark. Working knowledge of common data formats such as JSON, Avro and Parquet. Technology EnvironmentThe role will involve working with a modern … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/ ...

with data management technologies such as relational or columnar databases, data integration (ETL), or API development Familiarity with data formats such as JSON, XML, Avro, or Google Protocol Buffers Comfort engaging with the algorithms and mathematics behind languages, tooling, and data processing Academic background paired with hands‐on implementation ...

platforms.Solid Cloud and DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible.Experience building and supporting reactive, event-sourced systems.Hands-on experience with Apache Spark and Apache Flink.Strong understanding of Apache HTTP, Envoy, and REST API design.Experience working with Protobuf, Avro, and Parquet data formats.Knowledge ...

root-cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . For More Experienced Candidates Stronger ownership of technical … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, Kafka Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment: CI/CD tooling, containerised ...

accountable.Additional exposure to the following is desired: – Tech Stack you will learn· Hadoop and Flink· RUST, Javascript, React, Redux, Flow· Linux, Jenkins· Kafka, Avro, Kubernetes, Puppet· Involvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible. Experience building and supporting reactive, event-sourced systems. Hands-on experience with Apache Spark and Apache Flink. Strong understanding of Apache HTTP, Envoy, and REST API design. Experience working with Protobuf, Avro, and Parquet data ...

root‐cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, Kafka Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment: CI/CD tooling, containerised ...

GWTHighly motivatedDelivery focused and accountable. Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

firm then don’t worry.Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

firm then don't worry. Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

firm then don’t worry.Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

technical design reviews.Must Haves (Hard Skills)Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake.Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/Avro … engines (such as dbt) and scheduling tools (such as Airflow).Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka.Strong Programming Foundations: Proficiency in Python, SQL, or an Object-Oriented language (Java/Scala).Nice to HavesExperience operating data assets within a regulated ...

technical design reviews. Must Haves (Hard Skills)Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… Avro).Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow).Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python, SQL, or an Object-Oriented ...

bespoke integration approaches for source-system onboarding. Define conceptual, logical and physical data models, including data flows, data domains and data ownership considerations. Specify Apache NiFi architecture patterns, flow standards, interface contracts and reusable ingestion components. Define governance, metadata, lineage, auditability, security monitoring and data quality requirements. Produce High … lake architectures on AWS. Strong knowledge of AWS security, IAM, networking, storage and data platform services relevant to secure data engineering. Extensive experience with Apache NiFi architecture, flow design, governance and operational considerations. Experience designing integrations using REST APIs, SOAP services, file transfers, batch ingestion, legacy adapters and bespoke ...