1 to 25 of 31 Apache Avro Jobs in the UK

integration components and services, including Java-based microservices where required Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build ...

Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps & Platform Engineering Experience implementing CI/CD pipelines and DevOps practices. Experience with infrastructure automation and environment management. Familiarity with: Docker ...

Event streaming or real-time processing with Kafka, Kinesis, Flink, Spark Structured Streaming, or similar technologies.• Distributed processing with Spark and familiarity with Parquet, Avro, JSON, and open table formats.• Search or retrieval systems such as Elasticsearch/OpenSearch, vector databases, or embedding pipelines.• Feature stores, ML data pipelines ...

tenant isolation, row‐and column‐level security, and PII handling. Define the streaming and batch ingestion architecture, building on our investments in NATS JetStream, Apache Flink, and Debezium CDC patterns, and direct the engineering team on execution. Specify the catalog, lineage, and data‐quality foundation and own the operational … streaming vs batch). Hands‐on experience with: Dremio, Snowflake, or Databricks AWS data services (S3, Glue, Athena, Lake Formation, or equivalent) NATS/Apache Kafka or similar, and CDC patterns (Debezium or equivalent) Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation ...

Autodesk's internal data models* **API & Service Design**: Design APIs and services optimized for REST, GraphQL, and gRPC interfaces; define serialization formats (JSON Schema, Avro, Protobuf) and ensure backward compatibility at scale* **Reliability & Performance**: Ensure reliability, security, and performance of systems; define SLOs and drive observability (metrics, tracing, logging ...

Databases, and/or Data Integration (ETL) or API development. Knowledge of some Data Formats such as JSON, XML, and binary formats such as Avro or Google Protocol Buffers. Experience collaborating with business and technical teams to understand, translate, review, and playback requirements and collaborate to develop Model Engineering ...

Databases, and/or Data Integration (ETL) or API development. Knowledge of some Data Formats such as JSON, XML, and binary formats such as Avro or Google Protocol Buffers. Experience collaborating with business and technical teams to understand, translate, review, and playback requirements and collaborate to develop Model Engineering ...

experience designing schemas for relational databases, messaging, data lakes/lakehouses (Databricks, Snowflake, or equivalent), and API/message formats (JSON Schema, OpenAPI, Avro, Protobuf). Strong understanding of metadata and lineage management. 5+ years of experience, or demonstrated delivery against requirements in data modelling, data architecture ...

with data management technologies such as relational or columnar databases, data integration (ETL), or API development Familiarity with data formats such as JSON, XML, Avro, or Google Protocol Buffers Comfort engaging with the algorithms and mathematics behind languages, tooling, and data processing Academic background paired with hands-on implementation ...

with data management technologies such as relational or columnar databases, data integration (ETL), or API development Familiarity with data formats such as JSON, XML, Avro, or Google Protocol Buffers Comfort engaging with the algorithms and mathematics behind languages, tooling, and data processing Academic background paired with hands-on implementation ...

root-cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark. Working knowledge of common data formats such as JSON, Avro and Parquet. Technology EnvironmentThe role will involve working with a modern … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/ ...

root-cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark. Working knowledge of common data formats such as JSON, Avro and Parquet. For More Experienced CandidatesStronger ownership of technical design across multiple … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/ ...

accountable. Additional exposure to the following is desired: – Tech Stack you will learn· Hadoop and Flink· RUST, Javascript, React, Redux, Flow· Linux, Jenkins· Kafka, Avro, Kubernetes, Puppet· Involvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

with data management technologies such as relational or columnar databases, data integration (ETL), or API development Familiarity with data formats such as JSON, XML, Avro, or Google Protocol Buffers Comfort engaging with the algorithms and mathematics behind languages, tooling, and data processing Academic background paired with hands‐on implementation ...

GWTHighly motivatedDelivery focused and accountable. Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

firm then don't worry. Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

firm then don't worry. Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

firm then don't worry. Additional exposure to the following is desired: – Tech Stack you will learnHadoop and FlinkRUST, Javascript, React, Redux, FlowLinux, JenkinsKafka, Avro, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. Home work is encouraged but you will need to be able to come ...

DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible. Experience building and supporting reactive, event-sourced systems. Hands-on experience with Apache Spark and Apache Flink. Strong understanding of Apache HTTP, Envoy, and REST API design. Experience working with Protobuf, Avro, and Parquet data ...

root‐cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, Kafka Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment: CI/CD tooling, containerised ...

root‐cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . For More Experienced Candidates Stronger ownership of technical … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, Kafka Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment: CI/CD tooling, containerised ...

/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security … etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. #J-18808-Ljbffr ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security … etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent ...

Solid knowledge of Linux/Unix, OpenShift/Kubernetes, and Ansible. Distributed Systems: Experience building and supporting reactive, event-sourced systems, with hands-on Apache Spark and Apache Flink. APIs & Data Formats: Strong understanding of Apache HTTP, Envoy, and REST API design, plus experience with Protobuf, Avro ...