5 Apache Avro Jobs

Experience in: Database orchestration technologies, specifically Airflow and/or DBT Experience with streaming data architectures, specifically Kafka Knowledge of semi structured data: Parquet, Avro, JSONA deep understanding of AWS Cloud Data technologies (RDS, DynamoDB, Aurora) Knowledge and experience with Snowflake and other databases (PostgreSQL, MS SQL Server, MySQL more »

maintaining data processing applications. Strong understanding of distributed computing concepts and big data technologies. Familiarity with data formats and storage systems such as Parquet, Avro, HDFS, and S3. Experience with Datamodelling, ETL processes, and data warehousing concepts. Day Rate: Up to £450 Per Day Inside IR35 more »

such as Deequ, Great Expectations or Delta Live Experience working with continuous integration Good understanding of data modelling (medallion architectures, Kimball architectures), data schemas (Avro Schema, JSON Schema) and data serialisation (Parquet, ORC, Avro) Azure Services Knowledge: Azure Data Factory: Advanced knowledge of Azure Data Factory for orchestrating … Ability to design and implement data warehousing solutions using Azure Synapse Analytics. Azure Databricks: Proficiency in using Azure Databricks for data processing and analytics. Apache Spark: Deep understanding of Apache Spark for large-scale data processing. Azure Blob Storage and Azure Data Lake Storage: Expertise in setting up more »

and Development experience in SOA (for e.g., Service Orchestration, Service Choreography, Event Driven Architecture, Messaging middleware, etc.). • Experience developing schemas (e.g., XSD, JSON, AVRO, etc.) and related data structures. • Experience with Continuous Integration/Continuous Deployment. • Experience in DevOps, Postman, PCF, GCP, Azure, Jenkins, GIT Hub. • Experience with more »

Software Engineer for this role, you will collaborate with the founding team to expand the integration of our Big Data processing acceleration technology with Apache Spark to drive new optimizations and broader SQL operation coverage. Your contributions to our core solution will directly impact data infrastructure processing 10s of … as batch processing code, data parsing, shuffling and data partitioning algorithms. Maintain the solution up to date and compatible with a variety of supported Apache Spark runtimes. Independently and diligently write, test and deploy production code driven by modern software engineering practices. Work with the internals of leading open … and previous experience in early stage startups. Experience with Scala. Experience with query processing optimization. Understanding of Big Data file formats (e.g. Parquet, ORC, Avro). Since we are a remote company, we are open to candidates from overseas. No recruitment agencies please. more »