4 of 4 Remote/Hybrid Contract Apache Jobs

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...

engineers should be adept at Extract, Transform, Load (ETL) processes. This includes using frameworks such as the Medallion Architecture, as well as tools like Apache Airflow, Fivetran, Azure Data Factory, and Amazon Glue to automate data workflows and ensure data quality. Familiarity with cloud computing services, such … Cloud Platform, is increasingly important. Data engineers often leverage these platforms for scalable data storage and processing capabilities. Knowledge of big data frameworks like Apache Spark, Beam is crucial for handling large datasets and real-time data processing. Understanding data modelling techniques helps data engineers design efficient data structures ...

engineer with strong experience in streaming technologies, event-driven architecture, and distributed data systems. The Role Design and build real-time data pipelines using Apache Flink , Apache Kafka , and AWS Kinesis Develop scalable event-driven and streaming solutions Integrate data from source systems including Oracle, MySQL, and Sybase … enrichment, and quality capabilities Deliver low-latency data into enterprise data platforms Create and manage infrastructure using Terraform Skills & Experience Strong commercial experience with Apache Flink Strong experience with Apache Kafka Experience with AWS Kinesis Advanced SQL skills and strong relational database knowledge Experience working with Oracle, MySQL ...

bespoke integration approaches for source-system onboarding. Define conceptual, logical and physical data models, including data flows, data domains and data ownership considerations. Specify Apache NiFi architecture patterns, flow standards, interface contracts and reusable ingestion components. Define governance, metadata, lineage, auditability, security monitoring and data quality requirements. Produce High … lake architectures on AWS. Strong knowledge of AWS security, IAM, networking, storage and data platform services relevant to secure data engineering. Extensive experience with Apache NiFi architecture, flow design, governance and operational considerations. Experience designing integrations using REST APIs, SOAP services, file transfers, batch ingestion, legacy adapters and bespoke ...