15 of 15 Remote/Hybrid Parquet Jobs in London

similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON and Parquet Containerisation DevOps and DataOps We are interested in Engineers with a solid understanding of DevOps to enable ownership of cloud infrastructure and different environments ...

Data Build Tool (DBT) with a demonstrated ability indeveloping data models, contracts, tests, validation, and transformations Experienceworking with modern data distributed file formats (i.e., Parquet, Delta,Iceberg, Hudi) Demonstratedexperience with building data ingestion pipelines from REST API data sources. Strongability to produce technical documentation that can be understood ...

based environment Advanced experience working and understanding the tradeoffs of at least one of the following Data Lake table/file formats: Delta Lake, Parquet, Iceberg, Hudi Experience working with lakehouse/medallion architectures in Databricks Experience with experimentation support tooling (such as Optimizely) A genuine curiosity about ...

supporting non‐technical stakeholders in getting the most out of the data warehouse Experience ingesting and processing structured and unstructured files (CSV, JSON, XML, Parquet, PDF) from a variety of sources, including cloud storage buckets and third‐party exports. Desired but not essential skills and experience include Broader Google ...

invalidation, persistence strategy, data consistency MongoDB TimescaleDB NATS JetStream AWS S3/Glue Data lake staging, archival, batch ingestion into Snowflake Partitioning, file format (Parquet/ORC), Lake Formation governance Finance Domain Knowledge Candidates should have hands‐on data architecture experience in one or more of the following financial ...

evolve the platform. Experience is helpful, but not essential: Languages: Python, SQL, Terragrunt, JavaScript Cloud: AWS (primary), some GCP Warehouse & Storage: Snowflake, S3/Parquet Data & ETL: dbt, Fivetran Platform & Infra: Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault Observability: Datadog, Grafana Dashboarding: Preset Other : Claude What ...

scalable environments for our data platforms. You will leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You’ll tackle high-performance data workloads, ensuring seamless execution of massive ...

these capabilities responsibly. Modern Data Platform Expertise You're familiar with contemporary data platforms and tools, including data lakes, open table formats such as Parquet and Iceberg, ETL and ELT pipelines, multiple query engines, and BI tools such as Power BI. You're comfortable working across cloud environments including ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

ability to analyse and address issues and queries quickly and effectively.* Recent hands-on experience with Python plus SQL Server and ideally some Parquet exposure.* Evidence of strong delivery throughout the entire lifecycle.* Strong problem-solving skills and capability to write code quickly and accurately, plus experience of troubleshooting ...

building agent-driven data workflows. Google Cloud; services such as Cloud Run, GKE, Artifact Registry and Cloud SQL. Modern columnar and array formats (Parquet, Arrow, Zarr) and thoughtful storage layout for large datasets. Infrastructure as Code (IaC); ideally Terraform, Docker containers. Experience working with sensitive data under GDPR, HIPAA … transfer controls and provenance tracking. Background in biology, medicine or biomedical data (genomics, transcriptomics, proteomics, EHR, clinical images). Key Skills Python, GCP, DICOM, Parquet, Image Data, Cloud, SQL, BigQuery All profiles will be reviewed against the required skills and experience. Due to the high number of applications ...

bring the right engine, from BigQuery to Dataflow to Trino, to the job. The work has two parts. Hecate enables Iceberg for the Parquet datasets that already exist across Spotify, so teams get the benefits without rewriting their data. Alongside it, we're building the foundations ...

Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability Apache Arrow and Apache Parquet for in-memory data processing and columnar storage Gradle and Cargo for builds, and GitHub for collaboration You do not need prior experience with ...

Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability Apache Arrow and Apache Parquet for in-memory data processing and columnar storage Industry-standard build tooling, including Gradle, Cargo, and GitHub. Core Responsibilities Designing and implementing query planning ...

Macquarie Group is seeking a Data Platform Engineer to design, build and optimize a petabyte-scale data platform powered by modern cloud technologies in a hybrid work model. You will manage a Data Lake, ensure ...