1 to 25 of 41 Parquet Jobs in London

dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Exposure to data governance concepts: critical data elements (CDEs), lineage, data quality dimensions, audit frameworks. Experience running delivery (intake, triage, prioritization ...

dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Hands-on exposure to DCRM tooling and operational exception management processes. Experience with governance/catalog tools and lineage documentation methods ...

similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON and Parquet Containerisation DevOps and DataOps We are interested in Engineers with a solid understanding of DevOps to enable ownership of cloud infrastructure and different environments ...

Data Build Tool (DBT) with a demonstrated ability indeveloping data models, contracts, tests, validation, and transformations Experienceworking with modern data distributed file formats (i.e., Parquet, Delta,Iceberg, Hudi) Demonstratedexperience with building data ingestion pipelines from REST API data sources. Strongability to produce technical documentation that can be understood ...

with Azure data services (Azure Data Factory, ADLS Gen2, Azure SQL, Synapse or Databricks)Solid understanding of data warehousing and lakehouse concepts, Delta/Parquet formats and dimensional modeling (Kimball)Experience with Real-Time Intelligence (Eventstreams, Eventhouse/KQL, Activator)Ability to turn architecture documents and requirements into working ...

based environment Advanced experience working and understanding the tradeoffs of at least one of the following Data Lake table/file formats: Delta Lake, Parquet, Iceberg, Hudi Experience working with lakehouse/medallion architectures in Databricks Experience with experimentation support tooling (such as Optimizely) A genuine curiosity about ...

environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with a modern and evolving data stack. Candidates are not expected to have deep expertise ...

Python for data engineering and automation. Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet . Advanced SQL skills; experience with Oracle databases . Experience designing and supporting ETL/ELT pipelines and data quality frameworks . Working knowledge ...

supporting non‐technical stakeholders in getting the most out of the data warehouse Experience ingesting and processing structured and unstructured files (CSV, JSON, XML, Parquet, PDF) from a variety of sources, including cloud storage buckets and third‐party exports. Desired but not essential skills and experience include Broader Google ...

invalidation, persistence strategy, data consistency MongoDB TimescaleDB NATS JetStream AWS S3/Glue Data lake staging, archival, batch ingestion into Snowflake Partitioning, file format (Parquet/ORC), Lake Formation governance Finance Domain Knowledge Candidates should have hands‐on data architecture experience in one or more of the following financial ...

building lean processes Experience with distributed data processing Experience with monitoring, testing, and data quality Experience building robust pipelines from diverse sources e.g. parquet, SQL & no-SQL databases, API endpoints It would be helpful to have experience/expertise/knowledge in the following: Python AWS Kubernetes Spark & distributed ...

evolve the platform. Experience is helpful, but not essential: Languages: Python, SQL, Terragrunt, JavaScript Cloud: AWS (primary), some GCP Warehouse & Storage: Snowflake, S3/Parquet Data & ETL: dbt, Fivetran Platform & Infra: Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault Observability: Datadog, Grafana Dashboarding: Preset Other : Claude What ...

evolve the platform. Experience is helpful, but not essential:* **Languages:** Python, SQL, Terragrunt, JavaScript* **Cloud:** AWS (primary), some GCP* **Warehouse & Storage:** Snowflake, S3/Parquet* **Data & ETL:** dbt, Fivetran* **Platform & Infra:** Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault* **Observability:** Datadog, Grafana* **Dashboarding:** Preset* **Other**: Claude### **What ...

Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing SparkSQL for analytics Why else you'll love it here ...

modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity or experience with classical NLP techniques (BERT, topic modelling, summarisation ...

experience with Apache Spark and Apache Flink. Strong understanding of Apache HTTP, Envoy, and REST API design. Experience working with Protobuf, Avro, and Parquet data formats. Knowledge of Iceberg, Databricks, and Hadoop ecosystems. Skills that will help Follows TDD, ATDD and Clean Coding practices and used to setting ...

scalable environments for our data platforms. You will leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You’ll tackle high-performance data workloads, ensuring seamless execution of massive ...

modelling and transforming data between different formats. Knowledge of cloud platforms, ideally Azure, alongside Spark and Databricks. Familiarity with data storage technologies such as Parquet and Delta Lake. Experience with Docker, Kubernetes, Git/GitHub and Linux/Unix environments. Ability to contribute to shared codebases and collaborative development ...

modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity or experience with classical NLP techniques (BERT, topic modelling, summarisation ...

these capabilities responsibly. Modern Data Platform Expertise You're familiar with contemporary data platforms and tools, including data lakes, open table formats such as Parquet and Iceberg, ETL and ELT pipelines, multiple query engines, and BI tools such as Power BI. You're comfortable working across cloud environments including ...

Role Description The Technical Analyst – Data Lineage will support a Data Governance, Controls, and Reporting program for a top-tier banking client. Responsible for establishing and validating end-to-end lineage across critical datasets used ...

definition Kubernetes for data services and task orchestration Argo CD for application deployments Airflow purely for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data ...

experience Strong Python experience (PySpark, Pandas, Jupyter Notebooks etc.) Airflow/Algo for workflow management Git Any exposure to compressed file formats, such as Parquet or HDF5, is highly advantageous Linux/Bash skills are highly desirable Job title: Data Analyst Client: Elite FinTech Salary ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

Useful: Experience engineering systems for the cloud-based storage and processing of large datasets or using frameworks such as Spark and open standards like Parquet and Arrow. Experience working with other cloud and related technologies, such as Docker and Kubernetes. Experience working as a software developer in the quantitative ...