1 to 25 of 67 Permanent Parquet Jobs

data integration components and services, including Java-based microservices where required Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse ...

storage and exchange formats for each workload, including row‐based and columnar storage, Delta and Iceberg table formats, and file and message formats (e.g. Parquet, CSV, Excel, XML, JSON, Protobuf). Optimise platform performance and cost, including partitioning vs. liquid clustering, file sizing, Photon, and cluster and SQL warehouse ...

data integration components and services, including Java-based microservices where required Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse ...

machine learning features and predictive modellingConsume data from a variety of sources (RDBMS, APIs, FTPs and other cloud storage) & formats (Excel, CSV, XML, JSON, Parquet, Unstructured)Write advanced/complex SQL with performance tuning and optimizationBuild AI based solutions for solving business needs, process automations and improving operational efficiencyIdentify ...

best practices for structuring data for analytics, reporting and downstream consumption Confident working with large, complex datasets across various formats (databases, Excel, CSV, JSON, Parquet, APIs), while optimising performance Experience integrating data from social platforms such as YouTube and Meta, or working with advertising, audience, performance and revenue data ...

similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON and Parquet Containerisation DevOps and DataOps We are interested in Engineers with a solid understanding of DevOps to enable ownership of cloud infrastructure and different environments ...

similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON and Parquet Containerisation DevOps and DataOps We are interested in Engineers with a solid understanding of DevOps to enable ownership of cloud infrastructure and different environments ...

similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON and Parquet Containerisation DevOps and DataOps We are interested in Engineers with a solid understanding of DevOps to enable ownership of cloud infrastructure and different environments ...

with Azure data services (Azure Data Factory, ADLS Gen2, Azure SQL, Synapse or Databricks)Solid understanding of data warehousing and lakehouse concepts, Delta/Parquet formats and dimensional modeling (Kimball)Experience with Real-Time Intelligence (Eventstreams, Eventhouse/KQL, Activator)Ability to turn architecture documents and requirements into working ...

Glue Database Technologies Data processing libraries & Schema Management tools like Pandas and Pydantic Storage Formats: Familiarity with data types such as JSON and Parquet DevOps and DataOps Experience of IaC tools, particularly Terraform Understanding of implementing logging, monitoring and alerting for enterprise level pipelines Understanding of access controls, policies ...

Glue Database Technologies Data processing libraries & Schema Management tools like Pandas and Pydantic Storage Formats: Familiarity with data types such as JSON and Parquet DevOps and DataOps Experience of IaC tools, particularly Terraform Understanding of implementing logging, monitoring and alerting for enterprise level pipelines Understanding of access controls, policies ...

Glue Database Technologies Data processing libraries & Schema Management tools like Pandas and Pydantic Storage Formats: Familiarity with data types such as JSON and Parquet DevOps and DataOps Experience of IaC tools, particularly Terraform Understanding of implementing logging, monitoring and alerting for enterprise level pipelines Understanding of access controls, policies ...

paper Own Data Storage & Performance Work across relational, NoSQL, and analytical stores (Postgres, BigQuery, Snowflake, Cassandra, MongoDB) Optimise storage formats and access patterns (Parquet, Delta, ORC, Avro) Cloud, Security & Governance Implement secure, compliant data solutions with security by design Embed governance without killing developer velocity Consult and Influence Work ...

Data Build Tool (DBT) with a demonstrated ability indeveloping data models, contracts, tests, validation, and transformations Experienceworking with modern data distributed file formats (i.e., Parquet, Delta,Iceberg, Hudi) Demonstratedexperience with building data ingestion pipelines from REST API data sources. Strongability to produce technical documentation that can be understood ...

Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape and improve the agile process Work with ...

patterns (Debezium or equivalent) Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation at scale Data and table formats (Parquet, Avro, Apache Iceberg or Delta Lake) Experience designing SaaS data models, tenant isolation, shared schemas, access patterns, at scale. Ability to communicate architecture decisions ...

supporting non‐technical stakeholders in getting the most out of the data warehouse Experience ingesting and processing structured and unstructured files (CSV, JSON, XML, Parquet, PDF) from a variety of sources, including cloud storage buckets and third‐party exports. Desired but not essential skills and experience include Broader Google ...

building lean processes Experience with distributed data processing Experience with monitoring, testing, and data quality Experience building robust pipelines from diverse sources e.g. parquet, SQL & no-SQL databases, API endpoints It would be helpful to have experience/expertise/knowledge in the following: Python AWS Kubernetes Spark & distributed ...

integration.Exposure to Market Data systems, financial data models, or vendor contract management domains.Experience with high-performance analytics engines or in-memory data processing (DuckDB, Parquet, Kafka).Education & ExperienceBachelor’s/Master’s degree in Computer Science, Software Engineering, or equivalent practical experience.Track record of technical delivery and leadership ...

reliability of the data provided by the Utility. ResponsibilitiesDesign and develop scalable data pipelines using Python to read from APIs and structured files (Excel, Parquet). Translate domain and source specific business logic into efficient code implementations to turn source data into usable structured data for downstream applications. Implement ...

evolve the platform. Experience is helpful, but not essential:* **Languages:** Python, SQL, Terragrunt, JavaScript* **Cloud:** AWS (primary), some GCP* **Warehouse & Storage:** Snowflake, S3/Parquet* **Data & ETL:** dbt, Fivetran* **Platform & Infra:** Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault* **Observability:** Datadog, Grafana* **Dashboarding:** Preset* **Other**: Claude### **What ...

production environments Develop ingestion and egress patterns to support reliable data access and integration Work with structured and semi-structured data (e.g. JSON, Parquet) Build and maintain data models across a cloud data platform Design and implement APIs to expose data for downstream consumers Contribute to cloud-based data ...

Modern Tech Stack: Experience is helpful, but not essential: Languages: Python, SQL, Terragrunt, JavaScript Cloud: AWS (primary), some GCP Warehouse & Storage: Snowflake, S3/Parquet Data & ETL: dbt, Fivetran Platform & Infra: Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault Observability: Datadog, Grafana Dashboarding: Preset Other: Claude WHAT ...

modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity or experience with classical NLP techniques (BERT, topic modelling, summarisation ...

distributed systems — with at least 3 years managing engineering teams Experience and knowledge of building data-lakes in AWS (i.e. Spark, Athena, Iceberg, Parquet, Presto), including data modeling, data quality best practices, and self-service tooling. Strong expertise in building and operating real-time data at scale including Kafka ...