26 to 50 of 50 Parquet Jobs in the UK

real downstream consumers. Work with economists and engineers to turn modelling decisions into reliable production data. Essential Strong production Python. Datasets in pandas and Parquet/Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster ...

SOAP services, file transfers, batch ingestion, legacy adapters and bespoke interfaces. Experience with structured, semi-structured and unstructured data formats including CSV, XML, JSON, Parquet, Avro and proprietary formats. Experience designing solutions that operate across multiple security classifications and secure domains. Strong knowledge of data governance, lineage, metadata management ...

governance Experience designing REST, SOAP, file, batch, legacy and bespoke integrations Knowledge of structured, semi-structured and unstructured data, including CSV, XML, JSON, Parquet and Avro Experience designing solutions across multiple security classifications Strong knowledge of data governance, lineage, metadata, data quality, access control and auditability Strong stakeholder engagement ...

work effectively with remote teams"**Experience with the Following is Desirable:*** Go* Pinot or other time-series/OLAP-style database* Iceberg* Kubernetes* Jenkins* Parquet* Protocol Buffers/GRPC#LI-MZ1**Benefits of Working at CrowdStrike:*** Market leader in compensation and equity awards* Comprehensive physical and mental wellness programs* Competitive ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

tool, and says so ● Python (or R) pandas for cleaning, reshaping and joining data; numpy for the numeric work Reads and writes CSV, Excel, Parquet and JSON, and can pull data from a REST API Works comfortably in notebooks, but can also write a plain .py script that runs ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

event-driven architectures or data pipelines. Also Welcome Kubernetes. Deploying, running or debugging workloads, including K3s, RKE2 or OpenShift. Data at scale. Spark, Trino, Parquet, Iceberg or similar columnar and distributed query tooling. GPU workloads and serving models on-premise. Open source contributions or a portfolio worth a look. ...

desks and ability to analyse and address issues and queries quickly and effectively.Recent hands-on experience with Python plus SQL Server and ideally some Parquet exposure.Evidence of strong delivery throughout the entire lifecycle.Strong problem-solving skills and capability to write code quickly and accurately, plus experience of troubleshooting production ...

The OpportunityJoin a team building the data foundations that support the firm’s AI and analytics capabilities. This role sits within the engineering effort to develop a modern Lakehouse and AI data platform that enables ...

Job TitleSenior Data EngineerJob Description SummarySenior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer ...

learn new technologies and frameworksNice to have:Knowledge of databases, SQLFamiliarity with Boost ASIOFamiliarity with data serialization formats such Apache Arrow/Parquet, Google Protocol Buffers, FlatbuffersExperience with gRPC, http/REST and Websocket protocolsExperience with Google Cloud/AWS and/or containerization in Docker/KubernetesExperience with ...

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...

C# or C++Experience optimising software performance or efficiencyDesirable:Experience with distributed software systemsExperience with data processing or data serving systemsExperience using Rust, C#, Python, Parquet, Arrow, Delta Lake or KubernetesWhy join us Highly competitive compensation plus annual discretionary bonusLunch provided (via Just Eat for Business) and dedicated barista bar35 ...

data-engineering experience: schema design, storage formats, compression, I/O trade-offs, and pipelines processing hundreds of terabytes.Experience with columnar formats such as Parquet or Arrow, or comparable domain-specific formats used for event logs, telemetry, or replays.Experience designing and operating services or platforms used by other technical ...

engineering experience: schema design, storage formats, compression, I/O trade-offs, and pipelines processing hundreds of terabytes. Experience with columnar formats such as Parquet or Arrow, or comparable domain‐specific formats used for event logs, telemetry, or replays. Experience designing and operating services or platforms used by other ...

The OpportunityJoin a team building the data foundations that support the firm's AI and analytics capabilities. This role sits within the engineering effort to develop a modern Lakehouse and AI data platform that enables ...

What we are looking forState Street Alpha Data Implementation organization is seeking individuals with subject matter expertise spanning a wide array of financial instrument types and working knowledge of Asset Management Data Domains to join ...

ticks, Level 1 & 2 quotes, order books, reference data and corporate actions. Implement automated data quality, reconciliation and monitoring frameworks. Optimise data storage using Parquet, Arrow and modern data lake technologies. Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets. Essential Skills Strong Python and SQL. … data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome candidates ...

backend logic and data services.Build and optimise data pipelines on AWS to collect, transform, and serve large datasets. Work extensively with S3 (Hive-partitioned Parquet data lake), Athena for SQL querying and analysis, DynamoDB for low-latency caching, and Kinesis/SQS for streaming and queue-based integration — including … CloudWatch.Experience building RESTful APIs and services with FastAPI.Hands-on experience with Infrastructure as code with Terraform.Containerisation with Docker.Data-engineering fundamentals: working with Parquet, Hive-partitioned datasets, and SQL against large tables (Athena/Presto). Solid working knowledge of the PyData stack — pandas and NumPy for data manipulation, Jupyter ...

logic and data services. Build and optimise data pipelines on AWS to collect, transform, and serve large datasets. Work extensively with S3 (Hive-partitioned Parquet data lake), Athena for SQL querying and analysis, DynamoDB for low-latency caching, and Kinesis/SQS for streaming and queue-based integration — including … Experience building RESTful APIs and services with FastAPI.Hands-on experience with Infrastructure as code with Terraform. Containerisation with Docker. Data-engineering fundamentals: working with Parquet, Hive-partitioned datasets, and SQL against large tables (Athena/Presto). Solid working knowledge of the PyData stack — pandas and NumPy for data ...

ensuring FDJ United remains a leader in the data-driven online gambling industry. Your work will involve shaping lakehouse and data-lake paradigms, leveraging Parquet and Iceberg for high-performance data storage, and adopting scalable Spark frameworks for large-scale processing. Ultimately, you will enable self-service analytics, personalised … compliance requirements. Technical Mastery & Hands‐On Skills: Advanced knowledge of Java/Spring, Python, Docker, Kubernetes, AWS and DevOps methodologies. Familiarity with Spark, Iceberg, Parquet, and other solutions for large‐scale data processing. AI/ML & Analytics Background: Competent in ML & analytics, with exposure to LLMOps and MLOps practices ...

At Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.What unites ...

At Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market. What ...