11 of 11 Parquet Jobs in the UK

data collection Solid understanding of data modelling best practices for analytics and dashboards Confidence working with large, complex datasets across multiple formats (CSV, JSON, Parquet, databases, APIs) Strong problem-solving skills and the ability to diagnose and resolve data issues Excellent communication skills and experience working with cross-functional ...

Technical Profile Python Mastery: Expert-level Python 3.9+ (Design Patterns, OOP, logical data modelling, pandas, SQLAlchemy). Data Engineering: High-scale workload experience using Parquet or Arrow. Database Expert: Mastery of SQL Server or PostgreSQL (tuning, joins, transactions, and complex aggregations). Cloud & Infrastructure: Hands-on experience with ...

data pipelines from SAP and non-SAP sources into cloud platforms such as Snowflake, Databricks, and Fabric. Design medallion and lakehouse architectures using Parquet, Iceberg, Delta, and cloud-native services . Establish CI/CD frameworks, automation, metadata-driven ingestion, and orchestration strategies. Governance, Security & Quality Define data governance ...

PowerShell scripting o Jira, Confluence, ServiceNow o Azure Log Analytics, Workbooks, App Insights Additional Skills Familiarity with data formats such as GRIB, NETCDF, Parquet, JSON is a plus. Strong communication, analytical thinking, and problem-solving skills. Experience supporting production applications. Ability to anticipate problems and propose robust solutions. Azure ...

implement production-grade REST APIs using Python (FastAPI) to manage complex financial data submissions. ● Design secure ingestion pipelines for large-scale datasets (CSV/Parquet), leveraging S3, presigned URLs, automated webhooks, and high-performance data libraries such as Polars or Pandas. ● Develop sophisticated backend logic for consensus workflows, regional ...

ingest cost, reducing low-value telemetry, retention/lifecycle strategy security lake/long retention approaches (e.g., Amazon Security Lake (OCSF) + S3/Parquet; lakehouse stacks like Databricks/Snowflake/Trino/Athena) open detection layer awareness (e.g., OpenSearch Security Analytics; Splunk ES/ESCU where relevant ...

stack. AI Integration: Leverage Claude and LLMs for signal extraction, code optimization, and autonomous strategy refinement. High-Performance Computing: Implement predicate pushdown and efficient Parquet/Arrow storage to ensure backtests on decades of data run in minutes, not days. Options Logic: Programmatically handle Greeks, volatility surfaces, and risk ...

normalise high-velocity time-series datasets from multiple external sources ⚙️ Work heavily in Python with a modern stack including DuckDB and Polars (plus Parquet/PyArrow ) 🧩 Orchestrate workflows and improve reliability (they use Temporal — similar orchestration experience is fine) ✅ Improve data integrity and visibility: validations, automated checks, backfills, monitoring … client-facing outputs (dashboards/PDF/Plotly — least important) What’s in it for you? 📌 Modern data stack – DuckDB/Polars + Parquet/Arrow in a genuinely hands-on environment 📈 Ownership & impact – You’ll be close to the data flows and have real influence on performance ...

/low level performance Skills: 2 - 15 years of experience as a C++ Developer Strong knowledge of optimization/performance Knowledge of Arrow/Parquet is a plus Experience working with Boost ASIO is a plus An interest in Rust is a nice plus Experience working at a large ...

Apache Iceberg . Simplify existing data architecture to improve maintainability and reduce operational overhead. Assess and advise on converting Iceberg datasets into optimised Parquet formats. Implement solutions to improve pipeline stability and reduce AWS operational costs. Document existing architecture, identify technical debt, and create remediation strategies. Work closely with … Data Catalog) Amazon Athena (query tuning, partitioning strategies) AWS Lambda (event driven workloads) Apache Iceberg (schema evolution, table formats, time travel) Deep understanding of Parquet, Iceberg, ORC and performance trade offs. Proven track record of diagnosing and resolving complex pipeline performance issues. Strong SQL skills and experience with query ...

automated test suites Writing and maintaining: API tests Integration tests End-to-end workflow tests Validating: Data ingestion flows File-based workflows (CSV/Parquet) Region-specific and customer-isolated data paths Testing error cases, edge cases, and failure modes Ensuring backward compatibility of APIs and schemas Tooling & Process … Client's Tech Stack (Context) Backend: Python 3.11+, FastAPI Frontend: TypeScript, React Infra: AWS (ECS/Fargate, S3, RDS, CloudFront) Data: PostgreSQL, CSV/Parquet CI/CD: GitLab CI IaC: Terraform Testing: Pytest, Postman/Newman, custom scripts You don’t need to know everything — but you should ...