16 of 16 Parquet Jobs in the UK excluding London

Airflow, Step Functions, etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world ...

Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape and improve the agile process Work with ...

Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi-structured data (Parquet, JSON, CSV) Solid understanding of CI/CD and Git-based workflows Experience with data quality checks, testing and monitoring Why Apply? Genuine input ...

building lean processes Experience with distributed data processing Experience with monitoring, testing, and data quality Experience building robust pipelines from diverse sources e.g. parquet, SQL & no-SQL databases, API endpoints It would be helpful to have experience/expertise/knowledge in the following: Python AWS Kubernetes Spark & distributed ...

ways of working, and presenting these to the community Here's a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we're looking for We'd love to hear from ...

real downstream consumers. Work with economists and engineers to turn modelling decisions into reliable production data. Essential Strong production Python. Datasets in pandas and Parquet/Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster ...

governance Experience designing REST, SOAP, file, batch, legacy and bespoke integrations Knowledge of structured, semi-structured and unstructured data, including CSV, XML, JSON, Parquet and Avro Experience designing solutions across multiple security classifications Strong knowledge of data governance, lineage, metadata, data quality, access control and auditability Strong stakeholder engagement ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

Fabric-native governance and Microsoft Purview requirements where these are in project scope. Optimise, monitor and troubleshoot production workloads: Tune SQL, Spark, Delta/Parquet storage, partitioning and compute for performance, cost and reliability. Establish monitoring and alerting for pipelines, notebooks, semantic-model refreshes, data freshness and Fabric capacity … Lakehouse, Warehouse, Data Pipelines, Notebooks and Dev/Test/Production workspace patterns. Advanced SQL, Python and PySpark, with practical experience of Delta/Parquet, medallion architecture and production-grade batch/incremental ETL/ELT. Strong Git and CI/CD experience using Azure DevOps and/ ...

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...

relational database knowledge, including data extraction and quality management Experience working with Data Warehouse and ETL processes, alongside formats such as CSV, JSON and Parquet Excellent stakeholder management skills with the ability to influence decision-making at all levels of the business Successful candidates must be able to obtain ...

senior backend engineer in the UK to build scalable async REST endpoints using FastAPI and Python. You will manage large data pipelines with Pandas, Parquet/Arrow, and DuckDB, while owning identity and access via AWS IAM and OAuth2/OIDC. Work closely with data teams to design ...

ticks, Level 1 & 2 quotes, order books, reference data and corporate actions. Implement automated data quality, reconciliation and monitoring frameworks. Optimise data storage using Parquet, Arrow and modern data lake technologies. Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets. Essential Skills Strong Python and SQL. … data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome candidates ...

logic and data services. Build and optimise data pipelines on AWS to collect, transform, and serve large datasets. Work extensively with S3 (Hive-partitioned Parquet data lake), Athena for SQL querying and analysis, DynamoDB for low-latency caching, and Kinesis/SQS for streaming and queue-based integration — including … Experience building RESTful APIs and services with FastAPI.Hands-on experience with Infrastructure as code with Terraform. Containerisation with Docker. Data-engineering fundamentals: working with Parquet, Hive-partitioned datasets, and SQL against large tables (Athena/Presto). Solid working knowledge of the PyData stack — pandas and NumPy for data ...

Build and operate async REST endpoints (FastAPI, Pydantic) that serve large datasets reliably and quickly. Work with large tabular data in pandas, Parquet/Arrow and DuckDB, and manage the memory and performance trade-offs of moving big datasets efficiently. Own the identity and access layer, including AWS technologies … datastore, with least-privilege IAM. Useful Auth in production: OAuth2/OIDC, JWT validation, API keys, scopes. Tabular data at scale: pandas plus Parquet/Arrow, ideally DuckDB. Multi-tenant SaaS: RBAC, API-key management. Nice to have Infrastructure as code (CDK in TypeScript, or Terraform). React/ ...