23 of 23 Parquet Jobs in the UK excluding London

Airflow, Step Functions, etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. #J-18808-Ljbffr ...

Airflow, Step Functions, etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats. J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world ...

data integration components and services, including Java-based microservices where required Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse ...

Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape and improve the agile process Work with ...

patterns (Debezium or equivalent) Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation at scale Data and table formats (Parquet, Avro, Apache Iceberg or Delta Lake) Experience designing SaaS data models, tenant isolation, shared schemas, access patterns, at scale. Ability to communicate architecture decisions ...

Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi-structured data (Parquet, JSON, CSV) Solid understanding of CI/CD and Git-based workflows Experience with data quality checks, testing and monitoring Why Apply? Genuine input ...

Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization ...

handling Delta Lake formats Hands-on experience building production pipelines inside Databricks workspaces and Fabric capacities Practical knowledge of structural data formats including Parquet, JSON, and CSV Experience with orchestration tools, scheduling dependencies, and error handling in a cloud environment Familiarity with unit testing data pipelines and implementing automated ...

production environments Develop ingestion and egress patterns to support reliable data access and integration Work with structured and semi-structured data (e.g. JSON, Parquet) Build and maintain data models across a cloud data platform Design and implement APIs to expose data for downstream consumers Contribute to cloud-based data ...

building lean processes Experience with distributed data processing Experience with monitoring, testing, and data quality Experience building robust pipelines from diverse sources e.g. parquet, SQL & no-SQL databases, API endpoints It would be helpful to have experience/expertise/knowledge in the following: Python AWS Kubernetes Spark & distributed ...

ways of working, and presenting these to the community Here's a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we're looking for We'd love to hear from ...

ways of working, and presenting these to the community Here’s a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we’re looking for We’d love to hear from ...

Glue, Step Functions, Athena, S3 and serverless architectures Data modelling, transformation and orchestration tooling such as dbt Working with modern analytical storage formats including Parquet and Iceberg Building robust data ingestion and transformation workflows Data quality frameworks, schema evolution and metadata management Experience with distributed data processing and large ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

governance Experience designing REST, SOAP, file, batch, legacy and bespoke integrations Knowledge of structured, semi-structured and unstructured data, including CSV, XML, JSON, Parquet and Avro Experience designing solutions across multiple security classifications Strong knowledge of data governance, lineage, metadata, data quality, access control and auditability Strong stakeholder engagement ...

real downstream consumers. Work with economists and engineers to turn modelling decisions into reliable production data. Essential Strong production Python. Datasets in pandas and Parquet/Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster ...

Fabric-native governance and Microsoft Purview requirements where these are in project scope. Optimise, monitor and troubleshoot production workloads: Tune SQL, Spark, Delta/Parquet storage, partitioning and compute for performance, cost and reliability. Establish monitoring and alerting for pipelines, notebooks, semantic-model refreshes, data freshness and Fabric capacity … Lakehouse, Warehouse, Data Pipelines, Notebooks and Dev/Test/Production workspace patterns. Advanced SQL, Python and PySpark, with practical experience of Delta/Parquet, medallion architecture and production-grade batch/incremental ETL/ELT. Strong Git and CI/CD experience using Azure DevOps and/ ...

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...

event-driven architectures or data pipelines. Also Welcome Kubernetes. Deploying, running or debugging workloads, including K3s, RKE2 or OpenShift. Data at scale. Spark, Trino, Parquet, Iceberg or similar columnar and distributed query tooling. GPU workloads and serving models on-premise. Open source contributions or a portfolio worth a look. ...

Fabric-native governance and Microsoft Purview requirements where these are in project scope. Optimise, monitor and troubleshoot production workloads: Tune SQL, Spark, Delta/Parquet storage, partitioning and compute for performance, cost and reliability. Establish monitoring and alerting for pipelines, notebooks, semantic-model refreshes, data freshness and Fabric capacity … Lakehouse, Warehouse, Data Pipelines, Notebooks and Dev/Test/Production workspace patterns. Advanced SQL, Python and PySpark, with practical experience of Delta/Parquet, medallion architecture and production-grade batch/incremental ETL/ELT. Strong Git and CI/CD experience using Azure DevOps and/ ...

relational database knowledge, including data extraction and quality management Experience working with Data Warehouse and ETL processes, alongside formats such as CSV, JSON and Parquet Excellent stakeholder management skills with the ability to influence decision-making at all levels of the business Successful candidates must be able to obtain ...

logic and data services. Build and optimise data pipelines on AWS to collect, transform, and serve large datasets. Work extensively with S3 (Hive-partitioned Parquet data lake), Athena for SQL querying and analysis, DynamoDB for low-latency caching, and Kinesis/SQS for streaming and queue-based integration — including … Experience building RESTful APIs and services with FastAPI.Hands-on experience with Infrastructure as code with Terraform. Containerisation with Docker. Data-engineering fundamentals: working with Parquet, Hive-partitioned datasets, and SQL against large tables (Athena/Presto). Solid working knowledge of the PyData stack — pandas and NumPy for data ...