4 of 4 Permanent Parquet Jobs in Yorkshire

including Kafka, Azure Functions or Azure Service Bus. Strong understanding of Spark processing and performance tuning. Experience working with file formats and partitioning, including Parquet, JSON, XML and CSV. Experience with Azure DevOps/GitHub and CI/CD. Strong programming experience in Python, with knowledge of other programming ...

troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly S3, IAM and CloudWatch ...

Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape and improve the agile process Work with ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...