5 of 5 Permanent Parquet Jobs in the North of England

Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi-structured data (Parquet, JSON, CSV) Solid understanding of CI/CD and Git-based workflows Experience with data quality checks, testing and monitoring WhyApply? Genuine input into ...

Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

backend logic and data services.Build and optimise data pipelines on AWS to collect, transform, and serve large datasets. Work extensively with S3 (Hive-partitioned Parquet data lake), Athena for SQL querying and analysis, DynamoDB for low-latency caching, and Kinesis/SQS for streaming and queue-based integration — including … CloudWatch.Experience building RESTful APIs and services with FastAPI.Hands-on experience with Infrastructure as code with Terraform.Containerisation with Docker.Data-engineering fundamentals: working with Parquet, Hive-partitioned datasets, and SQL against large tables (Athena/Presto). Solid working knowledge of the PyData stack — pandas and NumPy for data manipulation, Jupyter ...

logic and data services. Build and optimise data pipelines on AWS to collect, transform, and serve large datasets. Work extensively with S3 (Hive-partitioned Parquet data lake), Athena for SQL querying and analysis, DynamoDB for low-latency caching, and Kinesis/SQS for streaming and queue-based integration — including … Experience building RESTful APIs and services with FastAPI.Hands-on experience with Infrastructure as code with Terraform. Containerisation with Docker. Data-engineering fundamentals: working with Parquet, Hive-partitioned datasets, and SQL against large tables (Athena/Presto). Solid working knowledge of the PyData stack — pandas and NumPy for data ...