26 to 50 of 69 Parquet Jobs in the UK

evolve the platform. Experience is helpful, but not essential: Languages: Python, SQL, Terragrunt, JavaScript Cloud: AWS (primary), some GCP Warehouse & Storage: Snowflake, S3/Parquet Data & ETL: dbt, Fivetran Platform & Infra: Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault Observability: Datadog, Grafana Dashboarding: Preset Other : Claude What ...

standards such as metrics and use of test tools Here’s a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT We’d love to hear from you if you have Good experience in creating ...

ways of working, and presenting these to the community Here’s a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we’re looking for We’d love to hear from ...

Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query optimization ...

Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization ...

evolve the platform. Experience is helpful, but not essential:* **Languages:** Python, SQL, Terragrunt, JavaScript* **Cloud:** AWS (primary), some GCP* **Warehouse & Storage:** Snowflake, S3/Parquet* **Data & ETL:** dbt, Fivetran* **Platform & Infra:** Kubernetes, Kafka, RabbitMQ, Argo, GitHub Actions, HashiCorp Vault* **Observability:** Datadog, Grafana* **Dashboarding:** Preset* **Other**: Claude### **What ...

Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing SparkSQL for analytics Why else you'll love it here ...

modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity or experience with classical NLP techniques (BERT, topic modelling, summarisation ...

experience with Apache Spark and Apache Flink. Strong understanding of Apache HTTP, Envoy, and REST API design. Experience working with Protobuf, Avro, and Parquet data formats. Knowledge of Iceberg, Databricks, and Hadoop ecosystems. Skills that will help Follows TDD, ATDD and Clean Coding practices and used to setting ...

scalable environments for our data platforms. You will leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You’ll tackle high-performance data workloads, ensuring seamless execution of massive ...

modelling and transforming data between different formats. Knowledge of cloud platforms, ideally Azure, alongside Spark and Databricks. Familiarity with data storage technologies such as Parquet and Delta Lake. Experience with Docker, Kubernetes, Git/GitHub and Linux/Unix environments. Ability to contribute to shared codebases and collaborative development ...

. Experience with SQL and noSQL environments, Azure SQL database and Storage Account – blob is must Familiarity with data formats such as GRIP, NETCDF, Parquet, and JSON is a plus. Knowledge of PowerShell, Git, ARM templates, Bicep, Terraform and deployment automation using Azure DevOps. Experience in using tooling such ...

modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity or experience with classical NLP techniques (BERT, topic modelling, summarisation ...

these capabilities responsibly. Modern Data Platform Expertise You're familiar with contemporary data platforms and tools, including data lakes, open table formats such as Parquet and Iceberg, ETL and ELT pipelines, multiple query engines, and BI tools such as Power BI. You're comfortable working across cloud environments including ...

Role Description The Technical Analyst – Data Lineage will support a Data Governance, Controls, and Reporting program for a top-tier banking client. Responsible for establishing and validating end-to-end lineage across critical datasets used ...

definition Kubernetes for data services and task orchestration Argo CD for application deployments Airflow purely for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data ...

experience MergeTree family design, partitioning, TTLs, materialised views, Distributed tables and sharding ETL/ELT pipeline design and operation; batch + near-real-time Parquet and object storage (Azure Blob or S3) as a query tier Query performance tuning and capacity planning Strong SQL; PostgreSQL competence Desired Criteria: Clickhouse ...

experience Strong Python experience (PySpark, Pandas, Jupyter Notebooks etc.) Airflow/Algo for workflow management Git Any exposure to compressed file formats, such as Parquet or HDF5, is highly advantageous Linux/Bash skills are highly desirable Job title: Data Analyst Client: Elite FinTech Salary ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

OLAP platform would be particularly valuable, although strong candidates from SQL Server, Oracle or other database backgrounds will also be considered. Exposure to Azure, Parquet or querying data in object storage would be beneficial. What's on offer £70,000-£80,000 depending on experience. Hybrid working, with ...

Useful: Experience engineering systems for the cloud-based storage and processing of large datasets or using frameworks such as Spark and open standards like Parquet and Arrow. Experience working with other cloud and related technologies, such as Docker and Kubernetes. Experience working as a software developer in the quantitative ...

SSIS, Azure Data Factory and related cloud services. Support API-based integrations, with Workday integration experience considered good to have. Contribute to data snapshotting, Parquet-based storage and Fabric/lakehouse development where required. Apply good engineering practices, including source control, testing, peer review, monitoring and controlled deployments. Identify ...

ability to analyse and address issues and queries quickly and effectively.* Recent hands-on experience with Python plus SQL Server and ideally some Parquet exposure.* Evidence of strong delivery throughout the entire lifecycle.* Strong problem-solving skills and capability to write code quickly and accurately, plus experience of troubleshooting ...

Role Description The Business Analyst/Project Manager will support a transformation program for a top-tier banking client, aligned to an ECB Remediation approach. The Technical Analyst will support a Data Governance & Controls remediation ...

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...