51 to 69 of 69 Parquet Jobs in the UK

engineering experience: schema design, storage formats, compression, I/O trade-offs, and pipelines processing hundreds of terabytes. Experience with columnar formats such as Parquet or Arrow, or comparable domain‐specific formats used for event logs, telemetry, or replays. Experience designing and operating services or platforms used by other ...

data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage. Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose. Develop … routing, transformation and orchestration. Experience of data modelling and schema design, including defining data contracts or Interface Control Documents (ICDs). Experience working with Parquet and S3-compatible object storage. Experience integrating REST APIs and troubleshooting data exchanges between systems. Experience deploying or supporting data solutions within OpenShift ...

building agent-driven data workflows. Google Cloud; services such as Cloud Run, GKE, Artifact Registry and Cloud SQL. Modern columnar and array formats (Parquet, Arrow, Zarr) and thoughtful storage layout for large datasets. Infrastructure as Code (IaC); ideally Terraform, Docker containers. Experience working with sensitive data under GDPR, HIPAA … transfer controls and provenance tracking. Background in biology, medicine or biomedical data (genomics, transcriptomics, proteomics, EHR, clinical images). Key Skills Python, GCP, DICOM, Parquet, Image Data, Cloud, SQL, BigQuery All profiles will be reviewed against the required skills and experience. Due to the high number of applications ...

Python and best practices. The role requires strong Python skills, experience with cloud platforms (Azure), Spark/Databricks, and data storage technologies such as Parquet/Delta Lake. #J-18808-Ljbffr ...

what to do when performance drifts. Nice to have Experience working with scientific or scholarly text. Familiarity with AWS (S3, Batch, Lambda, SageMaker) and Parquet or Iceberg data lake patterns. Experience running LLMs under real cost and latency budgets in production. Some exposure to agentic AI applications: tool ...

that deliver value to real users. Nice to have Experience working with scientific or scholarly text. Familiarity with AWS (S3, Batch, Lambda, SageMaker) and Parquet or Iceberg data lake patterns. Experience running LLMs under real cost and latency budgets in production. Some exposure to agentic AI applications: tool ...

bring the right engine, from BigQuery to Dataflow to Trino, to the job. The work has two parts. Hecate enables Iceberg for the Parquet datasets that already exist across Spotify, so teams get the benefits without rewriting their data. Alongside it, we're building the foundations ...

Backend: Distributed, event-driven core Java (90% of the code-base), MySQL, Kafka Frontend: Javascript, Vue.js Deployment: Kubernetes, Nomad Data analytics: Python & Jupyter notebooks, Parquet, Docker Testing: JUnit, JMH, JCStress, Jenkins, Selenium, many in-house tools... OS: Linux (Fedora for development, Rocky in production) The LMAX ...

Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability Apache Arrow and Apache Parquet for in-memory data processing and columnar storage Gradle and Cargo for builds, and GitHub for collaboration You do not need prior experience with ...

Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability Apache Arrow and Apache Parquet for in-memory data processing and columnar storage Industry-standard build tooling, including Gradle, Cargo, and GitHub. Core Responsibilities Designing and implementing query planning ...

high-quality production code Strong foundation in operating systems and systems programming Proven track record in performance engineering Familiarity with tools like Apache Arrow, Parquet, DataFusion, Clickhouse, or DuckDB is a plus Understanding of cutting-edge ML infrastructure stack (e.g. PyTorch, CUDA) is also a plus Experience with Rust ...

data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage. Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose. Develop … Strong Python and SQL skills. Hands-on experience with Apache NiFi and data pipeline orchestration. Experience with data modelling and schema design. Experience with Parquet and S3-compatible object storage. Experience with REST API integration. Experience with OpenShift or Kubernetes. Experience with GitLab CI/CD practices and pipelines. ...

At Movar, we understand that project delivery is getting increasingly complex. Since 2013, we’ve been helping companies of all sizes improve the way projects are delivered. Our mission is to be the number one ...

/platform engineer to join our Engineering team in Belfast. The role focuses on expanding our data platform with ClickHouse and PostgreSQL while leveraging Parquet and cloud object storage. You will participate in a dynamic development environment, influence technical decisions, and help deliver secure, high-quality software across ...

CommunicationsTypePermanentSalaryAnnualSenior Data Engineer - Remote(Senior Data Engineer, Data Engineer, SQL, T-SQL, SQL Server, Azure SQL, Python, PySpark, Microsoft Fabric, Databricks, Synapse, Delta/Parquet, Power BI, Agentic AI, Infrastructure as code, IaC, CI/CD, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Data Engineer … modelling experience and strong Python (PySpark, notebooks). Exposure to Microsoft Fabric is required, or at least Databricks/Synapse and the Delta/Parquet ecosystem, as well as Power BI semantic modelling. You will also need an understanding of information security and data protection in an engineering context ...

Senior Data Engineer – Remote (Senior Data Engineer, Data Engineer, SQL, T-SQL, SQL Server, Azure SQL, Python, PySpark, Microsoft Fabric, Databricks, Synapse, Delta/Parquet, Power BI, Agentic AI, Infrastructure as code, IaC, CI/CD, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Data Engineer … modelling experience and strong Python (PySpark, notebooks). Exposure to Microsoft Fabric is required, or at least Databricks/Synapse and the Delta/Parquet ecosystem, as well as Power BI semantic modelling. You will also need an understanding of information security and data protection in an engineering context ...

capabilities across large tables and diverse workloads. You will work with Java, Rust, Scala, or C++ and engage with Spark, DataFusion, Iceberg, Arrow, and Parquet in a high-performance environment that values #J-18808-Ljbffr ...

At Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market. What ...

Macquarie Group is seeking a Data Platform Engineer to design, build and optimize a petabyte-scale data platform powered by modern cloud technologies in a hybrid work model. You will manage a Data Lake, ensure ...