26 to 50 of 69 Parquet Jobs

distributed systems — with at least 3 years managing engineering teams Experience and knowledge of building data-lakes in AWS (i.e. Spark, Athena, Iceberg, Parquet, Presto), including data modeling, data quality best practices, and self-service tooling. Strong expertise in building and operating real-time data at scale including Kafka ...

Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query optimization ...

Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization ...

modelling and transforming data between different formats. Knowledge of cloud platforms, ideally Azure, alongside Spark and Databricks. Familiarity with data storage technologies such as Parquet and Delta Lake. Experience with Docker, Kubernetes, Git/GitHub and Linux/Unix environments. Ability to contribute to shared codebases and collaborative development ...

Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing SparkSQL for analytics Why else you'll love it here ...

scalable environments for our data platforms. You will leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You’ll tackle high-performance data workloads, ensuring seamless execution of massive ...

definition Kubernetes for data services and task orchestration Argo CD for application deployments Airflow purely for job scheduling and tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data ...

these capabilities responsibly. Modern Data Platform Expertise You're familiar with contemporary data platforms and tools, including data lakes, open table formats such as Parquet and Iceberg, ETL and ELT pipelines, multiple query engines, and BI tools such as Power BI. You're comfortable working across cloud environments including ...

Role Description The Technical Analyst – Data Lineage will support a Data Governance, Controls, and Reporting program for a top-tier banking client. Responsible for establishing and validating end-to-end lineage across critical datasets used ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

OLAP platform would be particularly valuable, although strong candidates from SQL Server, Oracle or other database backgrounds will also be considered. Exposure to Azure, Parquet or querying data in object storage would be beneficial. What's on offer £70,000-£80,000 depending on experience. Hybrid working, with ...

experience MergeTree family design, partitioning, TTLs, materialised views, Distributed tables and sharding ETL/ELT pipeline design and operation; batch + near-real-time Parquet and object storage (Azure Blob or S3) as a query tier Query performance tuning and capacity planning Strong SQL; PostgreSQL competence Desired Criteria: Clickhouse ...

management, and de-identification processes. Google Cloud Platform expertise, including BigQuery, Cloud Run, GKE, and Cloud SQL. Experience with modern data formats such as Parquet, Arrow, and Zarr. Infrastructure as Code using Terraform and Docker. Experience working within regulated environments involving sensitive healthcare or clinical data. Experience leveraging … coding assistants and agent-driven workflows. Key Technologies Python | GCP | BigQuery | SQL | DICOM | Medical Imaging | ETL/ELT | Cloud Storage | Parquet | AI & Machine Learning Guidant, Carbon60, Lorien & SRG - The Vertage Group Portfolio are acting as an Employment Business in relation to this vacancy. ...

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...

Python (3.11+), pandas and NumPy skills. Production-grade Streamlit dashboard development. Data integration using Snowflake, OpenSearch, REST APIs and SQL. AWS experience (boto3, S3, Parquet). Data visualisation with Plotly, Matplotlib or Seaborn. Experience with Docker, Kubernetes, CI/CD and pytest. Knowledge of statistical analysis, anomaly detection ...

experience Strong Python experience (PySpark, Pandas, Jupyter Notebooks etc.) Airflow/Algo for workflow management Git Any exposure to compressed file formats, such as Parquet or HDF5, is highly advantageous Linux/Bash skills are highly desirable Job title: Data Analyst Client: Elite FinTech Salary ...

Useful: Experience engineering systems for the cloud-based storage and processing of large datasets or using frameworks such as Spark and open standards like Parquet and Arrow. Experience working with other cloud and related technologies, such as Docker and Kubernetes. Experience working as a software developer in the quantitative ...

SSIS, Azure Data Factory and related cloud services. Support API-based integrations, with Workday integration experience considered good to have. Contribute to data snapshotting, Parquet-based storage and Fabric/lakehouse development where required. Apply good engineering practices, including source control, testing, peer review, monitoring and controlled deployments. Identify ...

Requisition ID 189316 -Posted 11/06/2026 Design, develop, test, deploy, and maintain scalable ETL/ELT pipelines for structured and unstructured data. Build batch and real-time processing solutions using BigQuery, Dataflow ...

Python and best practices. The role requires strong Python skills, experience with cloud platforms (Azure), Spark/Databricks, and data storage technologies such as Parquet/Delta Lake. #J-18808-Ljbffr ...

development (multi-page, session state, query-param nav)API/SQL data integration (Snowflake, OpenSearch, REST sources)Visualization: plotly, matplotlib, seabornAWS (boto3, S3/Parquet), Docker, Kubernetes, CI/CDSecrets/auth (JWT, cryptography, proxy handling); testing with pytestData Science/AnalyticsApplied statistics for anomaly detection (rolling median + ...

what to do when performance drifts. Nice to have Experience working with scientific or scholarly text. Familiarity with AWS (S3, Batch, Lambda, SageMaker) and Parquet or Iceberg data lake patterns. Experience running LLMs under real cost and latency budgets in production. Some exposure to agentic AI applications: tool ...

that deliver value to real users. Nice to have Experience working with scientific or scholarly text. Familiarity with AWS (S3, Batch, Lambda, SageMaker) and Parquet or Iceberg data lake patterns. Experience running LLMs under real cost and latency budgets in production. Some exposure to agentic AI applications: tool ...

other genome browser Experience working in a multi‐person team delivering production services Experience with large‐scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team You bring ...

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...