20 of 20 Permanent Parquet Jobs

lake storage layer. Experience in building ETL/ELT pipelines for batch and near-real-time data processing. Experience with data formats such as Parquet, Avro, ORC, JSON, and CSV. Good understanding of data modelling, data warehousing, and dimensional modelling concepts. Experience with data validation, reconciliation, data quality, audit ...

Secrets Manager Data Modelling Skills Star Schema Snowflake Schema Dimensional Modeling Data Vault Data Warehouse Design Data Lake Architecture Data Formats Experience working with: Parquet Avro JSON CSV ORC CI/CD & DevOps Git GitHub GitLab Bitbucket Jenkins GitHub Actions GitLab CI/CD AWS DevOps Tools Domain Experience ...

Functions. Proficiency in Unix scripting, data structures, and data serialization formats like JSON, AVRO, or Protobuf. Knowledge of big-data storage formats such as Parquet or Iceberg. Familiarity with data processing methodologies like batch, micro-batching, or streaming. Experience with data modeling techniques such as Dimensional, Data Vault, Kimball ...

environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with a modern and evolving data stack. Candidates are not expected to have deep expertise ...

environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with a modern and evolving data stack. Candidates are not expected to have deep expertise ...

patterns (Debezium or equivalent) Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation at scale Data and table formats (Parquet, Avro, Apache Iceberg or Delta Lake) Experience designing SaaS data models, tenant isolation, shared schemas, access patterns, at scale. Ability to communicate architecture decisions ...

invalidation, persistence strategy, data consistency MongoDB TimescaleDB NATS JetStream AWS S3/Glue Data lake staging, archival, batch ingestion into Snowflake Partitioning, file format (Parquet/ORC), Lake Formation governance Finance Domain Knowledge Candidates should have hands‐on data architecture experience in one or more of the following financial ...

similar) -Proficiency with relational and NoSQL databases. - Proficiency in data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), and big-data storage formats (Parquet, Iceberg, or similar), -Experience working in teams following Agile methodology - Experience with test-driven development (TDD) or behavior-driven development (BDD) practices, as well ...

standards such as metrics and use of test tools Here's a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT We'd love to hear from you if you have: Good experience in creating ...

tuning, prompt engineering, LLM evaluation, or model serving Technical stack: strong Python across production services and data pipelines, data engineering fundamentals (Spark, Delta tables, Parquet) Platform and infrastructure: Databricks (MLflow, Unity Catalog, Model Serving), AWS (EKS/Kubernetes, IAM), Terraform, GitHub Actions Cross-domain flexibility: comfort working across ...

data - grounded in a desire to genuinely understand the data‐generating processes and domain context. Comfortable working across common data formats (e.g. CSV, JSON, Parquet) - loading, inspecting, joining, and aggregating as needed - and understanding what it means for the work when data is slowly changing versus arriving in near ...

work effectively with remote teams"**Experience with the Following is Desirable:*** Go* Pinot or other time-series/OLAP-style database* Iceberg* Kubernetes* Jenkins* Parquet* Protocol Buffers/GRPC#LI-MZ1**Benefits of Working at CrowdStrike:*** Market leader in compensation and equity awards* Comprehensive physical and mental wellness programs* Competitive ...

/Pyspark 3/4 Python behave - for Behaviour Driven Development and testing Python coverage - Code coverage Strong knowledge of Databricks, including delta parquet data format and the medallion data architecture Strong knowledge of YAML. Understanding of Azure Devops Understanding of Git and code release best practices The role ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

Modern C++ Distributed systems Large-scale data processing High-performance microservices Data infrastructure and analytics pipelines Bonus points for SQL Boost ASIO Apache Arrow, Parquet, Protocol Buffers or FlatBuffers gRPC, REST or WebSockets Docker, Kubernetes, AWS or Google Cloud Rust, Python or Q This is an opportunity to solve ...

Glue Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. Hands‐on experience with AWS. #J-18808-Ljbffr ...

relational database knowledge, including data extraction and quality management Experience working with Data Warehouse and ETL processes, alongside formats such as CSV, JSON and Parquet Excellent stakeholder management skills with the ability to influence decision-making at all levels of the business Successful candidates must be able to obtain ...

Glue (ETL & Data Catalog), Lake Formation, Redshift, Athena, EMR/Spark, Kinesis/MSK, Step Functions, Lambda, EventBridge. Define storage/format strategies (e.g.,Parquet/Delta/Apache Iceberg) and partitioning, compaction, and indexing for performance and cost. Implementrow/column level security, encryption (KMS), secrets management … query planning, replication/HA, partitioning, backup/restore). Data Engineering:Spark/EMR,Glue(ETL/ELT & Studio),Airflow/Step Functionsorchestration,Parquet/Delta/Iceberg. Streaming/CDC:MSK/Kinesis,DMS(change data capture),event drivenpatterns. Security & Governance:IAM,KMS,Lake Formation,Secrets Manager ...

ticks, Level 1 & 2 quotes, order books, reference data and corporate actions. Implement automated data quality, reconciliation and monitoring frameworks. Optimise data storage using Parquet, Arrow and modern data lake technologies. Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets. Essential Skills Strong Python and SQL. … data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome candidates ...

Brahma AI operates at the intersection of enterprise Media Asset Management (MAM) and cutting-edge generative media. We build and scale industry-leading generative AI models, including hyper-realistic digital humans (ATMAN) and multilingual voice ...