5 of 5 Apache Iceberg Jobs in England

Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). Strong Python and SQL; C++, C#, or MATLAB a plus. Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg, TimescaleDB. Data Platform Capabilities: Low-Code No-Code Data Science solutions (Dataiku, Knime), Data Virtualisation (Dataiku, Ab Initio, Starburst), Data Visualisation (Power ...

problems. Familiarity with cost optimization for high-throughput data pipelines. Technologies we use: AWS serverless architectures Kubernetes Spark Flink Java Python Databricks RisingWave Parquet, Iceberg, Delta, Paimon Terraform GitHub AI at Chainalysis AI is not a feature at Chainalysis – it is a new way of working. One that turns ...

Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well as batch and streaming processing Knowledge of data quality, governance, cataloguing and observability tools (e.g. Datadog), with ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience … including Kafka, AWS SQS or equivalent cloud-native services. Strong understanding of SQL and NoSQL databases. Experience with modern data storage technologies including HDFS, Iceberg, Elasticsearch, S3 and Data Lake architectures. Containerisation and orchestration experience using Docker, Kubernetes and/or OpenShift. Knowledge of testing frameworks, engineering standards ...

detection, quality assessment, tracking) Improve training performance across single and multi-node setups using PyTorch and Ray Build evaluation and experimentation systems (Parquet/Iceberg) for model output analysis Own model versioning, lifecycle management, and promotion to production Optimise inference pipelines using Triton; build model ensembles and define request … preprocessing Practical distributed systems experience (Ray, DDP, or similar) Experience handling TB-scale or high-throughput data pipelines Familiarity with columnar formats: Arrow, Parquet, Iceberg Nice to Have Exposure to video or visual media pipelines (FFmpeg, encoding, frame extraction) Vector search or embedding system experience Triton or production inference ...