12 of 12 Permanent Apache Arrow Jobs in the UK

computations are represented, how query plans are optimized, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers operate … want to advance the capabilities of modern data systems and apply that work to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability ...

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This ...

consumers. Work with economists and engineers to turn modelling decisions into reliable production data. Essential Strong production Python. Datasets in pandas and Parquet/Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster or Airflow ...

schema design, storage formats, compression, I/O trade-offs, and pipelines processing hundreds of terabytes. Experience with columnar formats such as Parquet or Arrow, or comparable domain‐specific formats used for event logs, telemetry, or replays. Experience designing and operating services or platforms used by other technical users ...

messaging systems (e.g., Kafka) and Kotlin Coroutines/Flow for reactive programming. Familiarity with Kotlin-native frameworks (e.g., Ktor) or functional programming libraries (e.g., Arrow). What makes us great Alongside working with some of the most talented people out there and on-top of a competitive salary, which ...

reconstructing what was knowable on a given date. Statistical modelling and optimisation (statsmodels, cvxpy; PyTorch useful). Datasets in pandas and Parquet/Arrow, plus an analytical engine such as DuckDB. Nice to have Quantitative research background, academic or industry. Index construction and classification taxonomies. Working with model-derived ...

experience writing high-quality production code Strong foundation in operating systems and systems programming Proven track record in performance engineering Familiarity with tools like Apache Arrow, Parquet, DataFusion, Clickhouse, or DuckDB is a plus Understanding of cutting-edge ML infrastructure stack (e.g. PyTorch, CUDA) is also a plus ...

planning capabilities across large tables and diverse workloads. You will work with Java, Rust, Scala, or C++ and engage with Spark, DataFusion, Iceberg, Arrow, and Parquet in a high-performance environment that values #J-18808-Ljbffr ...

tick matters, say so. Exchange APIs and real‐time/streaming systems (Kafka, Redpanda, NATS). Time‐series stores (ClickHouse, kdb+) and the Arrow/Parquet world. Infrastructure‐as‐code (Terraform) and CI/CD. How we work Small team, no layers, no theatre. You’ll own things ...

latency histograms, throughput metrics, SLAs). Familiarity with scripting languages (e.g., Python, Bash) for tooling and automation. KDB+ Data Analysis with python (pytorch, pandas, arrow) #J-18808-Ljbffr ...

/or technical program manager. Data Engineering Proficiency : Expertise in SQL, Python, Pandas, Matlab, or R. Bonus if familiarity with Influx, Grafana, Spark, Polars, Arrow, Kafka, Beam, and Flink. [Bonus] In-Field Hardware Mastery : Proficiency with hardware engineering tools such as DAQs, DDS, Nix, RTOS, MCAP, H5, TDMS, LabVIEW ...

cloud-scale storage systems. Ideal candidates should have over 5 years of experience and a strong background in operating systems. Familiarity with tools like Apache Arrow and an understanding of machine learning infrastructure are preferred. The role requires in-person work in either NYC or London, supporting innovative ...