role-this is your chance to engineer solutions that truly matter. Key Responsibilities: Design, develop, and optimize scalable data pipelines using technologies such as Apache Spark, ApacheIceberg, Trino, OpenSearch, AWS EMR, NiFi, and Kubernetes containers. Ingest and move structured and unstructured data using approved methods into … of working with diverse data types and formats, including structured, semi-structured, and unstructured data. Familiarity with data ingestion tools and platforms such as Apache NiFi, Spark, and related open-source technologies. Demonstrated ability to collaborate across teams, including data scientists, software engineers, data stewards, and mission partners. Knowledge More ❯
design, implementation, testing, and support of next-generation features related to Dremio's Query Planner and Reflections technologies Work with open source projects like Apache Calcite and ApacheIceberg Use modular design patterns to deliver an architecture that's elegant, simple, extensible and maintainable Solve complex technical … distributed query engines. Hands on experience in query processing or optimization, distributed systems, concurrency control, data replication, code generation, networking, storage systems, heap management, Apache Arrow, SQL Operators, caching techniques, and disk spilling Hands on experience with multi-threaded and asynchronous programming models More ❯
not necessary) Agile The following is DESIRABLE, not essential: AWS or GCP Buy-side Data tools such as Glue, Athena, Airflow, Ignite, DBT, Arrow, Iceberg, Dremio Fixed Income performance, risk or attribution TypeScript and Node Role: Python Developer (Software Engineer Programmer Developer Python Fixed Income JavaScript Node Fixed Income … times a week. The tech environment is very new and will soon likely include exposure to the following: Glue, Athena, Airflow, Ignite, DBT, Arrow, Iceberg, Dremio This is an environment that has been described as the only corporate environment with a start-up/fintech attitude towards technology. Hours More ❯
diagram of proposed tables to enable discussion. Good communicator and comfortable with presenting ideas and outputs to technical and non-technical users. Worked on Apache Airflow before to create DAGs. Ability to work within Agile, considering minimum viable products, story pointing, and sprints. More information: Enjoy fantastic perks like More ❯
grasp of data governance/data management concepts, including metadata management, master data management and data quality. Ideally, have experience with Data Lakehouse toolset (Iceberg) What you'll get in return Hybrid working (4 days per month in London HQ + as and when required) Access to market leading More ❯
Scala Starburst and Athena Kafka and Kinesis DataHub ML Flow and Airflow Docker and Terraform Kafka, Spark, Kafka Streams and KSQL DBT AWS, S3, Iceberg, Parquet, Glue and EMR for our Data Lake Elasticsearch and DynamoDB More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work More ❯
and reporting Experience with data warehousing concepts and platforms, such as Snowflake and Amazon Redshift, and with databases such as Postgres, Solr, Accumulo, or Iceberg Experience integrating structured and unstructured data from various sources such as APIs, databases, or flat files, and with web services and communication protocols Experience More ❯
working with hierarchical reference data models. Proven expertise in handling high-throughput, real-time market data streams. Familiarity with distributed computing frameworks such as Apache Spark. Operational experience supporting real-time systems. Equal Opportunity Workplace We are proud to be an equal opportunity workplace. We do not discriminate based More ❯
and be responsible for building and maintaining sophisticated data pipelines that ingest data from major auto customers, utilizing advanced AWS services including EMR, S3, Iceberg and Databricks. Your work will directly impact thousands of auto dealerships across the US in the way they run their business and unleash the More ❯
and be responsible for building and maintaining sophisticated data pipelines that ingest data from major auto customers, utilizing advanced AWS services including EMR, S3, Iceberg and Databricks. Your work will directly impact thousands of auto dealerships across the US in the way they run their business and unleash the More ❯
data. What you offer Experience with AWS cloud. Experience programming, debugging, and running production systems in Python. Exposure to open-source technologies such as Iceberg, Trino, and Airflow. Passionate about the use and adoption of these capabilities, focused on user experience and ensuring our business sees real value from More ❯
to-end engineering experience supported by excellent tooling and automation. Preferred Qualifications, Capabilities, and Skills: Good understanding of the Big Data stack (Spark/Iceberg). Ability to learn new technologies and patterns on the job and apply them effectively. Good understanding of established patterns, such as stability patterns More ❯
analysis and automation. Proficiency in building and maintaining batch and streaming ETL/ELT pipelines at scale, employing tools such as Airflow, Fivetran, Kafka, Iceberg, Parquet, Spark, Glue for developing end-to-end data orchestration leveraging on AWS services to ingest, transform and process large volumes of structured and More ❯
delivering customer proposals aligned with Analytics Solutions. Experience with one or more relevant tools (Sqoop, Flume, Kafka, Oozie, Hue, Zookeeper, HCatalog, Solr, Avro, Parquet, Iceberg, Hudi). Experience developing software and data engineering code in one or more programming languages (Java, Python, PySpark, Node, etc). AWS and other More ❯
Reference Data Management, and Metadata Management. Be comfortable coding with Python or Scala and proficient in SQL In-depth understanding of Parquet, DeltaLake and Iceberg data formats Have a background in using multiple data storage technologies including relational, document, key/value, graph and object stores Have ability to More ❯
Reference Data Management, and Metadata Management. Be comfortable coding with Python or Scala and proficient in SQL In-depth understanding of Parquet, DeltaLake and Iceberg data formats Have a background in using multiple data storage technologies including relational, document, key/value, graph and object stores Have ability to More ❯
Reference Data Management, and Metadata Management. Be comfortable coding with Python or Scala and proficient in SQL In-depth understanding of Parquet, DeltaLake and Iceberg data formats Have a background in using multiple data storage technologies including relational, document, key/value, graph and object stores Have ability to More ❯
Reference Data Management, and Metadata Management. Be comfortable coding with Python or Scala and proficient in SQL In-depth understanding of Parquet, DeltaLake and Iceberg data formats Have a background in using multiple data storage technologies including relational, document, key/value, graph and object stores Have ability to More ❯
for each project including ETL mappings, code use guide, code location and access instructions. Design and optimize Data Pipelines using tools such as Spark, ApacheIceberg, Trino, OpenSearch, EMR cloud services, NiFi and Kubernetes containers Ensure the pedigree and provenance of the data is maintained such that the More ❯
Salary range: £70,000-£80,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as More ❯
Salary range: £70,000-£80,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as More ❯
using Python and pandas within a financial environment. Strong knowledge of relational databases and SQL. Familiarity with various technologies such as S3, Kafka, Airflow, Iceberg Proficiency working with large financial datasets from various vendors. A commitment to engineering excellence and pragmatic technology solutions. A desire to work in an … understanding of financial markets. Experienceworking with hierarchical referencedata models. Provenexpertise in handling high-throughput, real-time marketdata streams Familiarity with distributed computing frameworks suchas Apache Spark Operational experience supporting real time systems. Equal Opportunity Workplace We are proud to be an equal opportunity workplace. We do not discriminatebased upon More ❯
Milton Keynes, Buckinghamshire, United Kingdom Hybrid / WFH Options
Banco Santander SA
AWS Data Engineer S3 Data Centre of Excellence AWS Data Engineer S3 Data Centre of Excellence Country: United Kingdom Interested in part-time, job-share or flexible working? We want to talk to you! Join our community. We have an More ❯
Cardiff, South Glamorgan, United Kingdom Hybrid / WFH Options
RVU Co UK
Staff Software Engineer - Data Department: Engineering Employment Type: Full Time Location: Cardiff Description is the UK's first comparison platform for car insurance. We've been helping customers since 2002 by empowering them to make better decisions around insurance and More ❯
