12 of 12 Permanent Apache Jobs in the City of London

driving innovation in data engineering. Desirable Criteria Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives. Familiarity with Apache Airflow, data modelling, and metadata management Experience influencing enterprise data strategy and contributing to architectural governance. To apply for this position please submit your ...

implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design and manage secure data ingestion frameworks (e.g. Apache NiFi, APIs, SFTP/FTPS) for internal and external data sources Architect and maintain secure AWS-based data infrastructure (S3, IAM, KMS, Glue, Lake ...

scale data processing. • Proficiency in Python/SQL for data engineering and scripting. • Familiarity with risk analytics environments or financial data. • Strong experience with Apache Spark (Spark Engine), including performance optimization and distributed data processing. • Strong experience ingesting data from diverse sources including relational databases, REST APIs, FTP/ ...

strong focus on scalability, resilience, and performance. Develop and optimise scalable data pipelines , supporting both batch and streaming workloads, using technologies such as Apache Spark . Work extensively with data technologies , leveraging Python and SQL to deliver high-quality analytical and data-driven solutions. Lead the design and delivery ...

Azure Key Vault, Azure Monitor, Azure Log Analytics, Azure API Management (APIM), Azure DevOps. Databases : SQL Server, Oracle, PostgreSQL, Delta Lake Big Data : Apache Spark Version Control : GitHub, Git, Azure DevOps Visualization : Power BI & Integration with REST APIs for custom dashboards. Data Integration & Workflow Orchestration : Azure Data Factory, Databricks ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

will have strong experience in data engineering, with advanced proficiency in SQL and Python, alongside exposure to large-scale data processing frameworks such as Apache Spark. You should have a solid understanding of data architecture, data wrangling and data quality practices, as well as experience working within financial markets ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

design the data process' to be run the day each partner is migrated Build and maintain ETLs and workflow automation using Snowflake and Apache Airflow Thoroughly document all learnings and work-in-progress using version control Implement data quality checks and reconciliations at all steps in the process Collaborate … heavy data role Excellent communication skills, able to translate technical knowledge to non-technical colleagues Hands-on experience building and operating production workflows in Apache Airflow Hands-on experience delivering production-grade transformations in dbt, including tests and documentation Strong SQL skills and experience working across multiple SQL dialects. ...

Technical Skills: o Experience with batch processes, Guidewire Messaging framework, and OOTB plugins o Familiarity with Infra concepts and deployment architecture (WMB, WAS, and Apache) o Strong knowledge on Software architecture and design in Guidewire Ecosystem • Data Analysis: o Ability to write SQL queries for data analysis. • GW Cloud ...

/ELT pipelines and curated data models to support analytics and reporting. This position requires strong hands-on experience with Python, Apache Airflow and dbt as essential skills , as these will be central to building, orchestrating and transforming the bank’s data pipelines. You will use Python … develop ETL/ELT processes, Apache Airflow to design and manage workflow orchestration and scheduling, and dbt to build scalable transformation layers, data models and testing frameworks (including medallion architecture where applicable). Strong experience working in Unix/Linux environments is also key for scripting, deployment and operational ...