10 of 10 Apache Jobs in the City of London

pragmatically. Data product-oriented design . Strong judgment on when traditional approaches still make sense. Data Engineering & Platforms Strong understanding of distributed processing (e.g. Apache Spark). Familiarity with the Hadoop ecosystem . Working knowledge of SQL and NoSQL technologies. Understanding of ETL vs ELT trade-offs . Awareness ...

support Workflow orchestration (Airflow or Dagster) AWS data services (Athena, S3, MWAA, Iceberg) Data quality monitoring & incident management Nice to Have: Airbyte DBT Apache Spark Grafana FTP/data distribution platforms Financial data experience ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

will have strong experience in data engineering, with advanced proficiency in SQL and Python, alongside exposure to large-scale data processing frameworks such as Apache Spark. You should have a solid understanding of data architecture, data wrangling and data quality practices, as well as experience working within financial markets ...

preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...

Work with solution architects to design migration-day data processes for partner onboarding. Build and maintain ETL pipelines and workflow automation using Snowflake and Apache Airflow. Document processes, learnings, and development work using version control best practices. Implement robust data quality checks and reconciliation processes throughout pipelines. Collaborate with … functionally. Strong communication skills, with the ability to translate technical concepts to non-technical audiences. Hands-on experience building and managing production workflows in Apache Airflow. Experience delivering production-grade transformations in dbt, including testing and documentation. Advanced SQL skills across multiple dialects. Strong understanding of data modelling ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

services including Vertex AI, Model Garden, BigQuery, Cloud Run and Google’s ADK. • Experience with document and data processing using tools such as Apache Tika, plus storage or caching technologies like Postgres, Redis, GCS or BigQuery. • Solid DevOps practices including CI/CD, Docker, observability and automated testing. ...

with NoSQL databases. Experience with vector databases used in modern AI systems. Strong programming experience in Python Experience within the following technologies is desirable - Apache Spark, Apache Airflow, Kafka, Elasticsearch/Open Search, PostgreSQL, MongoDB, Pandas, Polars About the Package £100k - £110k on the base ...

data sets can be effectively combined and analysed. They can also utilise programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful - R, Apache Spark. Very useful - Python, SQL, JSON, LakeFS. Reasonable Adjustments ...