101 to 125 of 191 Apache Jobs in London

support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … latency, such as managing 1 billion+ records or terabyte-sized databases. Knowledge of Delta TablesorApache Iceberg:Understanding and experience with Delta Tables or Apache Iceberg for managing large-scale data lakes efficiently. Knowledge of Data Governance Tools:Experience with data governance tools like Azure Purview, UnityCatalog, or Apache ...

integration, and production support. Good understanding of ETL/ELT, data warehousing, data lakes, dimensional modelling, batch processing, and stream processing. Experience with Apache Beam, Kafka, REST APIs, Docker, Kubernetes, or GKE. Experience with data formats such as JSON, CSV, Avro, and Parquet. Working knowledge of Git, automated testing … observability, and release management. Ability to design solutions for scalability, resilience, security, performance, operability, and cost efficiency. Preferred Skills Experience with Cloud Composer or Apache Airflow, Dataproc or Spark, Dataform or dbt, and metadata-driven pipeline frameworks. Exposure to Terraform or another infrastructure-as-code tool. Knowledge of Dataplex ...

services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake ...

unify data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits Databricks strives to provide comprehensive benefits and perks. For specific details in your region, refer to the regional ...

Technical Skills: o Experience with batch processes, Guidewire Messaging framework, and OOTB plugins o Familiarity with Infra concepts and deployment architecture (WMB, WAS, and Apache) o Strong knowledge on Software architecture and design in Guidewire Ecosystem Data Analysis: Ability to write SQL queries for data analysis. Project Management ...

achieve meaningful results. You have worked with large-scale data pipelines, data platforms, or backend services. Experience with technologies such as Google Cloud, BigQuery, Apache Beam, Dataflow, dbt, Flyte, Airflow, Scala, or Java is valuable. You can navigate changing priorities, make informed decisions, and help your team move forward ...

quality Self-driven, motivated work ethic Energetic and enthusiastic personality What makes you stand out: IMC or another investment management qualification Experience with Apache Airflow Experience with Databricks Experience with asset management data Passion for design and architecture Why join our team: We help you be your best through ...

bandwidth. The goal: no software rewrites, predictable time to production, early integration, and fast iteration. You'll work with tools such as DBT, Python, Apache Airflow, Spark, Iceberg; AWS services like Lambda, Bedrock, and SageMaker, containers, and whatever adds the most value. You'll design tools that are simple ...

infrastructure. Well versed and proven experience in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast‐paced environment. Excellent communication and collaboration skills. Longstanding experience ...

infrastructure. Well versed and proven experience in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem‐solving skills and ability to work in a fast‐paced environment. Excellent communication and collaboration skills. Longstanding experience ...

product managers, to understand data requirements and deliver high-quality solutions Architect and implement data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies Optimize data pipeline performance, ensuring data accuracy, reliability, and timely delivery Mentor and guide junior data … Engineering, or a related field 7+ years of experience in data engineering, with at least 3 years in a leadership role Strong proficiency in Apache Spark, Azure Data Factory, and other data pipeline tools Experience with cloud platforms such as Azure, AWS, or GCP Solid understanding of data modeling ...

provenance. Hands‐on experience with data lineage/metadata tooling in enterprise environments (e.g., Collibra, Alation, Informatica EDC/IDMC, IBM Infosphere, Microsoft Purview, Apache Atlas, Amundsen, DataHub or similar). Proven ability to build lineage for complex platforms: data lakes, warehouses, marts, and distributed processing (Spark‐based pipelines … production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys & Apache Airflow. Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift). Familiarity with AWS S3 for large-scale data storage. ...

modern AI systems. Strong programming experience in Python. Data & Infrastructure Experience with some of the following technologies is highly desirable: Frameworks/Infrastructure Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch Databases PostgreSQL MongoDB Vector databases such as Qdrant, Milvus, or pgvector Python Libraries Pandas Polars Engineering Skills ...

13.x, REST API design* Data: MySQL (schema design, indexing, query optimization, replication/backup strategies)* Platform: Linux (Ubuntu), Docker, web server tuning (Nginx/Apache)* Cloud: AWS EC2, S3, Secrets Manager (plus IAM, VPC, security groups)* CI/CD: Jenkins, GitHub Actions* Testing: Phpunit, unit/integration/ ...

Glass UK seeks a hands-on Technical Delivery Lead to drive engineering strategy for a scalable multi-tenant data integration platform built on Apache Airflow and Google Cloud. You will lead the delivery of production-grade ETL/ELT pipelines and cloud solutions, collaborating with cross-functional teams ...

Experience • Strong hands-on experience administering Linux environments, ideally Ubuntu. • Good knowledge of web technologies and LAMP/LEMP architectures. • Experience configuring and supporting Apache or Nginx web servers. • Strong understanding of PHP and MySQL environments. • Knowledge of web technologies and authoring BASH/Shell scripts to run unattended ...

required, including 5 years' UK residency; successfully obtaining security clearance is a condition of employment These will help you succeed Strong working knowledge of Apache Iceberg and open table formats, with the ability to architect interoperable strategies that avoid vendor lock-in Practical experience designing or transitioning organisations towards ...

technical audiences alike Passion for collaboration, life-long learning, and driving business value through AI [Preferred] Experience using the Databricks Intelligence Platform and Apache SparkTM to process large-scale distributed datasets Willing to travel once every 4-8 weeks to see customers (as needed) About Databricks Databricks ...

crypto exchanges, trading venues, protocols, or market infrastructure. Experience developing collateral, position, accounting, market data, or reference data systems. Experience with Python. Experience with Apache, Kafka or other event streaming technologies. Experience using AI-assisted software development tools. Understanding of traditional financial instruments and financial market infrastructure. Our benefits ...

effectively with both business and technical stakeholders. Desirable Experience with Graphologi, PoolParty, TopBraid EDG, Semaphore or similar semantic management platforms. Experience with OpenMetadata, DataHub, Apache Atlas or related metadata platforms. Familiarity with SHACL validation frameworks and ontology quality assurance approaches. Experience supporting ontology publication and semantic asset lifecycle management. ...

Delivery and Continuous Understanding of system administration tasks on some of the following; Operating systems (e.g. Linux, CoreOS and Windows) Application stacks (e.g. nginx, Apache, Tomcat, JBoss, Websphere, Hybris, IIS, .Net, Node.js, Solr) RDBMS (e.g. PostgreSQL, MySQL, Oracle, MS SQL) & NoSQL (e.g. MongoDB, Cassandra, Riak) Containerisation and microservices (e.g. ...

Practical experience applying LLM and agentic tooling to engineering problems. Our stack: Flink, Doris, Dagster and Spark, plus OpenMetadata (or an equivalent such as Apache Atlas). Experience with comparable tools is just as relevant. Check out these videos and links to learn more about Talos and our culture ...

PyTorch, Pandas, NumPy Python/Linux environment MLOps tooling such as MLflow Jupyter Notebooks Relational and non‐relational databases: AWS Redshift, PostgreSQL, vector databases Apache Airflow for workflow orchestration AWS cloud infrastructure as well as on‐premise dedicated hardware Git Competitive salary Pension contribution - up to 6.5% of employer ...

such as RDF, OWL and SPARQL. Familiarity with feature store design and MLOps/DataOps pipeline integration. Experience with stream processing at scale using Apache Kafka or Apache Flink. Background in master data management or data mesh architecture. Consulting or comparable client‐facing delivery experience. Exposure to some ...

Knowledge of: MLOps Feature Engineering Exposure to data analysis and ML tools such as: scikit-learn pandas NumPy XGBoost LightGBM Familiarity with orchestration tools: Apache Oozie Apache Airflow Why you’ll love it here We want to make Prima a happy and empowering place to work. ...