351 to 375 of 421 Apache Jobs in the UK

Kingdom. The role demands strong expertise in Solution Architecture, Cloud-Native Architecture Design, and hands-on experience with Snowflake and Data Lakehouse architectures, including Apache Iceberg. Responsibilities involve creating Architecture Design Documents, managing S3-based Data Lakes, and ensuring platform performance tuning and observability. Ideal candidates should have substantial ...

ensuring efficiency in data integration and automation for analysts. The ideal candidate will possess extensive experience in distributed systems and open-source technologies like Apache Flink and Spark. Benefits include a matched pension scheme, generous holiday allowance, and various health plans. #J-18808-Ljbffr ...

will be Ableto demonstrate software skills in some of the following technologies on both Windows and Linux: C++JavascriptPythonXMLJSONHTMLCSSWeb backend servers (Angular, Django, NodeJS, React, Apache or similar)Web SocketsDockerTCP-IP/UDPSQL or other databasesDev tools (Wireshark, system log interrogation etc..)DatabasesCyber SecurityDesirable: Exposure to IP video and video ...

organisation. Responsibilities: Build scalable ETL and ELT pipelines within AWS Develop and optimise data models in Amazon Redshift Create and manage workflows using Apache Airflow Integrate NetSuite and other Finance data sources Establish trusted data-quality, monitoring and validation processes Own solutions from initial requirements through to production Experience … Required: Strong commercial Python and SQL experience Hands-on experience with AWS data services Amazon Redshift or comparable cloud data-warehouse experience Apache Airflow or similar orchestration experience Strong ETL/ELT, data-modelling and production pipeline knowledge The confidence to work directly with Finance and technical stakeholders Experience ...

Lead the engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Apache Airflow to orchestrate ETL pipelines and enables reliable movement of data from source systems through the platform into downstream vendor systems. You’ll guide … contribution where needed (critical ETLs, framework components, design spikes, performance fixes). ETL orchestration & data movement Design and develop robust ETL pipelines orchestrated in Apache Airflow , including scheduling strategies, dependency management, retries, backfills, and processing. Establish patterns for onboarding new source systems and delivering data to downstream vendor systems ...

Lead the engineering delivery and technical direction for a scalable, multi-tenant data integration platform on Google Cloud Platform (GCP). The platform uses Apache Airflow to orchestrate ETL pipelines and enables reliable movement of data from source systems through the platform into downstream vendor systems. You’ll guide … contribution where needed (critical ETLs, framework components, design spikes, performance fixes). ETL orchestration & data movement Design and develop robust ETL pipelines orchestrated in Apache Airflow , including scheduling strategies, dependency management, retries, backfills, and processing. Establish patterns for onboarding new source systems and delivering data to downstream vendor systems ...

software development team Hands‐on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands‐on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross‐functional … highly scalable data engineering solutions that power Moody’s next‐generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody’s technical teams ...

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross-functional … support highly scalable data engineering solutions that power Moodys next-generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moodys technical teams ...

governance frameworks covering data ownership, stewardship, classification (PII, sensitive, public), and retention policies Establish data lineage and cataloguing standards, working with tooling such as Apache Atlas, Collibra, or Snowflake Horizon Catalog Design and enforce data quality rules and SLAs at ingestion, transformation, and consumption layers Collaborate with the Risk … documentation, incremental strategies Application Databases Data Integration AWS – S3, RDS, Aurora, Redshift (migration context), Glue, Lake Formation, IAM, VPC Data Governance Data lineage, cataloguing (Apache Atlas/Collibra/Snowflake Horizon), GDPR, BCBS 239, MDM AI/ML Data Query & Performance SQL optimisation, clustering keys, partitioning, query profiling, cost ...

driven approach. Creating code that is open by default and easy for others to reuse. Required Skills Experience with Big Data technologies, such as Apache Hadoop, Spark or Pig AWS Apache NiFi Python or Java Built on over a 70 year heritage, Roke offers specialist knowledge in sensors ...

optimize our web infrastructure and customer-facing platforms. The ideal candidate will have 4-6 years of systems engineering experience, with proficiency in Apache, Nginx, MySQL, and cloud environments. Enjoy a leadership position, competitive pay, and professional development opportunities within a dynamic team. #J-18808-Ljbffr ...

cloud-scale storage systems. Ideal candidates should have over 5 years of experience and a strong background in operating systems. Familiarity with tools like Apache Arrow and an understanding of machine learning infrastructure are preferred. The role requires in-person work in either NYC or London, supporting innovative storage ...

established industry player seeks a Continuing Airworthiness Management Engineer to join a dynamic team supporting the UK MOD AH-64E Apache program. In this pivotal role, you will ensure compliance with military aviation regulations while working closely with various stakeholders. Your expertise will be crucial in conducting airworthiness reviews ...

management, and quality assurance.Collaboration: Engage with business and technology teams to align the platform with organizational goals.Mandatory Skillsets: Python, Snowflake, dbt, dremio, glue jobs, Apache iceberg, AWS ,SQL.Preferred Technical Skills:Programming: 5+ years of experience in Python/Java.Cloud Expertise: Strong understanding of AWS services (e.g., Lambda, Step Functions … .Data Platforms: Hands-on experience with Snowflake and data stack technologies like Apache Iceberg and Spark.Workflow Orchestration: Exposure to tools like Apache Airflow, Prefect, Dagster, or DBT.Data Services: Familiarity with AWS Glue, Lake Formation, EMR, EventBridge, Athena, and similar services.Metadata Tools: Experience with tools like Amundsen, Atlas, DataHub ...

Solid knowledge of Linux/Unix, OpenShift/Kubernetes, and Ansible. Distributed Systems: Experience building and supporting reactive, event-sourced systems, with hands-on Apache Spark and Apache Flink. APIs & Data Formats: Strong understanding of Apache HTTP, Envoy, and REST API design, plus experience with Protobuf, Avro ...

Bigdata Technologies – PySpark, Spark • Coding Languages – Python, SQL • AWS services – Glue, Cloud formation, Lake Formation, IAM, Lambda, Redshift • Exposure to cloud technologies – Snowflake, Redshift, Apache Flink and Apache Kafka. • Work in an Agile environment and participation in scrum daily standups, sprint planning reviews and retrospectives. • Understand project requirements ...

Terraform, CloudBees/JenkinsHands-on experience with cloud databases such as Amazon DynamoDB, Aurora, or SnowflakeExperience building data streaming and event-driven architectures with Apache Kafka or Apache FlinkExperience or strong interest in AI-assisted software development and agentic engineering practicesAI-assisted engineering: Our engineers are expected ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy click apply for full ...

needed — even if hands‐on coding isn’t a part of the day‐to‐day Hands‐on experience with distributed event streaming systems like Apache Kafka. Familiarity with OLAP databases such as Apache Pinot or ClickHouse Proficient in modern data lake and warehouse tools such as S3, Databricks … Snowflake Strong foundation in the .NET ecosystem, container orchestration with Kubernetes, and cloud platforms, especially AWS Experience with distributed data processing engines like Apache Flink or Apache Spark is nice to have EPIC JOB + EPIC BENEFITS = EPIC LIFE We pay 100% for benefits except ...

basis to support a MoD-owned data platform. The role focuses on building and maintaining secure data pipelines using Python, Azure and Kubernetes, with Apache Airflow and Apache Spark integrated into the stack. You will work within a government-aligned, secure environment, ensuring data remains hosted ...

transparency. Skills & Experience Essential Proven in-depth commercial experience in the following key areas: Databricks- Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning. Apache Spark & Scala- High-performance distributed data processing and optimisation. Cloud Data Engineering (AWS) - Cloud-native data platforms and distributed computing. Data Services Architecture & Engineering … knowledge of client core business functions Demonstrated leadership, project management, and development skills Relationship and consensus building skills Preferred Kafka/Event-Driven Architecture Apache Iceberg/Delta Lake Python AI/ML & LLM integration Financial Services/Capital Markets Education: Bachelor's degree/University degree or equivalent ...

Development Contribute hands-on as a primary engineer utilizing our modern tech stack: Core Java (multi-threading, concurrency, memory optimization), CouchbaseDB, Kafka, and Apache Storm. Solve complex real-time risk calculation requirements, handling heavy stream processing and highly concurrent data flows. Required Experience extensive development experience, with a strong … including multi-threading, memory management, garbage collection tuning, and concurrency utilities. Solid experience with real-time stream processing and distributed data architectures (e.g., Kafka, Apache Storm, CouchbaseDB/NoSQL, Spring). Familiarity with modern infrastructure paradigms, such as containerization (OpenShift/Kubernetes) and cloud platforms (GCP/ ...

Development Contribute hands-on as a primary engineer utilizing our modern tech stack: Core Java (multi-threading, concurrency, memory optimization), CouchbaseDB, Kafka, and Apache Storm. Solve complex real-time risk calculation requirements, handling heavy stream processing and highly concurrent data flows. Required Experience extensive development experience, with a strong … including multi-threading, memory management, garbage collection tuning, and concurrency utilities. Solid experience with real-time stream processing and distributed data architectures (e.g., Kafka, Apache Storm, CouchbaseDB/NoSQL, Spring). Familiarity with modern infrastructure paradigms, such as containerization (OpenShift/Kubernetes) and cloud platforms (GCP/ ...

lake. Translate architectural designs into implementation plans, work packages and practical technical delivery approaches. Lead API-based, file-based, legacy and bespoke integrations using Apache NiFi and AWS services. Manage the onboarding of source systems, including interface analysis, data mapping, transformation rules and security considerations. Define and coordinate unit … Significant experience leading enterprise-scale data engineering or integration programmes. Strong, hands-on AWS delivery experience, including secure engineering practices and operational observability. Strong Apache NiFi experience, including pipelines, processors, controller services, flow governance and operational monitoring. Proven experience delivering API integrations, file-based integrations, legacy system interfaces ...