276 to 300 of 338 Permanent Apache Jobs

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross-functional … highly scalable data engineering solutions that power Moody's next generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody's technical teams ...

governance frameworks covering data ownership, stewardship, classification (PII, sensitive, public), and retention policies Establish data lineage and cataloguing standards, working with tooling such as Apache Atlas, Collibra, or Snowflake Horizon Catalog Design and enforce data quality rules and SLAs at ingestion, transformation, and consumption layers Collaborate with the Risk … documentation, incremental strategies Application Databases Data Integration AWS – S3, RDS, Aurora, Redshift (migration context), Glue, Lake Formation, IAM, VPC Data Governance Data lineage, cataloguing (Apache Atlas/Collibra/Snowflake Horizon), GDPR, BCBS 239, MDM AI/ML Data Query & Performance SQL optimisation, clustering keys, partitioning, query profiling, cost ...

ESXi, Windows 11, Ubuntu LTS, Dell PowerEdge, HP ProLiant, NetApp, Palo Alto, SonicWall, Check Point, UniFi, Aruba, Veeam, Bitdefender, PRTG, SQL Server, MySQL, IIS, Apache, WSUS, and related technologies. Desirable ITIL Foundation. Microsoft certifications. VMware experience. Exposure to cybersecurity, SIEM, and MDR solutions. This is an excellent opportunity ...

strong and motivated FX Risk Lead Java Developer. Key Responsibilities: Design, develop, and maintain Java-based microservices for high-throughput application Integrate and optimize Apache Ignite based solution: in memory data grid for FX trades processing Implement real time data ingestion and processing pipelines using Apache Kafka/… with JVM performance optimization and memory profiling Familiarity with distributed systems architecture and in-memory computing concepts Proven experience with in-memory caching solution Apache Ignite in production environment Solid experience with MongoDB (CRUD, indexing, performance tuning) Deep SQL knowledge and query optimization in Oracle database Solid understanding ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

transparency.**Skills & Experience****Essential**Proven in-depth commercial experience in the following key areas:* Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning.* Apache Spark & Scala – High-performance distributed data processing and optimisation.* Cloud Data Engineering (AWS) – Cloud-native data platforms and distributed computing.* Data Services Architecture & Engineering … knowledge of client core business functions* Demonstrated leadership, project management, and development skills* Relationship and consensus building skills**Preferred*** Kafka/Event-Driven Architecture* Apache Iceberg/Delta Lake* Python* AI/ML & LLM integration* Financial Services/Capital Markets**Education:*** Bachelor’s degree/University degree or equivalent ...

lake. Translate architectural designs into implementation plans, work packages and practical technical delivery approaches. Lead API‐based, file‐based, legacy and bespoke integrations using Apache NiFi and AWS services. Manage the onboarding of source systems, including interface analysis, data mapping, transformation rules and security considerations. Define and coordinate unit … across different security classifications is mandatory. Essential skills and experience Strong, hands‐on AWS delivery experience, including secure engineering practices and operational observability. Strong Apache NiFi experience, including pipelines, processors, controller services, flow governance and operational monitoring. Proven experience delivering API integrations, file‐based integrations, legacy system interfaces ...

Development Contribute hands-on as a primary engineer utilizing our modern tech stack: Core Java (multi-threading, concurrency, memory optimization), CouchbaseDB, Kafka, and Apache Storm. Solve complex real-time risk calculation requirements, handling heavy stream processing and highly concurrent data flows. Required Experience extensive development experience, with a strong … including multi-threading, memory management, garbage collection tuning, and concurrency utilities. Solid experience with real-time stream processing and distributed data architectures (e.g., Kafka, Apache Storm, CouchbaseDB/NoSQL, Spring). Familiarity with modern infrastructure paradigms, such as containerization (OpenShift/Kubernetes) and cloud platforms (GCP/ ...

Development Contribute hands-on as a primary engineer utilizing our modern tech stack: Core Java (multi-threading, concurrency, memory optimization),CouchbaseDB, Kafka, and Apache Storm. Solve complex real-time risk calculation requirements , handling heavy stream processing and highly concurrent data flows. Required Experience extensive development experience , with a strong … including multi-threading, memory management, garbage collection tuning, and concurrency utilities. Solid experience with real-time stream processing and distributed data architectures (e.g., Kafka, Apache Storm,CouchbaseDB/NoSQL, Spring). Familiarity with modern infrastructure paradigms , such as containerization (OpenShift/Kubernetes) and cloud platforms (GCP/ ...

lake. Translate architectural designs into implementation plans, work packages and practical technical delivery approaches. Lead API-based, file-based, legacy and bespoke integrations using Apache NiFi and AWS services. Manage the onboarding of source systems, including interface analysis, data mapping, transformation rules and security considerations. Define and coordinate unit … Significant experience leading enterprise-scale data engineering or integration programmes. Strong, hands-on AWS delivery experience, including secure engineering practices and operational observability. Strong Apache NiFi experience, including pipelines, processors, controller services, flow governance and operational monitoring. Proven experience delivering API integrations, file-based integrations, legacy system interfaces ...

Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling.**What You'll Do:*** Write highly fault-tolerant Java code within Apache Spark to produce platform products used by our customers to query our event pipelines/ingestion for insight into active threat trends and related … between backend/cloud development and data platform engineering roles* 5+ years of experience building data platform product(s) or features with (one of) Apache Spark, Flink or Iceberg, or with comparable tools in GCP* 5+ years of experience programming with Java, Scala or Kotlin.* Proven experience owning robust ...

common technical standards and ways of working, and presenting these to the community Here's a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we're looking for We'd love ...

strong focus on cost efficiency, scalability, and operational excellence. Key Responsibilities Design and build scalable, real‐time data processing systems using technologies such as Apache Flink and Kafka. Develop high‐quality, maintainable code for Apache Flink (Java) data processing and .NET APIs. Lead and contribute to collaborative system … cost. What We’re Looking For Excellent collaboration and communication skills. Strong experience in distributed systems and event‐driven architectures. Hands‐on experience with Apache Flink (or similar stream processing frameworks) and Kafka or other messaging systems. Strong proficiency in .NET or Java. Experience building systems that are highly ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Benefits and Perks We value the total wellbeing ...

/month in the officeWe’re looking for a **Confluent Solution Architect** to lead the design and architecture of enterprise-grade **streaming solutions** with **Apache Kafka** and the **Confluent Platform**.**What you’ll do:*** Design scalable, secure, and resilient data streaming ecosystems* Translate business requirements into technical solutions … Lead client engagements and guide implementation teams* Integrate Kafka with enterprise systems, cloud platforms, and microservices**Must-Have Skills:** 8+ years in Apache Kafka & Confluent Platform Deep knowledge of Kafka internals & KRaft architecture Experience with Confluent Cloud, stream governance, RBAC, and data lineage Expertise in stream processing (Apache ...

sprint planning, and review meetings to ensure continuous communication and progress tracking.Design, build, and optimize scalable data pipelines using Azure Data Factory and Apache Airflow for scheduling and orchestration.Implement efficient data ingestion, transformation, and movement processes from various sources into Snowflake Data Warehouse.Develop reusable components and templates to accelerate … delivering high-quality solutions.Proven experience in building and maintaining data pipelines and workflows.Strong knowledge of Azure Data Factory for data movement and transformation.Proficiency with Apache Airflow (orchestration, scheduling, DAGs, error handling). 5+ Years experience in Data Engineering using Azure Data Factory, Snowflake and AirflowTravel:There could ...

sprint planning, and review meetings to ensure continuous communication and progress tracking. Design, build, and optimize scalable data pipelines using Azure Data Factory and Apache Airflow for scheduling and orchestration. Implement efficient data ingestion, transformation, and movement processes from various sources into Snowflake Data Warehouse. Develop reusable components … solutions. Proven experience in building and maintaining data pipelines and workflows. Strong knowledge of Azure Data Factory for data movement and transformation. Proficiency with Apache Airflow (orchestration, scheduling, DAGs, error handling). 5+ Years experience in Data Engineering using Azure Data Factory, Snowflake and AirflowTravel: There could ...

Requirements: Proven commercial experience working as a Data Architect within the Defence space. Active DV Clearance. Strong AWS data engineering experience Hands-on Apache NiFi expertise Experience delivering secure ETL/ELT and data integration pipelines Ability to lead technical delivery teams and mentor engineers Strong stakeholder management … communication skills AWS data platforms and secure engineering Apache NiFi pipeline development and support API, file-based, legacy and bespoke integrations Data mapping, transformation and validation CI/CD, DevSecOps and release management Testing (unit, integration, performance and security) Monitoring, logging, alerting and operational support Nice to have ...

cross‐functional teams and setting best practice standardsDesigning data models, ETL pipelines, warehousing solutions and distributed systemsBuilding and supporting real‐time streaming platforms using Apache Flink and related technologiesEnsuring compliance with industry standards and government regulationsTracking delivery against agreed designs and managing design deviationsSupporting pre‐sales and bids with … experience in data architecture on complex, secure programmesStrong hands‐on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonusProven experience using Apache Flink for real‐time, high‐throughput data processingExperience leading, mentoring and developing technical teamsConfident engaging with senior technical and non‐technical stakeholdersKnowledge of recognised ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

using JavaScript TypeScript HTML5 CSS3 and state management tools like Redux or Context API Integrate Kafka for event streaming and realtime data processing Utilize Apache Ignite for inmemory data grid and caching solutions Collaborate with crossfunctional teams to gather and analyze business requirements ideally within the investment banking … support CICD pipelines and automate deployments using Kubernetes and AWS Mentor and provide technical guidance to junior developers on Core Java ReactJS Kafka Apache Ignite and cloudnative best practices Participate actively in Agile ceremonies including sprint planning daily standups and retrospectives Ensure adherence to security compliance standards and best ...

microservices, Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling.What You'll Do: Write highly fault-tolerant Java code within Apache Spark to produce platform products used by our customers to query our event pipelines/ingestion for insight into active threat trends and related … combined between backend/cloud development and data platform engineering roles5+ years of experience building data platform product(s) or features with (one of) Apache Spark, Flink or Iceberg, or with comparable tools in GCP5+ years of experience programming with Java, Scala or Kotlin.Proven experience utilizing AI technologies ...

deployment using tools such as Ansible, Puppet or ChefDesirable Skills/Experience• Familiar with the Big Data tech stacks, including: o Trinoo Spark o Apache Hiveo Apache Airflowo Zeppelin notebooksOur PackageTailored benefits make a real difference. That’s why we offer a comprehensive range to support your growth ...

scalable ELT/ETL flows (batch and streaming) Interface between Data Analysts and Data Scientists Enrich data and load into big data environment (Apache Hadoop on Cloudera) Profil Technical education (Computer Science HTL, Computer Science Degree, Data Science etc.) Experience in data modeling (relational databases and Apache Hadoop … Know-how of core technologies such as SQL (MS SQL Server), Apache Hadoop (Kafka, NiFi, Flink, Scala and/or Java, Python) and Linux Showing interest in working with high-frequency data processing in a near-realtime environment Wir bieten State-of-the-Art Technologies We use the latest ...

also maintaining a strong focus on cost efficiency, scalability, and operational excellence.Key ResponsibilitiesDesign and build scalable, real-time data processing systems using technologies like Apache Flink, Kafka.Develop high-quality, maintainable code for Apache Flink (Java) data processing and .NET APIs.Lead and contribute to collaborative system design, while taking … performance, latency, throughput, and cost.What We’re Looking ForExcellent collaboration and communication skills.Strong experience in distributed systems and event-driven architectures.Hands-on experience with: Apache Flink (or similar stream processing frameworks) Kafka or other messaging systems.Strong Proficiency in .NET or Java.Experience building systems that are highly observable and operationally ...