276 to 300 of 350 Apache Jobs

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/ ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/ ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern observability ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern observability ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern observability ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern observability ...

software development lifecycleSolid background in Java/Scala and a scripting language like Python.Experience building large scale data pipelinesStrong familiarity with the Apache Hadoop ecosystem including : Spark, Kafka, Flink, Iceberg/Delta Lake/Hive, Apache Presto/Trino, etc.Experience with relational SQL and NoSQL databases, including Postgres ...

maintaining scalable data pipelines and integrations. Developing robust ETL and data transformation processes to support analytics and reporting requirements. Working with Snowflake, DBT, Apache Airflow, AWS, SQL and Python to deliver high-quality data solutions. Collaborating with Data Architecture, DevOps and business stakeholders to ensure trusted and accessible data … ideally within AWS. Data modelling, transformation and orchestration methodologies. Delivering high-quality, well-documented and thoroughly tested data solutions. Experience with Snowflake, DBT and Apache Airflow. Interest in AI and emerging technologies within the data engineering landscape. THE BENEFITS: You will receive a salary ...

technical design reviews.Must Haves (Hard Skills)Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake.Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/Avro).Robust … engines (such as dbt) and scheduling tools (such as Airflow).Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka.Strong Programming Foundations: Proficiency in Python, SQL, or an Object-Oriented language (Java/Scala).Nice to HavesExperience operating data assets within a regulated ...

technical design reviews. Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi‐structured financial data logs (JSON/… such as dbt) and scheduling tools (such as Airflow). Stream/Message Queues: Solid understanding of event‐driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python, SQL, or an Object‐Oriented language (Java/Scala). Nice to Haves Experience operating data assets ...

technical design reviews. Must Haves (Hard Skills)Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… engines (such as dbt) and scheduling tools (such as Airflow).Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python, SQL, or an Object-Oriented language (Java/Scala).Nice to HavesExperience operating data assets within ...

software development team Hands-on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands-on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross-functional … highly scalable data engineering solutions that power Moody's next generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody's technical teams ...

governance frameworks covering data ownership, stewardship, classification (PII, sensitive, public), and retention policies Establish data lineage and cataloguing standards, working with tooling such as Apache Atlas, Collibra, or Snowflake Horizon Catalog Design and enforce data quality rules and SLAs at ingestion, transformation, and consumption layers Collaborate with the Risk … documentation, incremental strategies Application Databases Data Integration AWS – S3, RDS, Aurora, Redshift (migration context), Glue, Lake Formation, IAM, VPC Data Governance Data lineage, cataloguing (Apache Atlas/Collibra/Snowflake Horizon), GDPR, BCBS 239, MDM AI/ML Data Query & Performance SQL optimisation, clustering keys, partitioning, query profiling, cost ...

ESXi, Windows 11, Ubuntu LTS, Dell PowerEdge, HP ProLiant, NetApp, Palo Alto, SonicWall, Check Point, UniFi, Aruba, Veeam, Bitdefender, PRTG, SQL Server, MySQL, IIS, Apache, WSUS, and related technologies. Desirable ITIL Foundation. Microsoft certifications. VMware experience. Exposure to cybersecurity, SIEM, and MDR solutions. This is an excellent opportunity ...

strong and motivated FX Risk Lead Java Developer. Key Responsibilities: Design, develop, and maintain Java-based microservices for high-throughput application Integrate and optimize Apache Ignite based solution: in memory data grid for FX trades processing Implement real time data ingestion and processing pipelines using Apache Kafka/… with JVM performance optimization and memory profiling Familiarity with distributed systems architecture and in-memory computing concepts Proven experience with in-memory caching solution Apache Ignite in production environment Solid experience with MongoDB (CRUD, indexing, performance tuning) Deep SQL knowledge and query optimization in Oracle database Solid understanding ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

transparency.**Skills & Experience****Essential**Proven in-depth commercial experience in the following key areas:* Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning.* Apache Spark & Scala – High-performance distributed data processing and optimisation.* Cloud Data Engineering (AWS) – Cloud-native data platforms and distributed computing.* Data Services Architecture & Engineering … knowledge of client core business functions* Demonstrated leadership, project management, and development skills* Relationship and consensus building skills**Preferred*** Kafka/Event-Driven Architecture* Apache Iceberg/Delta Lake* Python* AI/ML & LLM integration* Financial Services/Capital Markets**Education:*** Bachelor’s degree/University degree or equivalent ...

lake. Translate architectural designs into implementation plans, work packages and practical technical delivery approaches. Lead API‐based, file‐based, legacy and bespoke integrations using Apache NiFi and AWS services. Manage the onboarding of source systems, including interface analysis, data mapping, transformation rules and security considerations. Define and coordinate unit … across different security classifications is mandatory. Essential skills and experience Strong, hands‐on AWS delivery experience, including secure engineering practices and operational observability. Strong Apache NiFi experience, including pipelines, processors, controller services, flow governance and operational monitoring. Proven experience delivering API integrations, file‐based integrations, legacy system interfaces ...

Development Contribute hands-on as a primary engineer utilizing our modern tech stack: Core Java (multi-threading, concurrency, memory optimization), CouchbaseDB, Kafka, and Apache Storm. Solve complex real-time risk calculation requirements, handling heavy stream processing and highly concurrent data flows. Required Experience extensive development experience, with a strong … including multi-threading, memory management, garbage collection tuning, and concurrency utilities. Solid experience with real-time stream processing and distributed data architectures (e.g., Kafka, Apache Storm, CouchbaseDB/NoSQL, Spring). Familiarity with modern infrastructure paradigms, such as containerization (OpenShift/Kubernetes) and cloud platforms (GCP/ ...

Development Contribute hands-on as a primary engineer utilizing our modern tech stack: Core Java (multi-threading, concurrency, memory optimization),CouchbaseDB, Kafka, and Apache Storm. Solve complex real-time risk calculation requirements , handling heavy stream processing and highly concurrent data flows. Required Experience extensive development experience , with a strong … including multi-threading, memory management, garbage collection tuning, and concurrency utilities. Solid experience with real-time stream processing and distributed data architectures (e.g., Kafka, Apache Storm,CouchbaseDB/NoSQL, Spring). Familiarity with modern infrastructure paradigms , such as containerization (OpenShift/Kubernetes) and cloud platforms (GCP/ ...

lake. Translate architectural designs into implementation plans, work packages and practical technical delivery approaches. Lead API-based, file-based, legacy and bespoke integrations using Apache NiFi and AWS services. Manage the onboarding of source systems, including interface analysis, data mapping, transformation rules and security considerations. Define and coordinate unit … Significant experience leading enterprise-scale data engineering or integration programmes. Strong, hands-on AWS delivery experience, including secure engineering practices and operational observability. Strong Apache NiFi experience, including pipelines, processors, controller services, flow governance and operational monitoring. Proven experience delivering API integrations, file-based integrations, legacy system interfaces ...

Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling.**What You'll Do:*** Write highly fault-tolerant Java code within Apache Spark to produce platform products used by our customers to query our event pipelines/ingestion for insight into active threat trends and related … between backend/cloud development and data platform engineering roles* 5+ years of experience building data platform product(s) or features with (one of) Apache Spark, Flink or Iceberg, or with comparable tools in GCP* 5+ years of experience programming with Java, Scala or Kotlin.* Proven experience owning robust ...

common technical standards and ways of working, and presenting these to the community Here's a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we're looking for We'd love ...

strong focus on cost efficiency, scalability, and operational excellence. Key Responsibilities Design and build scalable, real‐time data processing systems using technologies such as Apache Flink and Kafka. Develop high‐quality, maintainable code for Apache Flink (Java) data processing and .NET APIs. Lead and contribute to collaborative system … cost. What We’re Looking For Excellent collaboration and communication skills. Strong experience in distributed systems and event‐driven architectures. Hands‐on experience with Apache Flink (or similar stream processing frameworks) and Kafka or other messaging systems. Strong proficiency in .NET or Java. Experience building systems that are highly ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Benefits and Perks We value the total wellbeing ...