25 of 25 Spark Streaming Jobs

transformation within Capital One. What You'll Do: Be a part of team designing and building Enterprise Level scalable, low-latency, fault-tolerant streaming data platform that provides meaningful and timely insights Build the next generation Distributed Streaming Data Pipelines and Analytics Data Stores using streaming frameworks (Flink, Spark Streaming) using programming languages like Java, Scala, Python Be part of a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on medium to large scale datasets Work … Java 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 3+ years experience with Distributed data computing tools (Kafka, Spark, Flink) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (DynamoDB, OpenSearch) 2+ More ❯

build personalized customer experience. What You'll Do: Be a part of team designing and building Enterprise Level scalable, low-latency, fault-tolerant streaming data platform that provides meaningful and timely insights Build the next generation Distributed Streaming Data Pipelines and Analytics Data Stores using streaming frameworks (Flink, Spark Streaming) using programming languages like Java, Scala, Python Be part of a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on medium to large scale datasets Work … Java 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 3+ years experience with Distributed data computing tools (Kafka, Spark, Flink) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (DynamoDB, OpenSearch) 2+ More ❯

for customers' auto loans. What You'll Do: Be a part of team designing and building Enterprise Level scalable, low-latency, fault-tolerant streaming data platform that provides meaningful and timely insights Build the next generation Distributed Streaming Data Pipelines and Analytics Data Stores using streaming frameworks (Flink, Spark Streaming) using programming languages like Java, Scala, Python Be part of a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on medium to large scale datasets Work … Java 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 3+ years experience with Distributed data computing tools (Kafka, Spark, Flink) 2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (DynamoDB, OpenSearch) 2+ More ❯

and transforming raw data into valuable business intelligence. What You'll Do: Lead design and build Enterprise Level scalable, low-latency, fault-tolerant streaming data platform that provides meaningful and timely insights Build the next generation Distributed Streaming Data Pipelines and Analytics Data Stores using streaming frameworks (Flink, Spark Streaming) using programming languages like Java, Scala, Python Lead a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on medium to large scale datasets Influence best practices … 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud) 4+ years experience with Distributed data computing tools (Flink, Kafka, Spark) 4+ year experience working on real-time data and streaming applications 4+ years of experience with NoSQL implementation (DynamoDB, OpenSearch) 4+ years More ❯

record of building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real More ❯

learning techniques, deep learning, graph data analytics, statistical analysis, time series, geospatial, NLP, sentiment analysis, pattern detection). Proficiency in Python, R, or Spark to extract insights from data. Experience with Data Bricks/Data QI and SQL for accessing and processing data (PostgreSQL preferred but general SQL … unit testing, version control, code review). Experience with Hadoop (especially the Cloudera and Hortonworks distributions), other NoSQL (especially Neo4j and Elastic), and streaming technologies (especially Spark Streaming). Deep understanding of data manipulation/wrangling techniques. Experience using development and deployment technologies (e.g. More ❯

infrastructure. Build efficient data models for real-time analytics. Proven experience in managing real-time data pipelines across multiple initiatives. Expertise in distributed streaming platforms (Kafka, Spark Streaming, Flink). Experience with GCP (preferred), AWS, or Azure for real-time data ingestion and storage. More ❯

infrastructure. Build efficient data models for real-time analytics. Proven experience in managing real-time data pipelines across multiple initiatives. Expertise in distributed streaming platforms (Kafka, Spark Streaming, Flink). Experience with GCP (preferred), AWS, or Azure for real-time data ingestion and storage. More ❯

record of building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real More ❯

Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13 billion. Job Description: ============= Spark - Must have Scala - Must Have Hive & SQL - Must Have Hadoop - Must Have Communication - Must Have Banking/Capital Markets Domain - Good to have Note … Candidate should know Scala/Python (Core) coding language. Pyspark profile will not help here. Scala/Spark • Good Big Data resource with the below Skillset: § Spark § Scala § Hive/HDFS/HQL • Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.) • Experience in Big data … technologies, real time data processing platform (Spark Streaming) experience would be an advantage. • Consistently demonstrates clear and concise written and verbal communication • A history of delivering against agreed objectives • Ability to multi-task and work under pressure • Demonstrated problem solving and decision-making skills • Excellent analytical More ❯

Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13 billion. Job Description: ============= Spark - Must have Scala - Must Have Hive & SQL - Must Have Hadoop - Must Have Communication - Must Have Banking/Capital Markets Domain - Good to have Note … Candidate should know Scala/Python (Core) coding language. Pyspark profile will not help here. Scala/Spark • Good Big Data resource with the below Skillset: § Spark § Scala § Hive/HDFS/HQL • Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.) • Experience in Big data … technologies, real time data processing platform (Spark Streaming) experience would be an advantage. • Consistently demonstrates clear and concise written and verbal communication • A history of delivering against agreed objectives • Ability to multi-task and work under pressure • Demonstrated problem solving and decision-making skills • Excellent analytical More ❯

project requirements Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, streaming and 'big data' technologies Implement data pipelines to ingest data to the platform, standardize and transform the data Support the development of analytics tools … and analytics experts to strive for greater functionality in our data systems. Design and architect solutions with Big Data technologies (e.g Hadoop, Hive, Spark, Kafka) Design and implement systems that run at scale leveraging containerized deployments Design, build, and scale data pipelines across a variety of source systems … Systems, or another quantitative field Minimum 5 years of experience in a Data Engineer role Required Skills: Experience with big data tools: Hadoop, Spark, etc. Experience with relational SQL and NoSQL databases, including Postgres Experience with AWS cloud or remote services: EC2, EMR, RDS, Redshift Experience with stream More ❯

in implementing cloud based data solutions using AWS services such as EC2, S3, EKS, Lambda, API Gateway, Glue and bid data tools like Spark, EMR, Hadoop etc. Hands on experience on data profiling, data modeling and data engineering using relational databases like Snowflake, Oracle, SQL Server; ETL tools … like Informatica IICS; scripting using Python, R, or Scala; workflow management tools like Autosys Experience with stream processing systems like Kafka, Spark streaming etc Experience in Java, JMS, SOAP, REST, JSON, XML technologies, along with Unix or Linux scripting Implementation experience of DevOps CI/CD More ❯

scalability. Evaluate and implement data storage solutions, including relational databases, NoSQL databases, data lakes, and cloud storage services. Define company data assets, including Spark, SparkSQL, and HiveSQL jobs to populate data models. Data Integration and API Development Build and maintain integrations with internal and external data sources and … Science, Data Science, or Information Science-related field required; Master's degree preferred. Experience with real-time data processing frameworks (e.g., Apache Kafka, Spark Streaming). Experience with data visualization tools (e.g., Tableau, Power BI, Looker). At least three years of related experience required. More ❯

security threats, actors and their techniques. Experience with data science, big data analytics technology stack, analytic development for endpoint and network security, and streaming technologies (e.g., Kafka, Spark Streaming, and Kinesis). • Strong sense of ownership combined with collaborative approach to overcoming challenges and More ❯

security threats, actors and their techniques. Experience with data science, big data analytics technology stack, analytic development for endpoint and network security, and streaming technologies (e.g., Kafka, Spark Streaming, and Kinesis). Strong sense of ownership combined with a collaborative approach to overcoming challenges More ❯

Information Systems, or another quantitative field. They should also have experience using the following software/tools: Experience with big data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow … etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with stream-processing systems: Storm, Spark-Streaming, etc. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. Salary: 30000 per annum + benefits Apply For This Job If you would More ❯

security threats, actors and their techniques. Experience with data science, big data analytics technology stack, analytic development for endpoint and network security, and streaming technologies (e.g., Kafka, Spark Streaming, and Kinesis). Amazon is an equal opportunities employer. We believe passionately that employing a More ❯

Lambda, EC2, VPC, S3) for security response and/or automation Experience with data science, machine learning, big data analytics, and/or streaming technologies (e.g., Kafka, Spark Streaming, Kinesis) Posted: December 10, 2024 (Updated about 1 hour ago) Amazon is committed to a More ❯

data lakes, and associated microservices using Java, NiFi flows, and Python. Search engine technology such as Solr, ElasticSearch Hands on Experience in Handling Spark and Kafka Cluster management Experience as software engineer lead or architect directly supporting Government technical stakeholders DESIRED QUALIFICATIONS: Experiencing interacting with AWS SDK, AWS … automated data management from end to end and sync up between all the clusters. Developed and Configured Kafka brokers to pipeline data into spark streaming. Developed Spark scripts by using scala shell commands as per the requirement. Developed spark code and spark-SQL/streaming for faster testing and processing of data. Experience with version control and release tools such as Ant, Maven, Subversion and GIT Understanding of incorporating application framework/design patterns at an enterprise level Ability to produce quality code that adheres to coding standards More ❯

FTE - Full Time Employee, ( US Citizens & Green Card Holder ) Required Skill/Responsibility/Role Hands on Design and development experience in Kafka, Spark Streaming, Scala, Hive, SQL. Shell Scripting, Java Programming Involve in planning, designing and strategizing the roadmap around On-premise and cloud solutions. … Experience in designing and developing real time data processing pipelines Expertise in working with Hadoop data platforms and technologies like Kafka, Spark, Impala, Hive and HDFS in multi-tenant environments Expert in Java programming ,SQL and shell script, DevOps Good understanding of current industry landscape and trends on More ❯

edge solutions in Big Data, Data Science, and Cloud Computing for both government and commercial clients. Dive deeper with cutting-edge tech like Spark, AWS, Azure, Cloudera, Kubernetes, and Google Cloud to build impactful solutions and gain real-time insights with Spark Streaming and More ❯

unit testing, version control, code review). Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. Streaming technologies and processing engines, Kinesis, Kafka, Pub/Sub and Spark Streaming. Experience of working with CI/CD technologies, Git, Jenkins More ❯

architectures that business engineering teams buy into and build their applications around. Required Qualifications, Capabilities, and Skills: Experience across the data lifecycle with Spark-based frameworks for end-to-end ETL, ELT & reporting solutions using key components like Spark SQL & Spark Streaming. Strong knowledge … end-to-end engineering experience supported by excellent tooling and automation. Preferred Qualifications, Capabilities, and Skills: Good understanding of the Big Data stack (Spark/Iceberg). Ability to learn new technologies and patterns on the job and apply them effectively. Good understanding of established patterns, such as More ❯

MongoDB, Cassandra). Experience with data warehousing concepts and technologies is a plus. Excellent communication and collaboration skills. Bonus Points: Experience with data streaming technologies like Kafka or Spark Streaming. Experience with containerization technologies like Docker and Kubernetes. Familiarity with Agile methodologies. O ur Equal Employment More ❯