151 to 175 of 184 Remote/Hybrid Apache Spark Jobs

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

Help shape the technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture … programming. Ability to design, build, and maintain distributed data systems in a cloud‐based environment. Hands‐on experience with big data frameworks such as Spark or Databricks. Knowledge of cloud infrastructure (AWS, GCP, or Azure). Judgement to balance scalability, performance, and maintainability. Experience with data modelling and workflow ...

shared platform capabilities for data ingestion, transformation, orchestration, and processing across batch and real-time workloads. Own the integration strategy across tools such asdbt, Spark and Beam Lead the implementation and adoption of data quality, observability, security, testing and deployment including creating and sharing standards Partner closely with Analytics … excellence, and supporting others Hands on experience transitioning to an AI-native Data Engineering landscape A strong Python and SQL engineer with experience usingdbt, Spark and Terraform Strong experience working with GCP (or a comparable cloud platform), Infrastructure as Code, and CI/CD practices. Deep understanding of data ...

software development lifecycleSolid background in Java/Scala and a scripting language like Python. Experience building large scale data pipelinesStrong familiarity with the Apache Hadoop ecosystem including : Spark, Kafka, Flink, Iceberg/Delta Lake/Hive, Apache Presto/Trino, etc. Experience with relational SQL and NoSQL ...

Citi is hiring a Big Data Applications Support Analyst to monitor and troubleshoot a large-scale data platform built on Hadoop, Hive, Spark, and Kafka. You will work with global teams to ensure 24/7 availability and data integrity for business reporting. Ideal candidates have strong Linux ...

ideal candidate will have more than 10 years of experience in backend development and data platform engineering, with strong expertise in Java and Apache Spark. The role offers competitive compensation and benefits, reflecting the company's commitment to employee well-being. #J-18808-Ljbffr ...

society's experience with software. Come join one of the fastest‐growing startups, supported by best‐in‐class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. Pendo Core Values: Bias to Act, Hone Your Craft, The Team is Pendo, and Maniacal Focus. Location: Pendo is a hybrid ...

society's experience with software. Come join one of the fastest-growing startups, supported by best-in-class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. Pendo Core Values: Bias to Act, Hone Your Craft, The Team is Pendo, and Maniacal Focus. Location: Pendo is a hybrid ...

matter - packed with personal touches and delivered with care. We're not just about selling cards or gifts - we're here to spread joy, spark smiles and make every celebration feel extra special. And with values that guide how we work and support one another, we've built … place where people (and ideas) can truly thrive. If you're looking to make an impact, bring your spark and be part of something meaningful - we'd love to have you on the team. Lead Analytics Engineer | London - Hybrid (1-2 days per week) | Competitive Salary + Benefits About ...

information across our organization.**The Platform:** You'll be supporting a modern Big Data platform built on industry-standard technologies including Hadoop, Hive, Impala, Spark, Kafka, and other open-source tools. This platform handles massive volumes of data daily, supporting analytics, reporting, and business-critical operations.**The Team … issues, and hand over to teams in other regions to maintain 24/7 coverage• Troubleshoot problems - Investigate and resolve issues with Hadoop, Spark, databases, and data processing jobs, working with other teams when needed• Manage scheduled jobs - Monitor thousands of automated jobs running daily, investigate failures, and coordinate ...

enhance solution quality while applying sound engineering judgement and responsible AI practices. What You'll DoDesign and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services. Build real-time, event-driven solutions using technologies such as Kafka and Azure Event Hubs, applying … improve productivity while maintaining appropriate human oversight, governance, and solution quality. Bonus Points ForDevelopment experience using Scala or Java. Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems. Experience working with sensitive data and applying data privacy regulations, including GDPR.Curiosity to explore new technologies, including responsible ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey ...

home affairs is highly desirable. Technical expertise in data storage, processing, and containerization technologies such as Cloud or On‐prem Distributed File Systems, Spark, Kubernetes, and Elasticsearch. Our perks and quirks. What makes you Q will help you to realize your full potential, flourish and enjoy what ...

simple: to empower the next generation of short-term rental creators by providing flexible, user-friendly, and technology-driven tools that remove friction and spark innovation. Whether someone is managing their first property or scaling a portfolio, we’re here to help turn ambition into sustainable success. ...

bringing strong opinions, updating them when better ideas emerge, and communicating decisions clearly to technical and non-technical stakeholders alikeQualificationsDeep proficiency in Python and Spark for building and optimizing large-scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow … orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure deployment and scaling decisionsCI/CD fluency—ArgoCD, automated testing pipelines, and monitoring via Prometheus—treating deployment reliability as a first-class concernFamiliarity with analytical warehousing ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern web applications. With a globally distributed presence across the US, India, and Europe, the team thrives on collaboration, bringing together diverse perspectives … Computer Science or equivalent practical experience Experience developing finance applications is a plus Working experience with Terraform is a plus Experience creating workflows for Apache Airflow is a plus What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking ...

Azure PaaS components, including Azure DevOps, Vault, and other security and configuration services (AKS experience is beneficial) Integrate and optimize data processing frameworks with Spark, Databricks, or Cloudera platform (Hue, Hive, HDFS) Support ETL pipelines built for master data management, customer profiles, and consent management workflows Ensure environments … release cycles Exposure to Azure Cloud and PaaS services, including Azure DevOps and Vault Familiarity with data engineering and big data ecosystems, including Spark and Cloudera stacks (hue, hive, HDFS) Excellent problem‐solving skills and ability to communicate technical solutions effectively Previous experience working on data‐driven projects ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

iterative delivery—making pragmatic trade‐offs and documenting them clearly. Technology Stack Languages: Predominantly Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira, Confluence Deployment … plus) Experience debugging moderately complex systems and improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure ...

technologies and innovative approaches, including AI. The environment includes: C#/.NET Core Python SQL Server Microsoft Azure Azure Kubernetes Service (AKS) Kubernetes Redis Apache Kafka Spark HBase/HDFS REST APIs Microservices and distributed architectures We're particularly interested in individuals with: Extensive experience within Credit Risk ...

generative AI inference and agentic applications. Your depth of expertise in technologies including Golang and Python programming languages, popular distributed compute frameworks including Spark/Dask/Ray/Flink, container (e.g., Kubernetes) and serverless (e.g., AWS Lambda) runtime environments, and ML+AI workload patterns will provide an amplifying technical … control and data plane implementations required to realize a highly available, multi-tenant, large scale and a secure machine learning platform Develop Ray and Spark distributed compute engine solutions to accelerate diverse workloads from LLM pre-training and reinforcement learning to large-scale data processing, while maximizing compute unit ...