26 to 50 of 82 Remote Apache Spark Jobs

multidisciplinary team to support and enhance over 70 active BI products. What You'll Bring Essential Skills & Experience: Core Technologies: Azure Databricks (or similar Spark-based platforms) and Azure Data Factory (ADF). Languages: SQL and Python. Data Platforms: Azure Data Lake and SQL Server RDBMS. ETL Tools ...

experience with the SQL Server platform and T-SQL querying. Familiarity with cloud data platforms (e.g., AWS, Azure) and big data technologies (e.g., Kafka, Spark, Databricks, Snowflake). Expert-level knowledge of building ETL processes in Python. Expert-level knowledge of performance tuning and optimization in a Python/ ...

data engineering, ETL development or data platform delivery. Strong programming skills in Java or Python. Exposure to modern data technologies such as Kafka, Spark or Flink. Experience working within cloud environments, ideally AWS. Understanding of software engineering principles, version control and CI/CD practices. Familiarity with container technologies ...

scale datasets Preferred Qualifications Knowledge or interestin investment banking or financial instruments Experience with big data concepts, such as Lakehouse, ETL data pipelines (e.g. Spark, AWS Glue) Experience with near real time and event-based systems like Kafka Goldman Sachs Highlights At Goldman Sachs, we commit our people, capital ...

datasets Preferred Qualifications Knowledge or interest in investment banking or financial instruments Experience with big data concepts, such as Lakehouse, ETL data pipelines (e.g. Spark, AWS Glue) Experience with near real time and event-based systems like Kafka ABOUT GOLDMAN SACHS At Goldman Sachs, we commit our people, capital ...

scale datasets Preferred Qualifications Knowledge or interestin investment banking or financial instruments Experience with big data concepts, such as Lakehouse, ETL data pipelines (e.g. Spark, AWS Glue) Experience with near real time and event-based systems like Kafka At Goldman Sachs, we commit our people, capital and ideas ...

Computational Complexity.* Strive for excellence and continuous improvement in software architecture, software delivery and best practices.* Hands on experience in C#, OOP python, Spark/PySpark and other C#, Python framework ecosystem.* Debug and resolve issues in large datasets quickly.* Co-ordinating handoffs between different time zones and allocating ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

auction systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

designing and enforcing data contracts and quality SLAs in distributed data ecosystems. Proficiency in Python, Java, or Scala, and experience building pipelines with Databricks, Spark, or Kafka. Strong understanding of data governance principles, privacy regulations (GDPR, CCPA), and secure data handling practices. Familiarity with metadata management and data catalog ...

platforms that enable real‐time and batch data processing at scale. Responsibilities Drive platform capability improvements – continuously enhance distributed data platforms using knowledge of Spark, Databricks, Kafka, Pulsar, Cassandra, and object stores to maintain best‐in‐class capabilities. Enable application data consumption – understand how applications produce and consume data … structures, partitioning strategies, consistency models, and performance optimisation, as well as OLTP databases such as PostgreSQL and MySQL. Distributed compute frameworks – strong experience with Apache Spark and distributed data processing, including optimisation, performance tuning, and large‐scale batch/streaming workloads. Cloud‐native data platforms – demonstrated experience with ...

Experience with Infrastructure-as-Code tools such as Terraform Knowledge of DevOps, CI/CD, platform automation, and cloud‐native operational practices Familiarity with Apache Spark, dbt, and modern data integration technologies Delivery Experience Experience leading architecture for enterprise-wide Databricks implementations and cloud data modernization initiatives Proven ...

Scala. Strong knowledge of SQL and experience with data warehousing solutions. Familiarity with cloud platforms (AWS, Azure, Google Cloud) and big data technologies (Hadoop, Spark). Excellent problem-solving skills and a keen eye for detail. A collaborative spirit and a genuine passion for data! Our Client's Benefits ...

services, capital markets or FinTech. Additional AWS services such as SQS, SNS, Step Functions, EventBridge or Secrets Manager. Big data technologies including DuckDB, pandas, Spark, Databricks or Snowflake. AI/ML technologies including vector databases, semantic search or LLM orchestration frameworks. Continuous delivery and modern development workflows. Data science ...

Computer Vision: Familiarity with computer vision techniques and libraries like OpenCV. Experience with Big Data Tools: Experience with big data tools like Hadoop, Spark, or Hive. Knowledge of Reinforcement Learning: Understanding of reinforcement learning techniques and algorithms. Familiarity with Robotics: Experience with robotics and robotic process automation (RPA) tools. ...

encouraged to talk about it Experience in financial services,insuranceor another regulated sector Experience deploying real‐time or streaming ML models (e.g.Kafka, Flink, Spark Structured Streaming) Passion for automation, tooling, and building reusable systems Interest in responsible AI and ML model governance Why Compare the Market ...

both OnPrem and Cloud* Experience with processing large datasets as well as streaming data and unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java ...

best practices including CI/CD, deployment and monitoring. Mentor junior team members and contribute to technical design decisions. Essential Skills Python & SQL Databricks & Apache Spark Agentic AI/AI Agents Generative AI & LLMs RAG, Embeddings & Vector Search NLP MLflow/MLOps Experience deploying AI solutions into production ...

Strong commercial experience in data engineering and modern data platforms. Advanced programming skills in Java or Python. Expertise working with technologies such as Kafka, Spark, Flink or similar distributed data frameworks. Experience delivering solutions within AWS or other major cloud platforms. Strong understanding of data architecture, scalability and performance ...

scalable data analytical solutions in a large-scale business environment Experience in deep analytic focused work leveraging Distributed File System ecosystem tools: Linux, Python, Spark, Hive; and modern data platforms and tools (e.g. Databricks, NoSQL stores) and AI/ML frameworks (e.g. NLP). LLM integration patterns, RAG pipelines ...

language like Go, Java, JavaScript, Python Distributed Software Architecture exposure in high volume production scenarios Working with Data Mesh, BigData technologies such as EMR, Spark, Databricks Designing, tracking and testing to SLOs and Chaos Engineering toError Budgets Implementing Business Continuity (BCP) and Disaster Recovery (DRP) plans including tracking ...

writing, talks, or threads about building with AI. MLOps and model serving experience (BentoML, MLflow, Vertex AI, SageMaker). Streaming and batch ingestion pipelines (Spark, Airflow, Beam, Glue). Healthcare or life sciences domain exposure. AWS Professional certification or other relevant industry certifications. What We Offer At Newpage ...

including Feast or AWS Feature Store. Docker, Kubernetes, or Amazon ECS. Infrastructure as Code using Terraform or CloudFormation. Big data processing technologies such as Spark or Dask. Experience working within regulated or highly secure environments. What's On Offer Work on meaningful AI programmes supporting critical national security missions. ...

Python. Experience with Kubernetes and Docker. Understanding of CI/CD pipelines (e.g. GitLab). Exposure to data platforms, MLOps or machine learning environments. Spark or Scala. AWS services (e.g. S3). Elasticsearch or graph databases. OIDC/OAuth. Node.js or React. To apply for this Analytics Platform Engineer ...

loop, calibration, guardrails, auditability) Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...