151 to 175 of 186 Remote Apache Spark Jobs

collaboration and delivery. What We’re Looking For Strong proficiency in Python, Scala, or Java with hands‐on experience in Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks. Practical experience building real-time event streaming pipelines using technologies such as Kafka, Spark Streaming, or Amazon … environments. Bonus Points For Expertise in data modelling, schema design, and working with structured and semi‐structured datasets. Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation. Knowledge of automating cloud-based ingestion and transformation frameworks. Experience delivering data engineering solutions within highly regulated ...

days a week and delivering high-quality, consultative outcomes. Ideal candidates will have strong communication skills and a track record in Big Data technologies (Spark, Hadoop, EMR, MapR) and NoSQL/MPP databases, with a passion for #J-18808-Ljbffr ...

fundamentals of distributed systems Strong proficiency in Java, Scala, or similar languages Proven hands-on experience with streaming and data pipeline technologies - Kafka, Spark, Flink, or similar Experience with cluster coordination tools like Zookeeper Hands-on experience with container technologies such as Docker and Kubernetes for deploying and managing ...

collaboration and delivery. What We’re Looking For Strong proficiency in Python, Scala, or Java, with hands-on experience of Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks. Experience building real-time event streaming solutions using technologies such as Kafka, Spark Streaming, or Amazon Kinesis … solutions. Bonus Points For Expertise in data modelling, schema design, and working with structured and semi-structured datasets. Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation. Knowledge of automating ingestion and transformation frameworks within cloud-native data platforms. Experience delivering solutions within highly ...

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

Help shape the technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture … programming. Ability to design, build, and maintain distributed data systems in a cloud‐based environment. Hands‐on experience with big data frameworks such as Spark or Databricks. Knowledge of cloud infrastructure (AWS, GCP, or Azure). Judgement to balance scalability, performance, and maintainability. Experience with data modelling and workflow ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming).Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

shared platform capabilities for data ingestion, transformation, orchestration, and processing across batch and real-time workloads. Own the integration strategy across tools such asdbt, Spark and Beam Lead the implementation and adoption of data quality, observability, security, testing and deployment including creating and sharing standards Partner closely with Analytics … excellence, and supporting others Hands on experience transitioning to an AI-native Data Engineering landscape A strong Python and SQL engineer with experience usingdbt, Spark and Terraform Strong experience working with GCP (or a comparable cloud platform), Infrastructure as Code, and CI/CD practices. Deep understanding of data ...

bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. RESPONSIBILITIES Recruit, hire, and grow Baseten’s first SRE team ...

Elasticsearch, Kafka, and more across different environments Technologies We Use Different backend languages, including Java, Rust, and Go Open-source technologies like Cassandra, ElasticSearch, Spark, Kafka, Kubernetes, Flink Industry-standard build tooling, including Gradle and GitHub What We Value Demonstrated ability to collaborate and empathise with a variety ...

society's experience with software. Come join one of the fastest-growing startups, supported by best-in-class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. Pendo Core Values: Bias to Act, Hone Your Craft, The Team is Pendo, and Maniacal Focus. Location: Pendo is a hybrid ...

Global Liquidity Architecture Delivery (GLAD) team to build and maintain server-side processes for regulatory and liquidity reporting. The role focuses on Scala and Apache Spark on AWS, with collaboration across a global team. You will work with business users to translate requirements into robust data solutions, contribute ...

team. What We’re Looking For Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience … Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

Develop and maintain production-quality Python and SQL code.* Use Git-based source control and branch management practices across multiple deployment environments.* Work with Spark and PySpark to process and analyse large-scale datasets.* Develop and manage machine learning workflows within an MLOps environment.* Use tooling such as MLflow … Code or JupyterHub for model development, experimentation and deployment.* Work with technologies to access, manage and analyse data at scale (e.g., Apache Iceberg and Trino).**Delivery Excellence*** Participate fully in Agile ceremonies including sprint planning, backlog refinement, demonstrations and retrospectives.* Produce high-quality technical documentation and knowledge-sharing ...

Develop and maintain production-quality Python and SQL code. Use Git-based source control and branch management practices across multiple deployment environments. Work with Spark and PySpark to process and analyse large-scale datasets. Develop and manage machine learning workflows within an MLOps environment. Use tooling such as MLflow … Code or JupyterHub for model development, experimentation and deployment. Work with technologies to access, manage and analyse data at scale (e.g., Apache Iceberg and Trino). Delivery Excellence Participate fully in Agile ceremonies including sprint planning, backlog refinement, demonstrations and retrospectives. Produce high-quality technical documentation and knowledge-sharing ...

year apprenticeship where you will be put onto the Machine Learning standard with a pathway into Software You will virtually attend Cambridge Spark on a 1-day a week basis with check ins and sessions each quarter Permanent employment after apprenticeship in out DevOps Team Apprenticeship Standard Machine learning … engineer (level 6) Training Provider CAMBRIDGE SPARK LIMITED Working Week Monday to Friday, 9.00am to 5.00pm. Hybrid working role where you may be able to work from home for 2 days a week. Expected Duration 4 Years Positions Available 1 Closing Date Thursday, 31st December 2026 Start Date Tuesday ...

solution quality while applying sound engineering judgement and responsible AI practices. What You'll Do Design and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services. Build real-time, event-driven solutions using technologies such as Kafka and Azure Event Hubs, applying … productivity while maintaining appropriate human oversight, governance, and solution quality. Bonus Points For Development experience using Scala or Java. Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems. Experience working with sensitive data and applying data privacy regulations, including GDPR. Curiosity to explore new technologies, including responsible ...

technology estate Our Data Technology Stack Cloud: Microsoft Azure Data platforms: Azure Databricks, SQL Server, Cosmos DB and Blob Storage Data processing: Python, Spark and SQL Messaging: Azure Service Bus and Event Hubs DevOps and infrastructure: Azure DevOps, GitHub Actions and Terraform You don’t need to have worked … production data pipelines Strong understanding of ETL and ELT architectures and patterns Strong programming and data transformation skills, particularly using technologies such as Python, Spark and SQL Experience working with cloud-based data platforms Experience designing systems for reliability, observability, performance and data quality Strong understanding of data modelling ...

Senior Data Scientist - Python/Machine Learning Permanent Location: UK - Remote Salary: £65,000 - £70,000 (+ benefits) Skills: Python, Machine Learning, Statistical Modelling, AWS, Data Engineering An exciting opportunity has arisen for an experienced ...

them is desirable GCP Cloud: Compute Engine, Kubernetes Engine, IAM, VPC Networks, BigQuery, Dataproc, CloudStorage, Cloud Functions, CloudSQL, Memorystore PubSub, Iceberg Infrastructure & Technologies: Apache Airflow, Druid/Imply, Kafka, Spark, Postgres, Jenkins, Scylla, Redis, Docker, Prometheus, Github, Linux. Languages: Python, Scala, C++, JavaScript, Java. What Blis can offer ...

home affairs is highly desirable. Technical expertise in data storage, processing, and containerization technologies such as Cloud or On-prem Distributed File Systems, Spark, Kubernetes, and Elasticsearch. Our perks and quirks. What makes you Q will help you to realize your full potential, flourish and enjoy what ...

work collaboratively and autonomously. Strong problem-solving skills with a pragmatic and analytical outlook. Desirable: Familiar with analytics tools such as Power BI, Apache Superset, or similar for building interactive and impactful visualizations Familiarity with distributed systems (e.g., Spark, Kafka) and how they support scalable analytics solutions Passion ...

scaling data-intensive pipelines and platforms in production environments. Deep understanding of stream and batch processing frameworks - hands-on experience with Flink or Spark is essential. Comfortable working across multiple programming languages - we primarily work with Kotlin, Python, and TypeScript. Equally comfortable designing high-level architecture as diving deep … core data infrastructure tooling such as Kafka, Redis, and/or clustered Postgres. Nice to have experience Familiarity with open lakehouse technologies, such as Apache Iceberg and dbt. Experience building data infrastructure to support AI agents - including low-latency retrieval and analytical interfaces. Experience leading teams or managing engineers ...