126 to 150 of 151 Remote Apache Spark Jobs

days a week – based business need. To Be Considered: Demonstrable expertise and experience working on large-scale Data Engineering projects Strong experience in Python/PySpark, Databricks & Apache Spark Hands on experience with both batch & streaming pipelines Strong experience in AWS and associated tooling (Eg, S3, Glue, Redshift, Lambda, Terraform etc) Experience designing Data Engineering platforms from scratch More ❯

days a week – based business need. To Be Considered: Demonstrable expertise and experience working on large-scale Data Engineering projects Strong experience in Python/PySpark, Databricks & Apache Spark Hands on experience with both batch & streaming pipelines Strong experience in AWS and associated tooling (Eg, S3, Glue, Redshift, Lambda, Terraform etc) Experience designing Data Engineering platforms from scratch More ❯

days a week – based business need. To Be Considered: Demonstrable expertise and experience working on large-scale Data Engineering projects Strong experience in Python/PySpark, Databricks & Apache Spark Hands on experience with both batch & streaming pipelines Strong experience in AWS and associated tooling (Eg, S3, Glue, Redshift, Lambda, Terraform etc) Experience designing Data Engineering platforms from scratch More ❯

days a week – based business need. To Be Considered: Demonstrable expertise and experience working on large-scale Data Engineering projects Strong experience in Python/PySpark, Databricks & Apache Spark Hands on experience with both batch & streaming pipelines Strong experience in AWS and associated tooling (Eg, S3, Glue, Redshift, Lambda, Terraform etc) Experience designing Data Engineering platforms from scratch More ❯

and maintaining data pipelines. Proficiency in JVM-based languages (Java, Kotlin), ideally combined with Python and experience in Spring Boot Solid understanding of data engineering tools and frameworks, like Spark, Flink, Kafka, dbt, Trino, and Airflow. Hands-on experience with cloud environments (AWS, GCP, or Azure), infrastructure-as-code practices, and ideally container orchestration with Kubernetes. Familiarity with SQL … Snowflake, Databricks) Strong DevOps mindset with experience in CI/CD pipelines, monitoring, and observability tools (Grafana or equivalent). Exposure to analytics, reporting, and BI tools such as Apache Superset, Lightdash or OpenSearch Willingness to work across the stack by contributing to API development and, at times, UI components (Vue.js, Zoho, or similar). Excellent communication and collaboration More ❯

flexible, secure, and cost-efficient cloud solutions across products Profil Proven experience designing robust multi-cloud architectures (multi-cloud certifications preferred) Proficiency in Python, Databricks, Golang, Kafka, Typescript, and Apache Spark Deep understanding of data ingestion, processing, storage, governance, and monetization Passion for artificial intelligence and machine learning, with experience identifying and integrating use cases Ability to balance More ❯

in developing application using SpringBoot Experience in developing web infrastructure (Solr, kubernetes) Experience in git and basic Unix Commands You may also have Experience with large data processing technologies (Apache Spark) Apply now! Benefits and Contract Information Financial incentives: depending on circumstances, monthly family/marriage allowance of £278 monthly child allowance of £336 per child. Non resident More ❯

in developing application using SpringBoot Experience in developing web infrastructure (Solr, kubernetes) Experience in git and basic Unix Commands You may also have Experience with large data processing technologies (Apache Spark) Other helpful information Hybrid Working: At EMBL-EBI we are pleased to offer hybrid working options for all our employees. Our team work at least two days More ❯

Data Science. Proven experience managing large-scale data platforms or complex data pipelines. Deep understanding of analytics, ML workflows, and data product development. Strong grasp of tools such as Spark, Cassandra, and Redshift. Comfortable discussing technical architecture and data systems with engineers. Strong communicator with the ability to explain complex concepts to non-technical stakeholders. Experience working within SaaS More ❯

rather huge and includes Python (Pandas, numpy, scipy, scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA and More ❯

rather huge and includes Python (Pandas, numpy, scipy, scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA and More ❯

s graph database and develop complex database queries. Must Have's: Experience using Java to build enterprise products and applications. Knowledge of streaming analytic platforms such as Kafka, RabbitMQ, Spark, etc. Familiarity with Extract, Transform, Load (ETL) software patterns to ingest large and complex datasets. Proficiency with Git and GitLab CI/CD. Understanding of common Enterprise Integration Patterns More ❯

uncertainty quantification Hands-on experience with neural network analysis, deep learning frameworks (e.g., TensorFlow, PyTorch), and model evaluation Proficiency in Python, R, SQL, and data engineering tools such as Spark or Apache Beam and experience in designing, executing, and analyzing A/B tests Ability to develop and optimize ETL pipelines for large-scale data processing Solid understanding … improvements Develop scalable algorithms and automated data processing frameworks to optimize analytics workflows Technologies: AI AWS Azure Big Data Cloud ETL GCP Support Machine Learning Network PyTorch Python SQL Spark TensorFlow More: International crossfunctional teams working on cutting edge multimedia solutions. last updated 42 week of More ❯

Experience working with databases and BI tools such as Tableau, PowerBI, Microsoft SQL Server Reporting Services Proficiency in SQL, Python, and working knowledge of big data tools such as Spark, Kafka, or AWS Kinesis. Knowledge of DevOps tools like Jenkins/Git to assist development process Excellent communication and leadership skills; ability to collaborate across technical and non-technical … supporting data science, BI, or AI/ML product teams. AWS Certification (Solutions Architect, Data Analytics, or Big Data Specialty). Knowledge of integration tools such as Storm, and Spark, AWS Kinesis, Kafka a plus Familiarity with multi-cloud environments (e.g., Azure or GCP) is a plus. Working at ICF ICF is a global advisory and technology services provider More ❯

rather huge and includes Python (Pandas, numpy, scipy, scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA and More ❯

rather huge and includes Python (Pandas, numpy, scipy, scikit-learn, standard libraries, etc.), Python packages that wrap Machine Learning (packages for NLP, Object Detection, etc.), Linux, AWS/C2S, Apache NiFi, Spark, pySpark, Hadoop, Kafka, ElasticSearch, Solr, Kibana, neo4J, MariaDB, Postgres, Docker, Puppet, and many others. Work on this program takes place in Chantilly, VA, McLean, VA and More ❯

areas: Data integrity, scalability, and cost efficiency Managing complex data workflows and ML pipelines Ensuring model quality and performance Partnering cross-functionally across engineering, science, and product Tech Stack: Spark, Cassandra, Redshift AI/ML Use Cases: Predictive click models, data quality optimization, GenAI search applications What You'll Bring 5-10 years' experience overall, with 3+ years in More ❯

areas: Data integrity, scalability, and cost efficiency Managing complex data workflows and ML pipelines Ensuring model quality and performance Partnering cross-functionally across engineering, science, and product Tech Stack: Spark, Cassandra, Redshift AI/ML Use Cases: Predictive click models, data quality optimization, GenAI search applications What You'll Bring 5-10 years' experience overall, with 3+ years in More ❯

including technical design, coding standards, code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark - especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects Are familiar with Airflow or More ❯

on experience across AWS Glue, Lambda, Step Functions, RDS, Redshift, and Boto3. Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building Real Time event streaming pipelines (eg, Kafka, Spark Streaming, Kinesis). Proven experience developing modern data architectures including Data Lakehouse and Data … and data governance including GDPR. Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion for mentoring and enabling data More ❯

affordable charging infrastructure that is accessible to all, particularly the 40% of UK households without private driveways that rely on public charging. We believe our employees are the electric spark igniting our success. Were dedicated to our culture, and ensure that every individual feels charged up and empowered to accelerate their career which is why we have been recognised More ❯

also have Systems integration background or experience Experience of developing the Finance Data Strategy for large financial institutions, developing future state architecture Delivery experience in Big Data technologies and Apache ecosystem technologies such as Spark, Kafka, Hive etc and have experience building end to end data pipelines using on-premise or cloud-based data platforms. Hands-on experience More ❯

pipelines. Understanding of AGILE software development methodologies and use of standard software development tool suites. Preferred Skills and Experience: Experience with Docker and Kubernetes Experience with Hadoop Experience with Spark Experience with Accumulo Experience monitoring application performance with metrics (Prometheus, InfluxDB, Grafana) and logs with ELK Stack (ElsticSearch, Logstash, Kibana) Experience with asynchronous messaging systems (RabbitMQ, Apache Kafka More ❯

performing backup and restore operations and supporting development, test, and production systems. Key Responsibilities Cloudera Hadoop Administration Manage and support Cloudera Hadoop clusters and services (HDFS, YARN, Hive, Impala, Spark, Oozie, etc.). Perform cluster upgrades, patching, performance tuning, capacity planning, and health monitoring. Secure the Hadoop platform using Kerberos, Ranger, or Sentry. Develop and maintain automation and monitoring … on experience administering Cloudera Hadoop clusters. 2-3+ years of Databricks experience in production environments. 2+ years of Databricks administration experience on Azure (preferred). Strong knowledge of Spark and Delta Lake architecture. Experience with IAM, Active Directory, and SSO integration. Familiarity with DevOps and CI/CD for data platforms. Deep understanding of Hadoop ecosystem: Hive, Impala … Spark, HDFS, YARN. Experience integrating data from DB2 to Hadoop/Databricks using tools like Sqoop or custom connectors. Scripting skills in Shell and/or Python for automation and system administration. Solid foundation in Linux/Unix system administration. Preferred Qualifications Experience with cloud technologies (Azure or GCP preferred). Exposure to modern data lake and hybrid cloud More ❯

in real-time and batch data. Build and scale high-volume data pipelines handling billions of daily events across distributed systems. Work with cutting-edge tech including Java, Kafka, Spark, Kubernetes, Clickhouse, Snowflake, Redis . Collaborate with quants, data scientists, and compliance experts to improve surveillance strategies. Contribute to system architecture for low-latency monitoring and high-throughput processing … skills in Java (or similar: Kotlin, C#, C++) ; Python is a plus. Proven experience with real-time, low-latency systems and distributed data pipelines . Hands-on with Kafka, Spark, K8s, Clickhouse, Snowflake, and modern data architectures . Solid foundations in algorithms, system design, and optimisation . Curious, proactive, and comfortable working in a fast-moving, scaling environment. Why More ❯