126 to 150 of 618 Remote Apache Spark Jobs

CD, API design, MLOps) Building machine learning models and pipelines in Python, using common libraries and frameworks (e.g., TensorFlow, MLFlow) Distributed computing frameworks (e.g., Spark, Dask) Cloud platforms (e.g., AWS, Azure, GCP) and HP computing Containerization and orchestration (Docker, Kubernetes) Strong problem-solving skills and the ability to analyse More ❯

infrastructures. Proficiency in Terraform for infrastructure provisioning and declarative, versioned infrastructure management. 5+ years provisioning & maintaining Kubernetes clusters with hands-on experience deploying Airflow, Spark & Kafka across multiple node-pools. Solid understanding of AWS services (e.g., EC2, RDS, S3, IAM, Lambda, ECS/EKS). Experience with CI/ More ❯

conveying technical concepts in a comprehensible manner Skills across the following data competencies: SQL (AWS Athena/Hive/Snowflake) Hadoop/EMR/Spark/Scala Data structures (tables, views, stored procedures) Data Modelling - star/snowflake Schemas, efficient storage, normalisation Data Transformation DevOps - data pipelines Controls - selection More ❯

environments with AI/ML components or interest in learning data workflows for ML applications . Bonus if you have e xposure to Kafka, Spark, or Flink . Experience with data compliance regulations (GDPR). What you can expect from us: Salary 65-75k Opportunity for annual bonuses More ❯

etc.) Familiarity with Python for AI/ML development is a significant advantage. Experience with data engineering pipelines or big data technologies (e.g., Kafka, Spark) is a plus. The recruitment process you can expect for this role is an initial call with your dedicated Talent Acquisition Partner who will More ❯

SQL.+ In-depth experience with data manipulation and visualization libraries (e.g., Pandas, NumPy, Matplotlib, etc.).+ Solid understanding of big data technologies (e.g., Hadoop, Spark) and cloud platforms (AWS, Azure, Google Cloud).+ Strong expertise in the full data science lifecycle: data collection, preprocessing, model development, deployment, and monitoring.+ More ❯

the UK’s digital landscape. This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a … of compressed hours. The Role Will Include Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with … Engineer in secure or regulated environments. Expertise in the Elastic Stack (Elasticsearch, Logstash, Kibana) for data ingestion, transformation, indexing, and visualization. Strong experience with Apache NiFi for building and managing complex data flows and integration processes. Knowledge of security practices for handling sensitive data, including encryption, anonymization, and access More ❯

Data Engineer (Python SQL Spark Azure Databricks) London to £130k Are you a tech savvy Data Engineer with a first class education? You could be progressing your career working on complex and challenging systems at a Hedge Fund with over $17 billion under management. What's in it for … with Python and SQL, experience with version control and contributing to a shared codebase You have experience with modern data tools and technologies including Apache Spark, Azure Databricks experience would also be great You have a strong knowledge of data management principles and best practices You have experience … office traders You have an understanding of financial markets and investment management Apply now to find out more about this Data Engineer (Python SQL Spark Azure Databricks) opportunity. At Client Server we believe in a diverse workplace that allows people to play to their strengths and continually learn. We More ❯

Drive the evolution of a data-intensive platform with a vision for AWS transformation Lead efforts to optimize data pipelines and systems with Scala, Spark, Kafka, and Java Assess and streamline the development team, enhancing architecture efficiency in AWS Balance DevOps and development, overseeing Spark/HDFS on … bare metal and refining Scala code Requirements: Expert-level experience with Hadoop, HDFS, Scala, Java, Apache Spark, and Kafka Strong understanding of DevOps and infrastructure, especially with Spark/HDFS on bare metal servers CET working hours (location in CET not required) Proactive and team-oriented approach More ❯

Kibana, Grafana, or Prometheus. Proficiency with Terraform, Docker, and Kubernetes. Software development experience in Java, Scala, or Python. Desirable (but not required): Experience with Apache Spark jobs and pipelines. Knowledge of functional programming languages. Understanding of database design concepts. Ability to write and analyze SQL queries. Application Process More ❯

Scala Knowledge of two or more common Cloud ecosystems (Azure, AWS, GCP) with expertise in at least one. Deep experience with distributed computing with Apache Spark Working knowledge CI/CD for production deployments Working knowledge of MLOps Familiarity with designing and deploying performant end-to-end data More ❯

data governance standards. Data-Driven Culture Champion : Advocate for the strategic use of data across the organization. Skills-wise, you'll definitely: Expertise in Apache Spark Advanced proficiency in Python and Pyspark Extensive experience with Databricks Advanced SQL knowledge Proven leadership abilities in data engineering Strong experience in More ❯

excellence and be a person that actively looks for continual improvement opportunities. Knowledge and skills Experience as a Data Engineer or Analyst Databricks/Apache Spark SQL/Python BitBucket/GitHub. Advantageous dbt AWS Azure Devops Terraform Atlassian (Jira, Confluence) About Us What's in it for More ❯

. Knowledge and experience of the following would be advantageous: Knowledge of Enterprise Architecture Frameworks. Good knowledge of Azure DevOps Pipelines. Strong experience in Apache Spark framework. Previous experience in designing and delivering data warehouse and business intelligence solutions using on-premises Microsoft stack (SSIS, SSRS, SSAS). More ❯

improvement mindset. One or more of the following certifications are desired: AWS Certified Developer, Databricks, Agile/Scrum, Python Programmer Preferred Qualifications: Familiarity with Apache Spark or comparable distributed data processing frameworks, preferably for large-scale data transformations and analytics. Working knowledge of data governance platforms (e.g., Collibra More ❯

written and spoken) 3+ years of experience in data engineering or a related role Strong knowledge of Python , SQL , and big data technologies (e.g., Spark, Kafka, or Airflow) Experience with cloud platforms (AWS, Azure, GCP) and data warehousing solutions Familiarity with geospatial data or environmental datasets is a big More ❯

written and spoken) 3+ years of experience in data engineering or a related role Strong knowledge of Python , SQL , and big data technologies (e.g., Spark, Kafka, or Airflow) Experience with cloud platforms (AWS, Azure, GCP) and data warehousing solutions Familiarity with geospatial data or environmental datasets is a big More ❯

shape our digital platforms 🧠 What we’re looking for Proven experience as a Data Engineer in cloud environments (Azure ideal) Proficiency in Python, SQL, Spark, Databricks Familiarity with Hadoop, NoSQL, Delta Lake Bonus: Azure Functions, Logic Apps, Django, CI/CD tools 💼 What you’ll get from Mars A More ❯

shape our digital platforms 🧠 What we’re looking for Proven experience as a Data Engineer in cloud environments (Azure ideal) Proficiency in Python, SQL, Spark, Databricks Familiarity with Hadoop, NoSQL, Delta Lake Bonus: Azure Functions, Logic Apps, Django, CI/CD tools 💼 What you’ll get from Mars A More ❯

shape our digital platforms 🧠 What we’re looking for Proven experience as a Data Engineer in cloud environments (Azure ideal) Proficiency in Python, SQL, Spark, Databricks Familiarity with Hadoop, NoSQL, Delta Lake Bonus: Azure Functions, Logic Apps, Django, CI/CD tools 💼 What you’ll get from Mars A More ❯

shape our digital platforms 🧠 What we’re looking for Proven experience as a Data Engineer in cloud environments (Azure ideal) Proficiency in Python, SQL, Spark, Databricks Familiarity with Hadoop, NoSQL, Delta Lake Bonus: Azure Functions, Logic Apps, Django, CI/CD tools 💼 What you’ll get from Mars A More ❯

shape our digital platforms 🧠 What we’re looking for Proven experience as a Data Engineer in cloud environments (Azure ideal) Proficiency in Python, SQL, Spark, Databricks Familiarity with Hadoop, NoSQL, Delta Lake Bonus: Azure Functions, Logic Apps, Django, CI/CD tools 💼 What you’ll get from Mars A More ❯

will: Design and deploy production data pipelines from ingestion to consumption within a big data architecture. Work with technologies such as Python, Java, Scala, Spark, and SQL to extract, clean, transform, and integrate data. Build scalable solutions using AWS services like EMR, Glue, Redshift, Kinesis, Lambda, and DynamoDB. Process More ❯

expertise in Python & SQL with a deep understanding of data storage solutions (Postgres, Timescale, DBT). Experience with batch & stream data processing (Flink, Kinesis, Spark). Familiarity with DevOps practices (CI/CD, Terraform, GitHub Actions). AWS ecosystem knowledge & UNIX scripting skills. A passion for time-series data More ❯

efficient and performant queries. • Skilled in optimizing data ingestion and query performance for MSSQL or other RDBMS. • Familiar with data processing frameworks such as Apache Spark. • Highly analytical and tenacious in solving complex problems. Seniority level Seniority level Not Applicable Employment type Employment type Full-time Job function Job More ❯