1 to 25 of 132 Remote Apache Spark Jobs

Senior Data Engineer, you will be a key member of our dedicated data engineering, reporting, and analytics team. You will bring deep expertise in Apache Spark to the group, complementing the existing engineering talent and strengthening our overall data engineering capability. Spark is the core distributed processing …/week onsite during probation) Employment Type: Full-time Experience Level: Senior Position Responsibilities Design, develop, and optimize large-scale distributed data pipelines using Apache Spark as the core processing engine. Implement robust batch and streaming solutions using modern Lakehouse patterns (e.g., Delta Lake, streaming ingestion, structured transformations ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

data warehousing concepts, and distributed computing. Familiarity with CI/CD, version control, and DevOps practices. Nice-to-Have Experience with streaming technologies (e.g., Spark Structured Streaming, Event Hub, Kafka). Knowledge of MLflow, Unity Catalog, or advanced Databricks features. Exposure to Terraform or other IaC tools. Experience working ...

data warehousing concepts, and distributed computing. Familiarity with CI/CD, version control, and DevOps practices. Nice-to-Have Experience with streaming technologies (e.g., Spark Structured Streaming, Event Hub, Kafka). Knowledge of MLflow, Unity Catalog, or advanced Databricks features. Exposure to Terraform or other IaC tools. Experience working ...

practices for the full lifecycle of machine learning models, including deployment, monitoring, and retraining. 11. Manages infrastructure of cloud and big data technologies (e.g., Spark, Hadoop) to ensure predictive systems are scalable and efficient. 12. Ensures the integrity and quality of the data that feeds into the predictive models. ...

proficiency in programming languages such as Python, Java, or Scala. Experience with cloud platforms like Azure. Knowledge of big data technologies such as Hadoop, Spark, or Kafka. Proficiency in SQL and database management systems. Familiarity with data warehousing concepts and tools. Ability to work collaboratively with cross-functional teams. ...

proficiency in programming languages such as Python, Java, or Scala. Experience with cloud platforms like Azure. Knowledge of big data technologies such as Hadoop, Spark, or Kafka. Proficiency in SQL and database management systems. Familiarity with data warehousing concepts and tools. Ability to work collaboratively with cross-functional teams. ...

control. Excellent collaboration and communication skills, with strong attention to detail and data quality. Desirable: Exposure to AI/ML data preparation, Python or Spark, Data Vault 2.0, data governance, GDPR, and experience working in mobility, logistics, financial services, or tech-enabled environments. What ...

deliver intelligent data products, leveraging modern cloud and AI technologies. Key Responsibilities Design and implement robust data pipelines and ML workflows using Python, SQL, Spark, and Databricks. Develop and deploy machine learning models (including NLP, deep learning, and agentic AI) in production environments. Integrate data from diverse sources, including ...

platforms such as AWS, Azure, or GCP. Exposure to modern data tools such as Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and Terraform. What ...

integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. ...

integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … Excellent verbal and written communication skills BS or MS degree in Computer Science or equivalent Experience in distributed computing frameworks like - Hive/Hadoop, Apache Spark Experience in developing Finance or HR related applications Experience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

Lake Azure Synapse Azure SQL and Azure Storage Azure Data Factory ADF Expertise in building and managing complex data pipelines Databricks Handson experience with Spark based data processing notebooks and ML workflows Data Modeling Proficiency in conceptual logical and physical data modeling SQL Python Advanced skills for data manipulation ...

Lake Azure Synapse Azure SQL and Azure Storage Azure Data Factory ADF Expertise in building and managing complex data pipelines Databricks Handson experience with Spark based data processing notebooks and ML workflows Data Modeling Proficiency in conceptual logical and physical data modeling SQL Python Advanced skills for data manipulation ...

compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse ...