1 to 25 of 106 Permanent Apache Spark Jobs in the UK excluding London

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … data Billing & revenue assurance Build models and pipelines to support network analytics, customer 360, churn prediction, fraud detection , etc. 7. Performance Optimization & Monitoring Tune Spark/Glue jobs for performance and cost. Monitor Redshift/Athena/S3 efficiency and implement best practices. Perform data quality checks and validation ...

proficiency in programming languages such as Python, Java, or Scala. Experience with cloud platforms like Azure. Knowledge of big data technologies such as Hadoop, Spark, or Kafka. Proficiency in SQL and database management systems. Familiarity with data warehousing concepts and tools. Ability to work collaboratively with cross-functional teams. ...

control. Excellent collaboration and communication skills, with strong attention to detail and data quality. Desirable: Exposure to AI/ML data preparation, Python or Spark, Data Vault 2.0, data governance, GDPR, and experience working in mobility, logistics, financial services, or tech-enabled environments. What ...

years in a team leadership or mentorship role. Strong technical background in building and maintaining data pipelines and platforms using tools like Spark, dbt, Airflow, Kafka, or Databricks. Proficiency in one or more programming languages (e.g., Python, Scala, or Java) and SQL. Experience with cloud data platforms (e.g. ...

years in a team leadership or mentorship role. Strong technical background in building and maintaining data pipelines and platforms using tools like Spark, dbt, Airflow, Kafka, or Databricks. Proficiency in one or more programming languages (e.g., Python, Scala, or Java) and SQL. Experience with cloud data platforms (e.g. ...

years in a team leadership or mentorship role. Strong technical background in building and maintaining data pipelines and platforms using tools like Spark, dbt, Airflow, Kafka, or Databricks. Proficiency in one or more programming languages (e.g., Python, Scala, or Java) and SQL. Experience with cloud data platforms (e.g. ...

years in a team leadership or mentorship role. Strong technical background in building and maintaining data pipelines and platforms using tools like Spark, dbt, Airflow, Kafka, or Databricks. Proficiency in one or more programming languages (e.g., Python, Scala, or Java) and SQL. Experience with cloud data platforms (e.g. ...

deliver intelligent data products, leveraging modern cloud and AI technologies. Key Responsibilities Design and implement robust data pipelines and ML workflows using Python, SQL, Spark, and Databricks. Develop and deploy machine learning models (including NLP, deep learning, and agentic AI) in production environments. Integrate data from diverse sources, including ...

platforms such as AWS, Azure, or GCP. Exposure to modern data tools such as Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and Terraform. What ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … Excellent verbal and written communication skills BS or MS degree in Computer Science or equivalent Experience in distributed computing frameworks like - Hive/Hadoop, Apache Spark Experience in developing Finance or HR related applications Experience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB ...

repositories. 7. Document data architecture, processes, and workflows for reference and knowledge sharing. 8. Utilise programming languages (e.g., C#, Python, SQL) and technologies (e.g., Apache Spark, SSIS, .NET) to manipulate and analyse data. 9. Participate in code reviews, version control (e.g., using Git), and other software development best ...

Lake Azure Synapse Azure SQL and Azure Storage Azure Data Factory ADF Expertise in building and managing complex data pipelines Databricks Handson experience with Spark based data processing notebooks and ML workflows Data Modeling Proficiency in conceptual logical and physical data modeling SQL Python Advanced skills for data manipulation ...

compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse ...

ideal person for this role will have a strong background in data engineering , with experience building modern data solutions using technologies like Kafka , Spark , Databricks , dbt , and Airflow . You'll know your way around cloud platforms (AWS, GCP, or Azure) and be confident coding in Python , Java ...

ideal person for this role will have a strong background in data engineering , with experience building modern data solutions using technologies like Kafka , Spark , Databricks , dbt , and Airflow . You'll know your way around cloud platforms (AWS, GCP, or Azure) and be confident coding in Python , Java ...

also have Experience developing or using AI applications Experience with NoSQL databases (e.g. MongoDB). Experience with big data processing frameworks (e.g. Apache Spark). Knowledge of Large Language Model (LLM) integration, Retrieval-Augmented Generation (RAG), or Agentic AI frameworks. Experience with high-performance computing (HPC) environments (e.g. ...

collaborative, forward-thinking culture where engineers work on large-scale datasets, distributed systems, and modern cloud platforms. Teams leverage cutting-edge tools such as Spark, Airflow, Kafka, dbt, and Databricks to build resilient, scalable, and high-quality data solutions. Why this role? ** Lead a talented team of data engineers ...

collaborative, forward-thinking culture where engineers work on large-scale datasets, distributed systems, and modern cloud platforms. Teams leverage cutting-edge tools such as Spark, Airflow, Kafka, dbt, and Databricks to build resilient, scalable, and high-quality data solutions. Why this role? ** Lead a talented team of data engineers ...

collaborative, forward-thinking culture where engineers work on large-scale datasets, distributed systems, and modern cloud platforms. Teams leverage cutting-edge tools such as Spark, Airflow, Kafka, dbt, and Databricks to build resilient, scalable, and high-quality data solutions. Why this role? ** Lead a talented team of data engineers ...

collaborative, forward-thinking culture where engineers work on large-scale datasets, distributed systems, and modern cloud platforms. Teams leverage cutting-edge tools such as Spark, Airflow, Kafka, dbt, and Databricks to build resilient, scalable, and high-quality data solutions. Why this role? ** Lead a talented team of data engineers ...

frameworks (e.g. TensorFlow, PyTorch, Keras) Knowledge of prompt engineering, RAG (Retrieval Augmented Generation), and LLM fine-tuning techniques Familiarity with distributed computing frameworks (e.g. Spark) Knowledge of graph neural networks, reinforcement learning, or causal inference Experience with AI governance, model risk management, and regulatory compliance Experience using Pro code ...

high-quality, secure, and scalable data solutions gaining exposure to multiple cloud platforms (AWS, Azure, OCI, GCP) and technologies such as Palantir Foundry, Databricks, Spark, and Airflow. Other times you will be working directly with the client, using your analytical and data skills to support their decision-making. … Eligible for UK Security Clearance (or 5+ years UK residency . Desirable Skills & Exposure Familiarity with data lakes, warehouses, or streaming architectures (e.g. Databricks, Spark, Kafka). Experience with ETL tools, orchestration frameworks, or automation pipelines. Understanding of version control and CI/CD (e.g. GitHub, Azure DevOps). ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...