1 to 25 of 39 Apache Spark Jobs in Manchester

culture. Key Responsibilities Design, build, and maintain scalable data solutions to support business objectives. Work with Microsoft Fabric to develop robust data pipelines. Utilise Apache Spark and the Spark API to handle large-scale data processing. Contribute to data strategy, governance, and architecture best practices. Identify and … approaches. Collaborate with cross-functional teams to deliver projects on time . Key Requirements ? Hands-on experience with Microsoft Fabric . ? Strong expertise in Apache Spark and Spark API . ? Knowledge of data architecture, engineering best practices, and governance . ? DP-600 & DP-700 certifications are highly More ❯

culture. Key Responsibilities Design, build, and maintain scalable data solutions to support business objectives. Work with Microsoft Fabric to develop robust data pipelines. Utilise Apache Spark and the Spark API to handle large-scale data processing. Contribute to data strategy, governance, and architecture best practices. Identify and … approaches. Collaborate with cross-functional teams to deliver projects on time . Key Requirements ✅ Hands-on experience with Microsoft Fabric . ✅ Strong expertise in Apache Spark and Spark API . ✅ Knowledge of data architecture, engineering best practices, and governance . ✅ DP-600 & DP-700 certifications are highly More ❯

you be doing? Design and implement efficient ETL processes for data extraction, transformation, and loading. Build real-time data processing pipelines using platforms like Apache Kafka or cloud-native tools. Optimize batch processing workflows with tools like Apache Spark and Flink for scalable performance. Infrastructure Automation: Implement … Integrate cloud-based data services with data lakes and warehouses. Build and automate CI/CD pipelines with Jenkins, GitLab CI/CD, or Apache Airflow. Develop automated test suites for data pipelines, ensuring data quality and transformation integrity. Monitoring & Performance Optimization: Monitor data pipelines with tools like Prometheus More ❯

IaC), automation & configuration management; • Ansible (plus Puppet, Saltstack), Terraform, CloudFormation; • NodeJS, REACT/MaterialUI (plus Angular), Python, JavaScript; • Big data processing and analysis, e.g. Apache Hadoop (CDH), Apache Spark; • RedHat Enterprise Linux, CentOS, Debian or Ubuntu. • Java 8, Spring framework (preferably Spring boot), AMQP - RabbitMQ, • Open source More ❯

Utilise Azure Databricks and adhere to code-based deployment practices Essential Skills: Over 3 years of experience with Databricks (including Lakehouse, Delta Lake, PySpark, Spark SQL) Strong proficiency in SQL with 5+ years of experience Extensive experience with Azure Data Factory Proficiency in Python programming Excellent stakeholder/client More ❯

high-impact analytics and machine learning solutions. Key Responsibilities Engineer and maintain modern data platforms with a strong focus on Databricks, including Delta Lake, Apache Spark, and MLflow Build and optimise CI/CD pipelines, infrastructure-as-code (IaC), and cloud integrations (Azure preferred, AWS/GCP beneficial … Qualifications 7+ years of experience in platform engineering, DevOps, or cloud infrastructure, with a focus on data platforms Advanced hands-on experience with Databricks, Spark, Delta Lake, and Python Proficient with Azure services (Data Factory, Storage, DevOps) and experience with IaC tools (Terraform, Bicep, ARM) Experience supporting data pipelines More ❯

high-impact analytics and machine learning solutions. Key Responsibilities Engineer and maintain modern data platforms with a strong focus on Databricks, including Delta Lake, Apache Spark, and MLflow Build and optimise CI/CD pipelines, infrastructure-as-code (IaC), and cloud integrations (Azure preferred, AWS/GCP beneficial … Qualifications 7+ years of experience in platform engineering, DevOps, or cloud infrastructure, with a focus on data platforms Advanced hands-on experience with Databricks, Spark, Delta Lake, and Python Proficient with Azure services (Data Factory, Storage, DevOps) and experience with IaC tools (Terraform, Bicep, ARM) Experience supporting data pipelines More ❯

salesforce, hubspot, SaaS, data lakes, apis, adtech, GDPR, CCPA, B2B, Sales Consultant, Account Manager, DataOps, CICD, DevOPs, CI/CD, GenAI, RAG, AWS, Azure, Apache, Spark, Kafka – UK Wide - £80,000-£95,000 + OTE Our large Microsoft Partner client requires a Senior Data Analytics Sales Consultant to More ❯

salesforce, hubspot, SaaS, data lakes, apis, adtech, GDPR, CCPA, B2B, Sales Consultant, Account Manager, DataOps, CICD, DevOPs, CI/CD, GenAI, RAG, AWS, Azure, Apache, Spark, Kafka – UK Wide - £80,000-£95,000 + OTE Our large Microsoft Partner client requires a Senior Data Analytics Sales Consultant to More ❯

salesforce, hubspot, SaaS, data lakes, apis, adtech, GDPR, CCPA, B2B, Sales Consultant, Account Manager, DataOps, CICD, DevOPs, CI/CD, GenAI, RAG, AWS, Azure, Apache, Spark, Kafka – UK Wide - £80,000-£95,000 + OTE Our large Microsoft Partner client requires a Senior Data Analytics Sales Consultant to More ❯

your cloud/platform engineering capabilities Experience working with Big Data Experience of data storage technologies: Delta Lake, Iceberg, Hudi Knowledge and understanding of Apache Spark, Databricks or Hadoop Ability to take business requirements and translate these into tech specifications Competence in evaluating and selecting development tools and More ❯

your cloud/platform engineering capabilities Experience working with Big Data Experience of data storage technologies: Delta Lake, Iceberg, Hudi Knowledge and understanding of Apache Spark, Databricks or Hadoop Ability to take business requirements and translate these into tech specifications Competence in evaluating and selecting development tools and More ❯

your cloud/platform engineering capabilities Experience working with Big Data Experience of data storage technologies: Delta Lake, Iceberg, Hudi Knowledge and understanding of Apache Spark, Databricks or Hadoop Ability to take business requirements and translate these into tech specifications Competence in evaluating and selecting development tools and More ❯

Cassandra, and Redis. In-depth knowledge of ETL/ELT pipelines, data transformation, and storage optimization. Skilled in working with big data frameworks like Spark, Flink, and Druid. Hands-on experience with both bare metal and AWS environments. Strong programming skills in Python, Java, and other relevant languages. Proficiency More ❯

related field. - Proven experience (5+ years) in developing and deploying data engineering pipelines and products - Strong proficiency in Python - Experienced in Hadoop, Kafka or Spark - Experience leading/mentoring junior team members - Strong communication and interpersonal skills, with the ability to effectively communicate complex technical concepts to both technical More ❯

with innovative ideas or coding challenges. Familiarity with Agile practices in a collaborative team setting. Exposure to big data tools, such as Hadoop and Spark, for managing large-scale datasets. Experience with cloud platforms like Microsoft Azure. Why Join? They have created a super collaborative culture offering a great More ❯

with innovative ideas or coding challenges. Familiarity with Agile practices in a collaborative team setting. Exposure to big data tools, such as Hadoop and Spark, for managing large-scale datasets. Experience with cloud platforms like Microsoft Azure. Why Join? They have created a super collaborative culture offering a great More ❯

engineering teams in a consultancy or enterprise setting. Deep expertise in cloud platforms (AWS & Azure), data warehousing (Snowflake, BigQuery, Redshift), and big data technologies (Spark, Hadoop). Strong background in software engineering practices, CI/CD, and infrastructure-as-code. Excellent stakeholder management and communication skills. Strategic mindset with More ❯

engineering teams in a consultancy or enterprise setting. Deep expertise in cloud platforms (AWS & Azure), data warehousing (Snowflake, BigQuery, Redshift), and big data technologies (Spark, Hadoop). Strong background in software engineering practices, CI/CD, and infrastructure-as-code. Excellent stakeholder management and communication skills. Strategic mindset with More ❯

Terraform experience. Strong Kubernetes experience, we want to see people who have worked on Kubernetes implementation, involved in architecture, hands on migration etc K8S, Spark S3 Engine Terraform Ansible CI/CD Hadoop Linux/RHEL - on prem background/container management Grafanna or Elastic Search- for observability Kubernetes More ❯

Terraform experience. Strong Kubernetes experience, we want to see people who have worked on Kubernetes implementation, involved in architecture, hands on migration etc K8S, Spark S3 Engine Terraform Ansible CI/CD Hadoop Linux/RHEL - on prem background/container management Grafanna or Elastic Search- for observability Kubernetes More ❯

Terraform experience. Strong Kubernetes experience, we want to see people who have worked on Kubernetes implementation, involved in architecture, hands on migration etc K8S, Spark S3 Engine Terraform Ansible CI/CD Hadoop Linux/RHEL - on prem background/container management Grafanna or Elastic Search- for observability Kubernetes More ❯

tools and alerting frameworks SQL experience including queries/updates/table creation/basic database maintenance Exposure to data technologies such as Kafka, Spark or Delta Lake is useful but not mandat Bachelor's degree in Computer Science, Engineering, or related technical field This role offers competitive compensation More ❯

tools and alerting frameworks SQL experience including queries/updates/table creation/basic database maintenance Exposure to data technologies such as Kafka, Spark or Delta Lake is useful but not mandat Bachelor's degree in Computer Science, Engineering, or related technical field This role offers competitive compensation More ❯

tools and alerting frameworks SQL experience including queries/updates/table creation/basic database maintenance Exposure to data technologies such as Kafka, Spark or Delta Lake is useful but not mandat Bachelor's degree in Computer Science, Engineering, or related technical field This role offers competitive compensation More ❯