25 of 25 Apache Spark Jobs in the North West

culture. Key Responsibilities Design, build, and maintain scalable data solutions to support business objectives. Work with Microsoft Fabric to develop robust data pipelines. Utilise Apache Spark and the Spark API to handle large-scale data processing. Contribute to data strategy, governance, and architecture best practices. Identify and … approaches. Collaborate with cross-functional teams to deliver projects on time . Key Requirements ? Hands-on experience with Microsoft Fabric . ? Strong expertise in Apache Spark and Spark API . ? Knowledge of data architecture, engineering best practices, and governance . ? DP-600 & DP-700 certifications are highly More ❯

you be doing? Design and implement efficient ETL processes for data extraction, transformation, and loading. Build real-time data processing pipelines using platforms like Apache Kafka or cloud-native tools. Optimize batch processing workflows with tools like Apache Spark and Flink for scalable performance. Infrastructure Automation: Implement … Integrate cloud-based data services with data lakes and warehouses. Build and automate CI/CD pipelines with Jenkins, GitLab CI/CD, or Apache Airflow. Develop automated test suites for data pipelines, ensuring data quality and transformation integrity. Monitoring & Performance Optimization: Monitor data pipelines with tools like Prometheus More ❯

IaC), automation & configuration management; • Ansible (plus Puppet, Saltstack), Terraform, CloudFormation; • NodeJS, REACT/MaterialUI (plus Angular), Python, JavaScript; • Big data processing and analysis, e.g. Apache Hadoop (CDH), Apache Spark; • RedHat Enterprise Linux, CentOS, Debian or Ubuntu. • Java 8, Spring framework (preferably Spring boot), AMQP - RabbitMQ, • Open source More ❯

lifecycle management, including data pipelines, feature engineering, and model serving. Knowledge of MLOps practices, including versioning, monitoring, and automation. Familiarity with big data technologies (Spark, Hadoop, Databricks) is a plus. Strong problem-solving skills and ability to translate business needs into ML solutions. Excellent communication and leadership skills. Why More ❯

processes in the engineering delivery cycle including Agile and DevOps, Git, APIs, Containers, Microservices and Data Pipelines. Experience working with one or more of Spark, Kafka, or Snowflake My client have very limited interview slots and they are looking to fill this vacancy within the next 2 weeks. I More ❯

availability. Very strong technical background leading application development - with experience in some or all of the following technologies: Python, Java, Spring Boot, TensorFlow, PyTorch, Apache Spark, Kafka, Jenkins, Git/Bitbucket, Terraform, Docker, ECS/EKS, IntelliJ, JIRA, Confluence, React/Typescript, Selenium, Redux, Junit, Cucumber/Gherkin. More ❯

Utilise Azure Databricks and adhere to code-based deployment practices Essential Skills: Over 3 years of experience with Databricks (including Lakehouse, Delta Lake, PySpark, Spark SQL) Strong proficiency in SQL with 5+ years of experience Extensive experience with Azure Data Factory Proficiency in Python programming Excellent stakeholder/client More ❯

Knutsford Contract Role Job Description: AWS Services: Glue, Lambda, IAM, Service Catalogue, Cloud Formation, Lake Formation, SNS, SQS, Event Bridge Language & Scripting: Python and Spark ETL: DBT Good to Have: Airflow, Snowflake, Big Data (Hadoop), and Teradata Responsibilities: Serve as the primary point of contact for all AWS related More ❯

Cassandra, and Redis. In-depth knowledge of ETL/ELT pipelines, data transformation, and storage optimization. Skilled in working with big data frameworks like Spark, Flink, and Druid. Hands-on experience with both bare metal and AWS environments. Strong programming skills in Python, Java, and other relevant languages. Proficiency More ❯

cloud solutions Handling real-time data processing and ETL jobs Applying AI and data analytics to large datasets Working with big data tools like Apache Spark and AWS technologies such as Elastic MapReduce, Athena and Lambda Please contact Edward Laing here at ISR Recruitment to learn more about More ❯

related field. - Proven experience (5+ years) in developing and deploying data engineering pipelines and products - Strong proficiency in Python - Experienced in Hadoop, Kafka or Spark - Experience leading/mentoring junior team members - Strong communication and interpersonal skills, with the ability to effectively communicate complex technical concepts to both technical More ❯

of data visualization techniques and tools (e.g., Tableau, Power BI, matplotlib). Experience with cloud-based platforms (AWS, Azure) and big data tools (Hadoop, Spark) is a plus. Educational Background : A degree in Actuarial Science, Mathematics, Statistics, Computer Science, Data Science, or a related field. Progression towards actuarial exams More ❯

one or more of the following would also be beneficial : Scala AWS services like Kinesis, RDS, Elasticache, S3, Athena, Data pipeline, Glue, Lambda, EMR, Spark, EC2, ECS, CloudWatch or Elastic Beanstalk Jenkins or similar tools would also be a plus A team player - we highly regard collaboration. Knowledge or More ❯

s degree/PhD in Computer Science, Machine Learning, Applied Statistics, Physics, Engineering or related field Strong mathematical and statistical skills Experience with Python, Spark and SQL Experience implementing and validating a range of machine learning and optimization techniques Effective scientific communication for varied audiences Autonomy and ownership of More ❯

s degree/PhD in Computer Science, Machine Learning, Applied Statistics, Physics, Engineering or related field Strong mathematical and statistical skills Experience with Python, Spark, and SQL Experience implementing and validating a range of machine learning and optimization techniques Effective scientific communication for varied audiences Autonomy and ownership of More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

workflows professional motorsports organization. Experience using simulation tools to optimize vehicle performance. Experience with machine learning libraries. Experience with big data tools (e.g. Hadoop, Spark, SQL, and NoSQL Database experience). About GM Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace More ❯

ECOM are pleased to be exclusively recruiting for a Senior Data Engineer here in Manchester. You'll join a team where your work reaches millions. This role is within a forward-thinking company leading, offering a dynamic environment where you More ❯

As a Senior BI Developer, you will be at the forefront of creating Analytical Solutions and insights into a wide range of business processes throughout the organisation and playing a core role in our strategic initiatives to enhance data-driven More ❯

to join its innovative team. This role requires hands-on experience with machine learning techniques and proficiency in data manipulation libraries such as Pandas, Spark, and SQL. As a Data Scientist at PwC, you will work on cutting-edge projects, using data to drive strategic insights and business decisions. … e.g. Sklearn) and (Deep learning frameworks such as Pytorch and Tensorflow). Understanding of machine learning techniques. Experience with data manipulation libraries (e.g. Pandas, Spark, SQL). Git for version control. Cloud experience (we use Azure/GCP/AWS). Skills we'd also like to hear about More ❯