10 of 10 Apache Spark Jobs in West Yorkshire

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

control. Excellent collaboration and communication skills, with strong attention to detail and data quality. Desirable: Exposure to AI/ML data preparation, Python or Spark, Data Vault 2.0, data governance, GDPR, and experience working in mobility, logistics, financial services, or tech-enabled environments. What ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse ...

/or machine learning) Deep technical understanding of data engineering and AI/ML tooling, workflows, and trends in enterprise setting Proficiency in Python, Spark, and/or SQL with experience developing ETL applications Excellent communication and presentation skills, with comfort in objection handling Hands-on experience with ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

PyTorch) into real customer value Working closely with engineering teams using TypeScript, Node.js, React, and Next.js Partnering with data teams working across AWS, Kafka, Spark, and modern data pipelines Shaping products built on secure, scalable fintech architecture (Event-driven, microservices, serverless) Running discovery experiments and validating concepts at speed ...

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

driven solutions. • Technical Excellence: Promote and enable adoption of Technical Standards and Engineering Effectiveness within development squads. • Technical Experience: Demonstrate expertise in using SQL (Spark, Dremio), Python, GitHub and data orchestration tools (Airflow, Oozie) for data wrangling, building data pipelines and developing analytical interfaces. • AI-Assisted Analytics: Bring experience … practices, AI capabilities, advanced analytics and automation practices. You will need • Experience developing robust data products that are actionable and scalable.• Expertise of using Spark, Dremio, Teradata or other SQL technologies.• Strong knowledge and experience of using business intelligence, ETL frameworks and visualisation tools such as Tableau.• Experience with ...

driven solutions. • Technical Excellence: Promote and enable adoption of Technical Standards and Engineering Effectiveness within development squads. • Technical Experience: Demonstrate expertise in using SQL (Spark, Dremio), Python, GitHub and data orchestration tools (Airflow, Oozie) for data wrangling, building data pipelines and developing analytical interfaces. • AI-Assisted Analytics: Bring experience … practices, AI capabilities, advanced analytics and automation practices. You will need • Experience developing robust data products that are actionable and scalable.• Expertise of using Spark, Dremio, Teradata or other SQL technologies.• Strong knowledge and experience of using business intelligence, ETL frameworks and visualisation tools such as Tableau.• Experience with ...