6 Apache Spark Jobs in the North West

comfortable designing and constructing bespoke solutions and components from scratch to solve the hardest problems. Adept in Java, Scala, and big data technologies like Apache Kafka and Apache Spark, they bring a deep understanding of engineering best practices. This role involves scoping and sizing, and indeed estimating … be considered. Key responsibilities of the role are summarised below Design and implement large-scale data processing systems using distributed computing frameworks such as Apache Kafka and Apache Spark. Architect cloud-based solutions capable of handling petabytes of data. Lead the automation of CI/CD pipelines for more »

working in an Azure environment with a focus on Azure Data Lake, Databricks, Azure Data Factory and Power BI architectures. Working with Python, SQL, Spark and Databricks, you will build, rebuild and maintain data pipelines as well as have the opportunity to work with CI/CD pipelines, Azure more »

Linux, APIs, infrastructure design – load balancing, VMs, PostgreSQL, vector dbs. ML Learning Engineer – desirable skills: Version control (Git), computer vision libraries, Big Data (Hadoop, Spark), Cloud – AWS, Google Cloud, Azure, and a knowledge of secure coding techniques – PCI-DSS, PA-DSS, ISO27001. Note: as there are actually two roles more »

sponsors) Knowledge and experience of the following would be advantageous: Knowledge of Enterprise Architecture Frameworks Good knowledge of Azure DevOps Pipelines Strong experience in Apache Spark framework Previous experience in designing and delivering data warehouse and business intelligence solutions using on-premises Microsoft stack (SSIS, SSRS, SSAS) Knowledge more »

in applied machine learning on real use cases (brownie points for productionised reinforcement learning use cases!) Experience with ML/distributed ML frameworks like Spark-MLlib, TensorFlow etc. Experience with real-time scoring/evaluation of models with low latency constraints Great coding skills and strong software development experience … we use Spark, Python and Java a lot) Can work with large scale computing frameworks, data analysis systems and modeling environments. Examples include technologies like Spark, Hive, NoSQL stores etc. Bachelors, Masters or PhD program in Computer Science/Statistics or a related field Ad-tech background is more »

scripting. * Minimum of 1 year experience in investment banking or the financial sector. * Performance Tuning of Oracle/MySQL/Hive SQL Queries/Spark SQL Statements. * Experience in working with large databases - multi terabytes (3+ Terabytes). * Minimum of 5 years' experience in Big Data Space (Hive, Impala … Spark Sql, HDFS etc). * Any cloud experience (AWS/Azure/Google/Oracle). * Solid experience with Oracle objects(Packages,Procedure,functions) * Very clear concepts on Oracle architecture. * Very strong debugging skills. * Proficient in query tuning. * Detail oriented * Strong written and verbal communication skills * Ability to work more »