4 Spark SQL Jobs

Spark Architect/SME Contract Role- 6 months to begin with & its extendable Location: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance … issues. Skills: Spark Architecture – component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans. Spark SME – Be able to analyse Spark code failures through Spark Plans and make … correcting recommendations. Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations. Spark – SME Be able to understand Data Frames/Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations. Monitoring more »

role in designing, building, and maintaining their data infrastructure while collaborating closely with senior stakeholders across the organisation. Your expertise in Azure, Databricks, Spark, Python, and data modelling will be critical in driving the success of their data initiatives. Key Responsibilities: Lead the complete development cycle of data … of data modelling, data warehousing principles, and the innovative Lakehouse architecture. Exceptional proficiency in ETL methodologies, preferably utilising Azure Databricks or equivalent technologies (Spark, Spark SQL, Python, SQL), including deep insight into ETL/ELT design patterns. Proficient in Databricks, SQL more »

DB Developer: * Minimum of 8 years' development and experience in MySQL/Oracle comprehending the existing SQL queries and writing medium to complex SQL queries. * Minimum of 3 years' experience in Unix and shell scripting. * Minimum of 1 year experience in investment banking or the financial … sector. * Performance Tuning of Oracle/MySQL/Hive SQL Queries/Spark SQL Statements. * Experience in working with large databases - multi terabytes (3+ Terabytes). * Minimum of 5 years' experience in Big Data Space (Hive, Impala, Spark Sql, HDFS more »

Repo, Code Workbook, Pipeline Build, migration techniques, Data Connection and Security setup. Design, develop Data Pipelines, and have excellent skills in PySpark and Spark SQL, hands on with code Build and deployment in Palantir. Must lead a team of 6-7 technical associates with PySpark skills. more »