8 of 8 Apache Spark Jobs in West London

looking for A strong engineer who cares about performance, scalability and clean design. Key experience: Strong SQL and Python skills (pandas, NumPy, Spark) Hands-on experience with Apache Spark, Kafka, and Airflow Solid understanding of data warehousing and distributed computing Experience with Docker, Kubernetes, and CI/ ...

Responsibilities will include; Design, implement, and optimize end-to-end solutions using Fabric components: o Data Factory (pipelines, orchestration) o Data Engineering (Lakehouse, notebooks, Apache Spark) o Data Warehouse (SQL endpoints, schemas, MPP performance tuning) o Real-Time Analytics (KQL databases, event ingestion) o Manage and enhance OneLake ...

/or Computer Vision. Strong knowledge of Machine Learning algorithms and statistics. Experience with SQL/NoSQL databases and big data tools (Spark, Hadoop preferred). Experience with MLOps tools such as Docker, Kubernetes, CI/CD pipelines. Preferred Skills:- Experience with LLMs/Generative AI (OpenAI, Hugging Face ...

hedge fund, energy major or vendor serving these sectors. Knowledge of JavaScript frameworks, particularly React.js . Experience with Big Data technologies such as HDFS, Spark, Scala, Kudu, HBase, SQL, Impala , and traditional RDBMS. Quantitative skills in areas such as statistics, probability, optimization or derivative pricing. Some hands-on C++ ...

relevant in data governance, compliance, or automation programs. Strong proficiency in Python for data engineering and automation. Hands-on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet . Advanced SQL skills; experience with Oracle databases . Experience designing and supporting …/ELT pipelines and data quality frameworks . Working knowledge of Autosys & Apache Airflow Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift) Familiarity with AWS S3 for large-scale data storage. Experience supporting Tableau dashboards. Nice-to-Have Experience in regulated or enterprise ...

frameworks, scripting). Experience supporting or validating ETL/ELT pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of Autosys & Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift). Familiarity with … Experience supporting Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Ideally, you have experience with tools including PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Experience in regulated or enterprise data environments (investment banking, risk, finance, compliance). Exposure to data ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...

DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience Hands ...