1 to 25 of 34 Permanent Apache Spark Jobs in East London

data models, ETL processes, and data warehousing solutions.Programming: Utilize Python, Java, Scala, or GoLang to build and optimize data pipelines.Distributed Processing: Work with Hadoop, Spark, and other platforms for large-scale data processing.Real-Time Data Streaming: Develop and manage pipelines using CDC, Kafka, and Apache Spark.Database Management: Handle more »

Job DescriptionMy client are looking for a talented and motivated Big Data Architect (Azure, Databricks, Spark) to be based in their London office. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives, as they provide increasing … driving improvements in design, processes, and implementation to improve operational management, scalability, and extensibility.The following skills/experience is essential:Strong implementation experience using Spark and DatabricksStrong Cloud experience (ideally Azure)Previously heavily involved in an implementation programmeData WarehouseStrong stakeholder management experienceExcellent IT background, ideally within Telco/Wireless more »

within either Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data qualityPreparing data for predictive and prescriptive modellingHands on coding experience more »

processes, and data warehousing.- Significant exposure and hands on at least 2 of the programming languages - Python, Java, Scala, GoLang.- Significant experience with Hadoop, Spark and other distributed processing platforms andframeworks.- Experience working with Open table/storage formats like delta lake, apache iceberg orapache hudi.- Experience of … developing and managing real time data streaming pipelines using Changedata capture (CDC), Kafka and Apache Spark.- Experience with SQL and database management systems such as Oracle, MySQL orPostgreSQL.- Strong understanding of data governance, data quality, data contracts, and data securitybest practices.- Exposure to data governance, catalogue, lineage and associated more »

of the complex issues they are facing.out data-driven analysis, craft solutions to resolve business problems.Artificial Intelligence and data science approaches (Python, R, Matlab, Spark etc).database technologies such as Hadoop.tools that expand the companies tool kit, advancing their ability to serve clients.Experience neededStrong consulting backgroundcoaching and mentoring people more »

of databases. Snowflake is widely used, as are Docker and Kubernetes for containerisation. ETL and ELT tech are also used every day, primarily Airflow, Spark, Hive and a lot more.You’ll need to come from a strong academic background with some commercial experience in a data heavy software or more »

Analytics PlatformsMust have 8+ years' Experience with Relational Databases like Oracle, NoSQL Databases and/or Big Data technologies (e.g. Oracle, SQL Server, Postgres, Spark, Hadoop, other Open Source). Must have experience in Data Security Solutions (Identity and Access Management and Data Security Access Management)Must have 3+ more »

Sector, they are on the lookout for 2 AWS Data Engineers to come in on a contract basis.Key Skills/Requirements:Must have Python & Spark experienceMust have strong AWS experienceMust have Terraform experienceSQL & NoSQL experienceHave built out Data Warehouses & built Data PipelinesStrong Databricks & Snowflake experienceDocker, ECS, Kubernetes & Orchestration tools more »

data engineering or a similar role.> Proficiency in programming languages such as Python, Java, or Scala.> Strong experience with data processing frameworks such as Apache Spark, Apache Flink, or Hadoop.> Hands-on experience with cloud platforms such as AWS, Google Cloud, or Azure.> Experience with data warehousing more »

and best practices, and share knowledge with the team.QualificationsYou will have expertise within the following:Java and Python development knowledge (Essential)Previous experience with Spark or Hadoop(Essential)Trino orAirflow(Desirable)Architecture and capabilities.Designing and implementing complex solutions with a focus on scalability and security.Excellent communication and collaboration skills.Additional more »

given requirement.Ensure to develop unit test cases.Help in backlog grooming.Key skills: Extensive experience in developing Bigdata pipelines in cloud using Bigdata technologies such as Apache SparkExpertise in performing complex data transformation using Spark SQL queriesExperience in orchestrating data pipelines using Apache AirflowProficiency in Git based version control more »

Job DescriptionLead Data Engineer (Director) - Individual contributor - Azure, Data Factory, Databricks, Apache Spark - London Based I am hiring for a Lead Data Engineer for a crucial role within one of my Investment Bank clients in London. This role is at Director level as they require a very senior … Responsibilities: Leading data engineering practicesSupport current applications Introduce AI practices to the team/project Communicate key successes with stakeholders Key Skills: Azure Databricks Apache Spark Datascience, AI, ML Certifications or continued upskilling/contribution to blog posts within Data & AI beneficial but not essential. This is a … UK without sponsorship, if you are interested please apply or email me directly - aaron.dhammi@nicollcurtin.comLead Data Engineer (Director) - Individual contributor - Azure, Data Factory, Databricks, Apache Spark - London Based more »

in cloud development (ideally AWS)Knowledge and ideally hands-on experience with data streaming, event-based architectures and KafkaStrong communication and interpersonal skillsExperience with Apache Spark or Apache Flink would be ideal, but not essentialPlease note, this role is unable to provide sponsorship.If this role sounds of more »

s Degree in Computer Science, Engineering (or other related STEM subject)5+ years experience in data engineering2+ years in a leadership role.Experience working with Apache Spark, Azure Data Factory and other data pipelines tools.Strong programming skills.Impeccable communication skills.Precise attention to detail.Pioneering attitude.If you are a Lead Data Engineer more »

high level within a typical retail trading environment is key.Experience required:A background in leveraging hands on skills using tools such as Python, R, Spark, Hadoop, SQL and cloud based platforms such as GCP, Azure and AWS to manipulate and analyse various data sets in large volumesBackground in data more »

a team dynamic.Knowledge and understanding of OTC products (Interest Rate Swaps, Variance Swaps, CDS, etc.) bookings.Familiarity with C++ and Big Data tools such as Spark, Kafka, Elastic.Join us and be part of a team that values innovation, collaboration, and excellence. Take your career to new heights with a leading more »

as a Lead Big Data Engineer with excellent knowledge of Big Data-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of data types-Very strong knowledge of database technologies such as NoSQL, Relational more »

of Big Data-Great understanding of Cloud e.g. Azure and or AWS-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of data types-Very strong knowledge of database technologies such as NoSQL, Relational more »

ll also get exposure to Python, lots of SQL (of course) and depending on your level of experience, data stream processing tools like Kafka, Spark, etc. As this company continues to build new platforms and modernise, you’ll also be exposed to the cloud and various other modern tools more »

as TensorFlow, PyTorch, scikit-learn, and Spark.Solid understanding of statistical analysis, data mining, and predictive modeling techniques.Experience with big data technologies such as Hadoop, Spark, and distributed computing frameworks.What is in it for you?Significant Bonus Package. Flexible working for work/life balance.Private healthcare package.Significant pension contribution package.Choice more »

actionable insights for business improvementsLead a small team of data scientist on Neural Networks LLMs (CNN & RNN), ML, & NLPNLP/AI/ML/Spark/Python/Data scientist/Machine Learning Engineer/OCR/Deep LearningRequirementsBachelor's degree or equivalent experience in quantitative field (Statistics, Mathematics more »

an Architect and excellent knowledge of Big Data-Excellence experience across Azure-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of data types-Very strong knowledge of database technologies such as NoSQL, Relational more »

growing business.Requirements:Strong experience with Python (2+ Years)Experience working with REST MicroservicesStrong SQLExperience working with very large data sets.Knowledge of big data tools (Spark, Kafka etc)Experience working in finance (Preferred)Strong formal education - ideally in Computer ScienceIf this sounds of interest, then please do not hesitate to more »

of the complex issues they are facing.out data-driven analysis, craft solutions to resolve business problems.Artificial Intelligence and data science approaches (Python, R, Matlab, Spark etc).database technologies such as Hadoop.tools that expand the companies tool kit, advancing their ability to serve clients.Experience needed-in a consulting environment.coaching and more »

essential:-Proven experience as an Architect and excellent knowledge of Big Data-Excellent knowledge of Hadoop and tools such as Hbase/Hive and Spark etc-Excellent experience of ETL, data warehousing and handling a variety of data types-Very strong knowledge of database technologies such as NoSQL, Relational more »