176 to 200 of 213 Apache Spark Jobs in London

learning, data processing technologies and a broad set of AWS technologies. In order to drive the expansion of Amazon selection, we use cluster-computing technologies such as MapReduce and Spark to process billions of products and find the products/brands not already sold on Amazon. We work with structured and unstructured content such as text and images and More ❯

solutions that grow with the Amazon business. BDT team is building an enterprise-wide Big Data Marketplace leveraging AWS technologies. We work closely with AWS teams like EMR/Spark, Redshift, Athena, S3 and others. We are developing innovative products including the next-generation of data catalog, data discovery engine, data transformation platform, and more with state-of-the More ❯

are a skilled programmer with a strong command of major machine learning languages such as Python or Scala, and have expertise in utilising statistical and machine learning libraries like Spark MLlib, scikit-learn, or PyTorch to write clear, efficient, and well-documented code. Experience with optimisation techniques, control theory, causal modelling or elasticity modelling is desirable. Prior experience in More ❯

technology to solve a given problem. Right now, we use: • A variety of languages, including Java and Go for backend and TypeScript for frontend • Open-source technologies like Cassandra, Spark, Elasticsearch, React, and Redux • Industry-standard build tooling, including Gradle, CircleCI, and GitHub What We Value Passion for helping other developers build better applications. Empathy for the impact your More ❯

including technical design, coding standards, code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark - especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects Are familiar with Airflow or More ❯

you prefer Exceptional Benefits : From unlimited holiday and private healthcare to stock options and paid parental leave. What You'll Be Doing: Build and maintain scalable data pipelines using Spark with Scala and Java, and support tooling in Python Design low-latency APIs and asynchronous processes for high-volume data. Collaborate with Data Science and Engineering teams to deploy … Contribute to the development of Gen AI agents in-product. Apply best practices in distributed computing, TDD, and system design. What We're Looking For: Strong experience with Python, Spark, Scala, and Java in a commercial setting. Solid understanding of distributed systems (e.g. Hadoop, AWS, Kafka). Experience with SQL/NoSQL databases (e.g. PostgreSQL, Cassandra). Familiarity with More ❯

you prefer Exceptional Benefits : From unlimited holiday and private healthcare to stock options and paid parental leave. What You'll Be Doing: Build and maintain scalable data pipelines using Spark with Scala and Java, and support tooling in Python Design low-latency APIs and asynchronous processes for high-volume data. Collaborate with Data Science and Engineering teams to deploy … Contribute to the development of Gen AI agents in-product. Apply best practices in distributed computing, TDD, and system design. What We're Looking For: Strong experience with Python, Spark, Scala, and Java in a commercial setting. Solid understanding of distributed systems (e.g. Hadoop, AWS, Kafka). Experience with SQL/NoSQL databases (e.g. PostgreSQL, Cassandra). Familiarity with More ❯

It has come to our notice that Fractal Analytics' name and logo are being misused by certain unscrupulous persons masquerading as Fractal's authorized representatives to approach job seekers to part with sensitive personal information and/or money in More ❯

Familiarity with at least one of the following cloud providers: Amazon Web Services (AWS), Google Cloud Compute (GCP), or Microsoft Azure. Familiarity with at least one of the following: Spark, Hive, or Presto. Desirable Skills: Expertise with open standards in the identity space, including: OpenID, OpenID Connect (OIDC), OAuth, SAML, and LDAP. Familiarity with Scala programming language and popular … both object-oriented programming (OOP) and functional programming (FP) best practices. Familiarity with Amazon Web Services (AWS), Terraform, and infrastructure as code (IaC) best practices. Familiarity with Databricks and Apache Airflow products. Required Education: Bachelor's degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience. The More ❯

management disciplines, including data integration, modeling, optimisation, data quality and Master Data Management. Experience with database technologies such as RDBMS (SQL Server, Oracle) or NoSQL (MongoDB). Knowledge in Apache technologies such as Spark, Kafka and Airflow to build scalable and efficient data pipelines. Have worked on migration projects and some experience with management systems such as SAP More ❯

management disciplines, including data integration, modeling, optimisation, data quality and Master Data Management. Experience with database technologies such as RDBMS (SQL Server, Oracle) or NoSQL (MongoDB). Knowledge in Apache technologies such as Spark, Kafka and Airflow to build scalable and efficient data pipelines. Have worked on migration projects and some experience with management systems such as SAP More ❯

roles 5+ years of experience in big data technology with experience ranging from platform architecture, data management, data architecture and application architecture High Proficiency working with Hadoop platform including Spark/Scala, Kafka, SparkSQL, HBase, Impala, Hive and HDFS in multi-tenant environments Solid base in data technologies like warehousing, ETL, MDM, DQ, BI and analytical tools extensive experience … of distributed, fault-tolerant applications with attention to security, scalability, performance, availability and optimization Requirements 4+ years of hands-on experience in designing, building and supporting Hadoop Applications using Spark, Scala, Sqoop and Hive. Strong knowledge of working with large data sets and high capacity big data processing platform. Strong experience in Unix and Shell scripting. Experience using Source More ❯

. Manage and monitor the cost, efficiency, and speed of data processing. Our Data Tech Stack Azure Cloud (SQL Server, Databricks, Cosmos DB, Blob Storage) ETL/ELT (Python, Spark, SQL) Messaging (Service Bus, Event Hub) DevOps (Azure DevOps, Github Actions, Terraform) Who you are A driven, ambitious individual who's looking to build their career at an exciting … building and maintaining robust and scalable data pipelines Proficiency in ELT and ETL processes and tools Ability to write efficient code for data extraction, transformation, and loading (eg. Python, Spark and SQL) Proficiency with cloud platforms (particularly Azure Databricks and SQL Server) Ability to work independently Ability to communicate complex technical concepts clearly to both technical and non-technical More ❯

. Manage and monitor the cost, efficiency, and speed of data processing. Our Data Tech Stack Azure Cloud (SQL Server, Databricks, Cosmos DB, Blob Storage) ETL/ELT (Python, Spark, SQL) Messaging (Service Bus, Event Hub) DevOps (Azure DevOps, Github Actions, Terraform) Who you are A driven, ambitious individual who’s looking to build their career at an exciting … building and maintaining robust and scalable data pipelines Proficiency in ELT and ETL processes and tools Ability to write efficient code for data extraction, transformation, and loading (eg. Python, Spark and SQL) Proficiency with cloud platforms (particularly Azure Databricks and SQL Server) Ability to work independently Ability to communicate complex technical concepts clearly to both technical and non-technical More ❯

Data Engineer/Technical Support Engineer - Client Facing (Remote - UK) Location: 3 days per week in the office (Office in Sheffield, UK) Contract: 6-Month Contract Rate: £400 per day - Inside IR35 Role Overview: We are looking for a highly More ❯

Join us as a Machine Learning Engineer at Barclays, where you'll develop and implement AI tools using state-of-the-art technologies. As part of our Chief Technology Office, you'll work on testing and deploying the latest AI More ❯

equivalent education) in a STEM discipline. Proven experience in software engineering and development, and a strong understanding of computer systems and how they operate. Hands on Experience in Java , Spark , Scala ( or Java) Production scale hands-on Experience to write Data pipelines using Spark/any other distributed real time/batch processing. Strong skill set in SQL More ❯

leading innovative technical projects. As part of this role, you will be responsible for some of the following areas: Design and build distributed data pipelines using languages such as Spark, Scala, and Java Collaborate with cross-functional teams to deliver user-centric solutions Lead on the design and development of relational and non-relational databases Apply Gen AI tools … scale data collection processes Support the deployment of machine learning models into production To be successful in the role you will have: Creating scalable ETL jobs using Scala and Spark Strong understanding of data structures, algorithms, and distributed systems Experience working with orchestration tools such as Airflow Familiarity with cloud technologies (AWS or GCP) Hands-on experience with Gen More ❯

leading innovative technical projects. As part of this role, you will be responsible for some of the following areas: Design and build distributed data pipelines using languages such as Spark, Scala, and Java Collaborate with cross-functional teams to deliver user-centric solutions Lead on the design and development of relational and non-relational databases Apply Gen AI tools … scale data collection processes Support the deployment of machine learning models into production To be successful in the role you will have: Creating scalable ETL jobs using Scala and Spark Strong understanding of data structures, algorithms, and distributed systems Experience working with orchestration tools such as Airflow Familiarity with cloud technologies (AWS or GCP) Hands-on experience with Gen More ❯

Principal Data Engineer (Contract) Hybrid (London-based) | £450-£650/day A leading scientific and cultural institution is seeking a Principal Data Engineer to design and scale cloud-native data infrastructure powering biodiversity research, digitisation, and AI workflows. Key Responsibilities More ❯

Experience programming in Java, C++, Python or related language - Experience with neural deep learning methods and machine learning PREFERRED QUALIFICATIONS - Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc. - Experience with large scale distributed systems such as Hadoop, Spark etc. Amazon is an equal opportunity employer and does not discriminate on More ❯

Data Science teams in an eCommerce or conversion rate optimisation-focused environment is a plus. Hands-on experience with Machine & Deep Learning, AI and Neural Networks tools including Python, Spark, Tensor Flow. Competencies across core programming language including Python, Java, C/C++, R. That you can work in a cross-functional environment, managing stakeholders across multiple stakeholders and … and flows. Advanced Mathematics skills including experience with Bayesian statistics, linear algebra and MVT calculus, advanced data modelling and algorithm design experience. Design and deployment experience using Tensor Flow, Spark ML, CNTK, Torch or Caffe. The perks A flexible environment, that allows 1-2 days of remote working per week. 28 days holiday + a competitive pension scheme. Private More ❯

culture of innovation Collaborate with Product Managers to define ambiguous problem spaces, co-create requirements, and drive experimentation Build and optimize production-grade ML pipelines using tools like Databricks, Spark, and cloud-native infrastructure Apply a wide range of ML techniques including deep learning, NLP, ranking, clustering, and forecasting Translate complex business needs into scalable ML solutions that directly … related NLP technologies Proficiency in Python and/or Scala; experience with ML libraries such as TensorFlow, PyTorch, HuggingFace, or scikit-learn Experience with Databricks, distributed data systems (e.g., Spark, Hadoop), and cloud platforms (AWS, GCP, or Azure) Ability to thrive in ambiguous environments, working closely with cross-functional teams to define and deliver impactful solutions Strong communication skills More ❯

deploying models in production and adjusting model thresholds to improve performance Experience designing, running, and analyzing complex experiments or leveraging causal inference designs Experience with distributed tools such as Spark, Hadoop, etc. A PhD or MS in a quantitative field (e.g., Statistics, Engineering, Mathematics, Economics, Quantitative Finance, Sciences, Operations Research) Hybrid work at Stripe Office-assigned Stripes spend at … deploying models in production and adjusting model thresholds to improve performance Experience designing, running, and analyzing complex experiments or leveraging causal inference designs Experience with distributed tools such as Spark, Hadoop, etc. A PhD or MS in a quantitative field (e.g., Statistics, Engineering, Mathematics, Economics, Quantitative Finance, Sciences, Operations Research) Hybrid work at Stripe Office-assigned Stripes spend at More ❯

The Trade Desk is a global technology company with a mission to create a better, more open internet for everyone through principled, intelligent advertising. Handling over 1 trillion queries per day, our platform operates at an unprecedented scale. We have More ❯