1 to 25 of 72 Hadoop Jobs in London

data design and data modeling experience, and advanced SQL across large, complex datasets.Extensive experience with big data technologies and cloud data warehouses (Snowflake, BigQuery, Hadoop/Hive) at terabyte-and-growing scale.Experience building reusable frameworks, platforms, or tooling that other engineers depend on (not just point solutions).Hands ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI, Qlik ...

Azure, GCP).Knowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon).Familiarity with data lake architectures and distributed processing frameworks (e.g., Spark, Hadoop).Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelines.Understanding of data governance, security, and compliance frameworks.Exposure to AI/ML workloads ...

traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices and CI/CD for data ...

Azure, GCP) and their AI/ML services.Data Engineering: Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL, data pipelines, and automated workflows for moving and processing data.Data Governance: Adherence to data governance, security, and privacy standards throughout ...

.Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline lifecycle development, including extensive ...

Bonus Points If You Have: Certifications: Google Cloud or AWS Certified Solutions Architect certification. ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink. Data Architecture: Proven ability in designing ...

business needs.Bonus Points If You Have:Certifications: Google Cloud or AWS Certified Solutions Architect certification.ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink.Data Architecture: Proven ability in designing and building ...

similar languages relevant to data manipulation and scripting.Data Engineering: Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL/ELT, data warehousing, and automated workflows for moving and processing data.Data Governance: Strong understanding and practical experience with data governance ...

Python, or similar languages relevant to data manipulation and scripting.Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL/ELT, data warehousing, and automated workflows for moving and processing data.Familiarity with data modelling techniques (e.g., dimensional, relational) and data ...

deployment.Deep understanding of Quantexa components such as:Entity Resolution (ER)Scoring FrameworkContextual Network GenerationUI ConfigurationStrong background in Big Data technologies, including:Scala, Apache Spark, Hadoop, ElasticSearchExperience with data modelling and design in the context of large-scale analytics platforms.Strong stakeholder management and communication skills – capable of interfacing with both ...

learning or data science.Cloud certifications (e.g. Azure, GCP, AWS) and relevant AI/ML credentials.Experience with big data and distributed computing (e.g. Spark, Databricks, Hadoop or cloud-native alternatives).Experience with containerisation and DevOps/MLOps tools (e.g. Docker, Kubernetes, Git, CI/CD, MLflow, Airflow).Strong experience with ...

degree, or experience in a professional field or military- Experience as technical specialist in design and architecture- Experience in database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis)- Experience in consulting, design and implementation of serverless distributed solutions- Experience in cloud based solution (AWS or equivalent), system, network and operating ...

technologies in scope include:Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/CD tooling, containerised or Kubernetes-based deployment approaches where relevantYou will also work with internal data management ...

software development lifecycleSolid background in Java/Scala and a scripting language like Python.Experience building large scale data pipelinesStrong familiarity with the Apache Hadoop ecosystem including : Spark, Kafka, Flink, Iceberg/Delta Lake/Hive, Apache Presto/Trino, etc.Experience with relational SQL and NoSQL databases, including Postgres/ ...

from concept to production.Strong knowledge of Python and core ML libraries (e.g., spaCy, NumPy, SciPy, Transformers).Understanding of big data and cloud technologies (Spark, Hadoop, Azure, AWS, or GCP).Deep expertise in MLOps principles, scalable model deployment, model governance, and the practical application of responsible AI within enterprise environments.Excellent ...

analytical needs into scalable Databricks-based solutions.Your expertiseYou have:• proven experience with Azure and Databricks in large-scale data environments• strong understanding of Spark, Hadoop ecosystems and big data engineering concepts• hands-on data engineering experience• experience delivering at least one end-to-end Databricks project• strong knowledge ...

Experience with data warehousing concepts and technologies.Knowledge of other BI tools (e.g., Tableau, Looker) and data modelling techniques.Familiarity with big data technologies (e.g., Hadoop, Spark).Familiarity with modern data architecture (e.g., Data Mesh, Fabric).An understanding of infrastructure as code (e.g., Terraform)Experience with CI/CD pipelines ...

/LLMOps principles, including CI/CD for ML, model monitoring, and governance frameworks.Proficiency with large-scale data processing and storage technologies (SQL, Spark, Hadoop) is a plus.Excellent stakeholder management and communication skills, with proven ability to translate complex AI concepts for diverse audiences.You can understand client challenges ...

enabled tools responsibly to improve productivity while maintaining appropriate human oversight, governance, and solution quality.Bonus Points ForDevelopment experience using Scala or Java.Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems.Experience working with sensitive data and applying data privacy regulations, including GDPR.Curiosity to explore new technologies, including responsible ...

Azure, GCP or equivalent) and their AI/ML services.Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop) is a plus.Experience with MLOps tools and best practices for model deployment, monitoring, and governance.Education and Experience:Bachelor's or Master's, or equivalent degree ...

country where you are applying, as well as apply for travel visas as required by travel needsPreferred qualifications • Additional data and analytics experience in Hadoop framework and coding using Impala, Hive, or PySpark • Experience generating new knowledge or creating innovative solutions for a firm • Relevant industry expertise • Working knowledge ...

experience (focus on machine learning): SQL and Python’s Data Science stack; good knowledge of at least one big data framework (e.g., PySpark, Hive, Hadoop) is a plus. R, SPSS, and SAS are considered nice-to-haveStrong understanding of machine learning methods and experience applying them to complex, data ...

such as Python or Go.Expertise in constructing highly available and fault-tolerant big data environments in the cloud, utilizing NoSQL and RDMS technologies like Hadoop, Spark, PostgreSQL, and cloud-based management tools such as EMR, Athena, and RDS.Ability to implement systems with a security-first mindset, adhering to firm ...

complex business requirements and render them as prototype systems with quick turnaround time.Implementation and tuning experience in the Big Data Ecosystem, (such as EMR, Hadoop, Spark, R, Presto, Hive), Database (such as Oracle, MySQL, PostgreSQL, MS SQL Server), NoSQL (such as DynamoDB, HBase, MongoDB, Cassandra, design principles) and Data ...