1 to 25 of 30 Hadoop Jobs in London

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI, Qlik ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on coding experience, such ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI ...

traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices and CI/CD for data ...

.Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline lifecycle development, including extensive ...

business needs.Bonus Points If You Have:Certifications: Google Cloud or AWS Certified Solutions Architect certification.ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink.Data Architecture: Proven ability in designing and building ...

deployment and monitoring using CI/CD tools and infrastructure-as-code practices Hands-on expertise with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and real-time data streaming technologies (e.g., Kafka, Kinesis, Spark) Track record of collaborating with cross-functional teams (engineering, analytics, business stakeholders) to deliver ...

planningRequired Skills & ExperienceTechnical SkillsStrong proficiency in SQL and data modellingExcellent programming skills in Python, Scala, or JavaHands-on experience with distributed data frameworks (Spark, Hadoop)Experience with data pipelines and ETL toolsExpertise in at least one cloud platform:Azure (Synapse, Data Factory, Databricks)AWS (S3, Glue, Redshift, Lambda ...

query languages such as SQL, PL/SQL, Python, Java, Ruby or R.Experience with relevant modern data technologies such as BigQuery, Databricks, Snowflake, MongoDB, Hadoop, Redis, Kafka or similar.The ability to select technologies based on scalability, performance, security, interoperability and business requirements.Experience documenting complex architectures clearly through diagrams, workflows ...

software development lifecycleSolid background in Java/Scala and a scripting language like Python.Experience building large scale data pipelinesStrong familiarity with the Apache Hadoop ecosystem including : Spark, Kafka, Flink, Iceberg/Delta Lake/Hive, Apache Presto/Trino, etc.Experience with relational SQL and NoSQL databases, including Postgres/ ...

warehousing principles and best practices Dimensional modelling Enterprise data architecture concepts Modernisation Experience Experience delivering migrations and modernisation initiatives involving technologies such as: Teradata Hadoop Oracle SAS Preferred Experience Financial Services or Banking sector experience. Experience working within regulated environments. Large-scale cloud transformation programmes. Agile delivery methodologies. Experience ...

analytical needs into scalable Databricks-based solutions.Your expertiseYou have:• proven experience with Azure and Databricks in large-scale data environments• strong understanding of Spark, Hadoop ecosystems and big data engineering concepts• hands-on data engineering experience• experience delivering at least one end-to-end Databricks project• strong knowledge ...

enabled tools responsibly to improve productivity while maintaining appropriate human oversight, governance, and solution quality.Bonus Points ForDevelopment experience using Scala or Java.Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems.Experience working with sensitive data and applying data privacy regulations, including GDPR.Curiosity to explore new technologies, including responsible ...

Relational databases (SQL Server, PostgreSQL, Oracle, MySQL) Data visualisation tools (Power BI, Tableau, Spotfire) Cloud platforms, particularly Microsoft Azure Big data technologies (Spark, Elasticsearch, Hadoop) Statistical modelling and advanced analytics Machine learning and pattern recognition techniques Web technologies such as HTML and JavaScript ...

business objectives.A bit about you: Strong Python knowledge. Ability to read through code. Demonstrable experience collaborating with engineers.Experience with big data technologies such as Hadoop, Spark, or similar. Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and data warehousing solutions.Strong knowledge in at least a few of the following ...

including Cortex AI, data sharing, Snowpark, and Native AppsFamiliarity with financial services cloud migration patterns and the technical landscape of legacy platforms (Teradata, Oracle, Hadoop)Experience with real-time data architectures, streaming ingestion, and event-driven data platforms in regulated environmentsBackground in data mesh, data product design, or enterprise ...

Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. You will understand ...

related fields. 5+ years of experience in Big Data; prior experience with Tier-1 Cloud Service Providers (CSPs) is highly preferred. Expertise in the Hadoop ecosystem and deep understanding of components like Spark, Flink, Elasticsearch, Doris, ClickHouse, and Trino., Strong knowledge of big data architecture, including Data Lake (Iceberg ...

languages such as Python, SQL, or SparkExpertise in data visualisation tools such as Qlik, Power BI (Business Intelligence) or TableauData engineering tools such as Hadoop, Kafka, SQLExperience of analytics and being able to review and present insights in large data sets formed of complex data, including with software such ...

following would be beneficial but not essential:Working with big data technology, ideally Spark but others will also be useful such as Hadoop or ElasticsearchExperience with DevOps tooling and approaches (automation and tooling is something we are passionate about, we even have a dedicated Developer Tooling team)Building data ...

enhance collaboration and delivery.What We’re Looking ForStrong proficiency in Python, Scala, or Java, with hands-on experience of Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.Experience building real-time event streaming solutions using technologies such as Kafka, Spark Streaming, or Amazon Kinesis within AWS environments.Proven … practical, high-quality technical solutions.Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets.Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation.Knowledge of automating ingestion and transformation frameworks within cloud-native data platforms.Experience delivering solutions within highly regulated ...

improve collaboration and delivery.What We're Looking ForStrong proficiency in Python, Scala, or Java with hands-on experience in Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.Practical experience building real-time event streaming pipelines using technologies such as Kafka, Spark Streaming, or Amazon Kinesis within … multidisciplinary teams and client environments.Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets.Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation.Knowledge of automating cloud-based ingestion and transformation frameworks.Experience delivering data engineering solutions within highly regulated industries ...

/or Parity Experience in using various Solidity frameworks like Truffle, Mix IDE or Intellij-Sol. Contributions to relevant open source projects; Experience Hadoop, Spark, or Docker. Basic or deep knowledge of Ethereum Whisper or IPFS/Swarm.My client is based in the City of LondonPython, Ethereum, Blockchain ...

multi-threaded JavaAble to build to develop high throughput services where performance mattersExperienced with continuous integrationA strong communicatorAdditional exposure to the following is desired:Hadoop and FlinkJunit 5, Mockito, AvroLinux, JenkinsDocker, Kubernetes, PuppetInvolvement in the Java communityMy client is based London. ...

development experience ideally purely in RubyExperience designing and building Rails applicationsRubyJavascriptWorking knowledge of LinuxUnderstanding of Applications on AWSExperience with mobile applications would be beneficialCurrently Hadoop is used but not requiredThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...