1 to 25 of 27 Hadoop Jobs

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on coding experience, such ...

Scala, and SQL. Hands-on experience building and operating large-scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud-native data platforms, with strong expertise ...

NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team workloads Experience supporting ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI ...

traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices and CI/CD for data ...

.Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline lifecycle development, including extensive ...

deployment and monitoring using CI/CD tools and infrastructure-as-code practices Hands-on expertise with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and real-time data streaming technologies (e.g., Kafka, Kinesis, Spark) Track record of collaborating with cross-functional teams (engineering, analytics, business stakeholders) to deliver ...

planningRequired Skills & ExperienceTechnical SkillsStrong proficiency in SQL and data modellingExcellent programming skills in Python, Scala, or JavaHands-on experience with distributed data frameworks (Spark, Hadoop)Experience with data pipelines and ETL toolsExpertise in at least one cloud platform:Azure (Synapse, Data Factory, Databricks)AWS (S3, Glue, Redshift, Lambda ...

warehousing principles and best practices Dimensional modelling Enterprise data architecture concepts Modernisation Experience Experience delivering migrations and modernisation initiatives involving technologies such as: Teradata Hadoop Oracle SAS Preferred Experience Financial Services or Banking sector experience. Experience working within regulated environments. Large-scale cloud transformation programmes. Agile delivery methodologies. Experience ...

analytical needs into scalable Databricks-based solutions.Your expertiseYou have:• proven experience with Azure and Databricks in large-scale data environments• strong understanding of Spark, Hadoop ecosystems and big data engineering concepts• hands-on data engineering experience• experience delivering at least one end-to-end Databricks project• strong knowledge ...

continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices, testing ...

Relational databases (SQL Server, PostgreSQL, Oracle, MySQL) Data visualisation tools (Power BI, Tableau, Spotfire) Cloud platforms, particularly Microsoft Azure Big data technologies (Spark, Elasticsearch, Hadoop) Statistical modelling and advanced analytics Machine learning and pattern recognition techniques Web technologies such as HTML and JavaScript ...

including Cortex AI, data sharing, Snowpark, and Native AppsFamiliarity with financial services cloud migration patterns and the technical landscape of legacy platforms (Teradata, Oracle, Hadoop)Experience with real-time data architectures, streaming ingestion, and event-driven data platforms in regulated environmentsBackground in data mesh, data product design, or enterprise ...

preferred. Having built ELT or ETL pipelines processing large amounts of data. Experience with dbt or Airbyte is preferred. Experience developing production Spark or Hadoop applications Gained experience with a workload scheduling and orchestration tool such as Apache Airflow/Cloud Composer Familiarity with modern DataOps (e.g. quality ...

Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. You will understand ...

related fields. 5+ years of experience in Big Data; prior experience with Tier-1 Cloud Service Providers (CSPs) is highly preferred. Expertise in the Hadoop ecosystem and deep understanding of components like Spark, Flink, Elasticsearch, Doris, ClickHouse, and Trino., Strong knowledge of big data architecture, including Data Lake (Iceberg ...

troubleshoot software. What you will need: Bachelor's degree in computer science or related discipline required. Experience with distributed data processing systems (e.g. Hadoop, Kafka). Hands on experience writing code with modern procedural programming languages such as Java and Python. Good understanding of design concepts and ability ...

improve collaboration and delivery.What We’re Looking ForStrong proficiency in Python, Scala, or Java with hands-on experience in Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.Practical experience building real-time event streaming pipelines using technologies such as Kafka, Spark Streaming, or Amazon Kinesis within … multidisciplinary teams and client environments.Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets.Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation.Knowledge of automating cloud-based ingestion and transformation frameworks.Experience delivering data engineering solutions within highly regulated industries ...

Job Description As an Advanced Data Engineer within the Data Engineering Unit, you will be an integral part of our team, focusing on designing, building, and maintaining scalable data pipelines and architectures. You will work ...

/or Parity Experience in using various Solidity frameworks like Truffle, Mix IDE or Intellij-Sol. Contributions to relevant open source projects; Experience Hadoop, Spark, or Docker. Basic or deep knowledge of Ethereum Whisper or IPFS/Swarm.My client is based in the City of LondonPython, Ethereum, Blockchain ...

development experience ideally purely in RubyExperience designing and building Rails application could be at homeRubyJavascriptHTML5CSS3Working knowledge of LinuxExperience with mobile applications would be beneficialCurrently Hadoop is used but not requiredThe environment is that of Facebook or Google, relaxed open with time to think and make the right decisions. ...

Experience with building highly scalable distributed systems and batch processing pipelines with a focus on data integrity Experience with multiple data storage technologies (MySQL, Hadoop, Cassandra) Experience with AWS (DynamoDB, RDS, Step functions, EC2, Kinesis) is a plus. Experience with unit, integration, and end-to-end testing paradigms Experience ...

scalable ELT/ETL flows (batch and streaming) Interface between Data Analysts and Data Scientists Enrich data and load into big data environment (Apache Hadoop on Cloudera) Profil Technical education (Computer Science HTL, Computer Science Degree, Data Science etc.) Experience in data modeling (relational databases and Apache Hadoop) Know-how of core technologies such as SQL (MS SQL Server), Apache Hadoop (Kafka, NiFi, Flink, Scala and/or Java, Python) and Linux Showing interest in working with high-frequency data processing in a near-realtime environment Wir bieten State-of-the-Art Technologies ...

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time ...

systems. Exposure to AI, Machine Learning or Large Language Model solutions. Experience validating AI-driven classification, routing or decision-making workflows. Experience with Oracle, Hadoop and Elasticsearch. Familiarity with observability and monitoring platforms. Knowledge of enterprise AI platforms and LLM integrations such as Orchestra, Llama or ChatGPT based solutions. … Technology Environment: Playwright LoadRunner JMeter Selenium REST Assured Java-based applications Oracle Hadoop Elasticsearch Git CI/CD Pipelines AI/LLM Platforms Agile Delivery At mthree, our values support courageous teammates, needle movers, and learning champions all while striving to support the health and well-being ...