1 to 25 of 42 Hadoop Jobs

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI, Qlik ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on coding experience, such ...

Scala, and SQL. Hands-on experience building and operating large-scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud-native data platforms, with strong expertise ...

NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team workloads Experience supporting ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI ...

traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices and CI/CD for data ...

.Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline lifecycle development, including extensive ...

business needs.Bonus Points If You Have:Certifications: Google Cloud or AWS Certified Solutions Architect certification.ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink.Data Architecture: Proven ability in designing and building ...

deployment and monitoring using CI/CD tools and infrastructure-as-code practices Hands-on expertise with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and real-time data streaming technologies (e.g., Kafka, Kinesis, Spark) Track record of collaborating with cross-functional teams (engineering, analytics, business stakeholders) to deliver ...

planningRequired Skills & ExperienceTechnical SkillsStrong proficiency in SQL and data modellingExcellent programming skills in Python, Scala, or JavaHands-on experience with distributed data frameworks (Spark, Hadoop)Experience with data pipelines and ETL toolsExpertise in at least one cloud platform:Azure (Synapse, Data Factory, Databricks)AWS (S3, Glue, Redshift, Lambda ...

query languages such as SQL, PL/SQL, Python, Java, Ruby or R.Experience with relevant modern data technologies such as BigQuery, Databricks, Snowflake, MongoDB, Hadoop, Redis, Kafka or similar.The ability to select technologies based on scalability, performance, security, interoperability and business requirements.Experience documenting complex architectures clearly through diagrams, workflows ...

software development lifecycleSolid background in Java/Scala and a scripting language like Python.Experience building large scale data pipelinesStrong familiarity with the Apache Hadoop ecosystem including : Spark, Kafka, Flink, Iceberg/Delta Lake/Hive, Apache Presto/Trino, etc.Experience with relational SQL and NoSQL databases, including Postgres/ ...

business needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache ...

warehousing principles and best practices Dimensional modelling Enterprise data architecture concepts Modernisation Experience Experience delivering migrations and modernisation initiatives involving technologies such as: Teradata Hadoop Oracle SAS Preferred Experience Financial Services or Banking sector experience. Experience working within regulated environments. Large-scale cloud transformation programmes. Agile delivery methodologies. Experience ...

analytical needs into scalable Databricks-based solutions.Your expertiseYou have:• proven experience with Azure and Databricks in large-scale data environments• strong understanding of Spark, Hadoop ecosystems and big data engineering concepts• hands-on data engineering experience• experience delivering at least one end-to-end Databricks project• strong knowledge ...

performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/CD, Infrastructure ...

performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/CD, Infrastructure ...

continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices, testing ...

enabled tools responsibly to improve productivity while maintaining appropriate human oversight, governance, and solution quality.Bonus Points ForDevelopment experience using Scala or Java.Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems.Experience working with sensitive data and applying data privacy regulations, including GDPR.Curiosity to explore new technologies, including responsible ...

Relational databases (SQL Server, PostgreSQL, Oracle, MySQL) Data visualisation tools (Power BI, Tableau, Spotfire) Cloud platforms, particularly Microsoft Azure Big data technologies (Spark, Elasticsearch, Hadoop) Statistical modelling and advanced analytics Machine learning and pattern recognition techniques Web technologies such as HTML and JavaScript ...

business objectives.A bit about you: Strong Python knowledge. Ability to read through code. Demonstrable experience collaborating with engineers.Experience with big data technologies such as Hadoop, Spark, or similar. Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and data warehousing solutions.Strong knowledge in at least a few of the following ...

including Cortex AI, data sharing, Snowpark, and Native AppsFamiliarity with financial services cloud migration patterns and the technical landscape of legacy platforms (Teradata, Oracle, Hadoop)Experience with real-time data architectures, streaming ingestion, and event-driven data platforms in regulated environmentsBackground in data mesh, data product design, or enterprise ...

preferred. Having built ELT or ETL pipelines processing large amounts of data. Experience with dbt or Airbyte is preferred. Experience developing production Spark or Hadoop applications Gained experience with a workload scheduling and orchestration tool such as Apache Airflow/Cloud Composer Familiarity with modern DataOps (e.g. quality ...

Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. You will understand ...

related fields. 5+ years of experience in Big Data; prior experience with Tier-1 Cloud Service Providers (CSPs) is highly preferred. Expertise in the Hadoop ecosystem and deep understanding of components like Spark, Flink, Elasticsearch, Doris, ClickHouse, and Trino., Strong knowledge of big data architecture, including Data Lake (Iceberg ...