1 to 25 of 108 Hadoop Jobs

implementing complex ETL/ELT processes from concept to production.Proficiency in SQL and data exploration across large, complex datasets.Experience with big data technologies (Hadoop, Hive, BigQuery, Snowflake) and working with terabyte-scale or larger datasets.Solid foundation in data structures, algorithms, and object-oriented design.Professional experience building REST services ...

Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data qualityPreparing data for predictive and prescriptive modellingHands on coding experience, such as PythonReporting ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on coding experience, such ...

Scala, and SQL. Hands-on experience building and operating large-scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud-native data platforms, with strong expertise ...

Azure, GCP).Knowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon).Familiarity with data lake architectures and distributed processing frameworks (e.g., Spark, Hadoop).Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelines.Understanding of data governance, security, and compliance frameworks.Exposure to AI/ML workloads ...

NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team workloads Experience supporting ...

driven architectures Airflow dbt Git/GitHub CI/CD Infrastructure as Code Modern data platforms and distributed processing technologies Experience with Databricks or Hadoop-style data processing environments is useful, although candidates with strong transferable experience across modern data platforms will be considered. Customer Data & Retail Context Experience ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools (e.g. Tableau, PowerBI ...

traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices and CI/CD for data ...

Azure, GCP) and their AI/ML services.Data Engineering: Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL, data pipelines, and automated workflows for moving and processing data.Data Governance: Adherence to data governance, security, and privacy standards throughout ...

.Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline lifecycle development, including extensive ...

business needs.Bonus Points If You Have:Certifications: Google Cloud or AWS Certified Solutions Architect certification.ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink.Data Architecture: Proven ability in designing and building ...

Azure, GCP) and their AI/ML services.Data Engineering: Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL, data pipelines, and automated workflows for moving and processing data.Data Governance: Adherence to data governance, security, and privacy standards throughout ...

similar languages relevant to data manipulation and scripting.Data Engineering: Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL/ELT, data warehousing, and automated workflows for moving and processing data.Data Governance: Strong understanding and practical experience with data governance ...

Python, or similar languages relevant to data manipulation and scripting.Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL/ELT, data warehousing, and automated workflows for moving and processing data.Familiarity with data modelling techniques (e.g., dimensional, relational) and data ...

deployment.Deep understanding of Quantexa components such as:Entity Resolution (ER)Scoring FrameworkContextual Network GenerationUI ConfigurationStrong background in Big Data technologies, including:Scala, Apache Spark, Hadoop, ElasticSearchExperience with data modelling and design in the context of large-scale analytics platforms.Strong stakeholder management and communication skills – capable of interfacing with both ...

Quantexa components such as: Entity Resolution (ER) Scoring Framework Contextual Network Generation UI Configuration Strong background in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable of interfacing ...

planningRequired Skills & ExperienceTechnical SkillsStrong proficiency in SQL and data modellingExcellent programming skills in Python, Scala, or JavaHands-on experience with distributed data frameworks (Spark, Hadoop)Experience with data pipelines and ETL toolsExpertise in at least one cloud platform:Azure (Synapse, Data Factory, Databricks)AWS (S3, Glue, Redshift, Lambda ...

learning or data science.Cloud certifications (e.g. Azure, GCP, AWS) and relevant AI/ML credentials.Experience with big data and distributed computing (e.g. Spark, Databricks, Hadoop or cloud-native alternatives).Experience with containerisation and DevOps/MLOps tools (e.g. Docker, Kubernetes, Git, CI/CD, MLflow, Airflow).Strong experience with ...

degree, or experience in a professional field or military- Experience as technical specialist in design and architecture- Experience in database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis)- Experience in consulting, design and implementation of serverless distributed solutions- Experience in cloud based solution (AWS or equivalent), system, network and operating ...

following areas:Provide subject matter expertise and hands-on delivery of data capture, curation, and consumption pipelines on cloud platforms using native and Hadoop-based services.Ability to conduct comprehensive technical and architectural assessments of existing platforms, define future-state architectures, and articulate the migration strategy.Ability to design and build ...

query languages such as SQL, PL/SQL, Python, Java, Ruby or R.Experience with relevant modern data technologies such as BigQuery, Databricks, Snowflake, MongoDB, Hadoop, Redis, Kafka or similar.The ability to select technologies based on scalability, performance, security, interoperability and business requirements.Experience documenting complex architectures clearly through diagrams, workflows ...

technologies in scope include:Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/CD tooling, containerised or Kubernetes-based deployment approaches where relevantYou will also work with internal data management ...

software development lifecycleSolid background in Java/Scala and a scripting language like Python.Experience building large scale data pipelinesStrong familiarity with the Apache Hadoop ecosystem including : Spark, Kafka, Flink, Iceberg/Delta Lake/Hive, Apache Presto/Trino, etc.Experience with relational SQL and NoSQL databases, including Postgres/ ...

business needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache ...