1 to 25 of 56 Hadoop Jobs

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on coding experience, such ...

Scala, and SQL. Hands on experience building and operating large scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud native data platforms, with strong expertise ...

Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on coding experience, such ...

Quantexa components such as: Entity Resolution (ER), Scoring Framework, Contextual Network Generation, UI Configuration. Strong background in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch. Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable of interfacing ...

/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Tableau, PowerBI, Qlik) Experience ...

closely with stakeholders and translate business requirements into effective AI solutions Desirable Generative AI experience Large Language Models (LLMs) LangChain or similar frameworks Spark, Hadoop or other big data technologies Experience with distributed computing, streaming platforms or workflow orchestration tools Experience monitoring and optimising model performance in production environments ...

data science. Cloud certifications (e.g. Azure, GCP, AWS) and relevant AI/ML credentials. Experience with big data and distributed computing (e.g. Spark, Databricks, Hadoop or cloud-native alternatives). Experience with containerisation and DevOps/MLOps tools (e.g. Docker, Kubernetes, Git, CI/CD, MLflow, Airflow). Strong ...

Experience to Docker and APIs Familiarity with CI/CD practices Interest in Agile and DevOps practices. Exposure to big data tools e.g. Spark, Hadoop, NiFi Understanding of ML algorithms and deployment This is not an exhaustive list, and we are keen to hear from you even ...

Data Platforms: BigQuery, Snowflake, Redshift, Databricks, PostgreSQL, MySQL, MongoDB, or other SQL/NoSQL technologies Data Tooling: Airflow, dbt, AWS Glue, Step Functions, Spark, Hadoop, or similar tooling Infrastructure: AWS, Terraform, Docker, Kubernetes, or other cloud/platform technologies CI/CD: GitHub Actions, Jenkins, ArgoCD, or similar deployment ...

Oracle, Snowflake, Databricks).* Experience with data integration (ETL/ELT, APIs).* Familiarity with cloud data platforms (AWS, Azure, GCP) and big data (Hadoop, Spark, Kafka) a plus.Domain Knowledge:* Exceptional understanding of the Funds business lifecycle, financial instruments, market data, and investment operations.* In-depth knowledge of regulatory ...

/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record with data ...

/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record with data ...

detail.* Strong partner management and ability to influence across teams and organisations.* Continuous learning and improvement mentality.## **Desired*** Experience with big data technologies (e.g. Hadoop, Hive, Spark).* Experience with generative AI, LLMs, or retrieval-augmented generation (RAG).* Exposure to Agentic AI concepts, including autonomous agents, tool ...

performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/CD, Infrastructure ...

performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable: CI/CD, Infrastructure ...

continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices, testing ...

/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record with data ...

addition to the above, we would be thrilled if you also had: Experience of working on migrations of enterprise scale data platforms including Hadoop and traditional data warehouses An understanding of machine learning model development lifecycle, feature engineering, training and testing Good understanding or hands‐on experience of Kafka ...

infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes) and the different ...

predictive models using Python and ML libraries and managed their lifecycle, from development to production. • Experienced Big data and distributed computing platforms such as Hadoop, Spark, Docker, Kubernetes, and Airflow. • Experience working in an agile environment. • Evaluate vendor and opensource software, write proof of concepts and design new systems. ...

experience with MLOps, production ML systems, model lifecycle management, and monitoring. Experience working with large-scale data systems and distributed computing frameworks (e.g. Spark, Hadoop). Knowledge of experimental design, scientific methodology, and analysis. Strong partner management and confirmed ability to influence large organisations without direct authority. Demonstrated ability ...

cases Partner with product, analytics, and business teams to translate requirements into scalable data solutions Architect solutions using big data technologies such as Hadoop, Spark, and distributed data processing framework Build and optimize ETL/ELT pipelines and ensure high data quality, reliability, and performance Implement automation, monitoring … experience with a PhD • Demonstrated leadership in delivering high-quality, large-scale, enterprise-class applications. • Solid experience in big data engineering, with knowledge of Hadoop, Apache Spark, Python, and SQL. • Expertise in Java or Scala and container-based technologies (Docker, Kubernetes). • Proficiency in creating and managing large-scale ...

data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ELT and reporting/analytic tools and environments (e.g., Apache Beam, Hadoop, Spark, Pig, Hive, MapReduce). Responsibilities: Be a trusted technical advisor to customers and solve complex machine learning challenges. Coach customers on the practical challenges ...

based ensembles (and experience applying them with packages like Pandas, sci-kit-learn, SciPy) is highly beneficial Experience processing big data, ideally in a Hadoop/Spark environment, would be beneficial Proven experience implementing CI/CD and DevOps practices, with the ability to improve deployment reliability, testing standards ...

Cortex AI, data sharing, Snowpark, and Native Apps. Familiarity with financial services cloud migration patterns and the technical landscape of legacy platforms (Teradata, Oracle, Hadoop). Experience with real‐time data architectures, streaming ingestion, and event‐driven data platforms in regulated environments. Background in data mesh, data product design ...