to streamline data workflows and reduce manual interventions. Must have: AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python. Good to have: Cloudera - Spark, Hive, Impala, HDFS, Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin. Qualifications: Bachelor's or Master's degree in Computer Science, Data Engineering, or a More ❯
Experience in working with data visualization tools Experience in GCP tools – Cloud Function, Dataflow, Dataproc and Bigquery Experience in data processing framework – Beam, Spark, Hive, Flink GCP data engineering certification is a merit Have hands on experience in Analytical tools such as powerBI or similar visualization tools Exhibit understanding More ❯
data structures. Encouraging self-learning among the team. Essential Skills & Qualifications: A confident engineer with an authoritative knowledge of Java and Hadoop including HDFS, Hive, and Spark. Comfortable working with large data volumes and able to demonstrate a firm understanding of logical data structures and analysis techniques. Strong skills More ❯
eg. Python, R, Scala, etc.; (Python preferred). Proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques and More ❯
Lincoln, Lincolnshire, United Kingdom Hybrid / WFH Options
Adecco
CRITERIA Degree in Computer Science, Information Systems, or a related field. or a combination of education and relevant experience Query languages e.g. SQL, Java, Hive, R Data Management technologies e.g. ETL tools, data integration platforms Proven experience as a Data Architect, Data Engineer (or a related role, with a More ❯
in coding languages e.g. Python, C++, etc.; (Python preferred). Proficiency in database technologies e.g. SQL, No-SQL and Big Data technologies e.g. pySpark, Hive, etc. Experience working with structured and unstructured data e.g. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques and how More ❯
working with structured and unstructured data (e.g., text, PDFs, images, call recordings, video) Proficiency in database and big data technologies including SQL, NoSQL, PySpark, Hive, etc. Cloud & AI Ecosystems Experience working with cloud platforms such as AWS, GCP, or Azure Understanding of API integration and deploying solutions in cloud More ❯
working with structured and unstructured data (e.g., text, PDFs, images, call recordings, video) Proficiency in database and big data technologies including SQL, NoSQL, PySpark, Hive, etc. Cloud & AI Ecosystems Experience working with cloud platforms such as AWS, GCP, or Azure Understanding of API integration and deploying solutions in cloud More ❯
Newcastle Upon Tyne, Tyne And Wear, United Kingdom
Accenture
working with structured and unstructured data (e.g., text, PDFs, images, call recordings, video) Proficiency in database and big data technologies including SQL, NoSQL, PySpark, Hive, etc. Cloud & AI Ecosystems Experience working with cloud platforms such as AWS, GCP, or Azure Understanding of API integration and deploying solutions in cloud More ❯
engineers on the team to elevate technology and consistently apply best practices. Qualifications for Software Engineer Hands-on experience working with technologies like Hadoop, Hive, Pig, Oozie, Map Reduce, Spark, Sqoop, Kafka, Flume, etc. Strong DevOps focus and experience building and deploying infrastructure with cloud deployment technologies like Ansible More ❯
Centre of Excellence. Skills, knowledge and expertise: Deep expertise in the Databricks platform, including Jobs and Workflows, Cluster Management, Catalog Design and Maintenance, Apps, Hive Metastore Management, Network Management, Delta Sharing, Dashboards, and Alerts. Proven experience working with big data technologies, i.e., Databricks and Apache Spark. Proven experience More ❯
BDT supports Amazon subsidiaries such as IMDB and Audible, providing interfaces for internal customers to access and query data using AWS services like Redshift, Hive, Spark, and Oracle. We build scalable solutions that grow with Amazon's business. As BDT grows and the data processing landscape evolves, our data More ❯
Bristol, Avon, South West, United Kingdom Hybrid / WFH Options
Hargreaves Lansdown Asset Management Limited
techniques in production-grade code, with a focus on scalability and reliability. Experience with large-scale data analysis, manipulation, and distributed computing platforms (e.g., Hive, Hadoop). Familiarity with advanced machine learning methods, including neural networks, reinforcement learning, and other cutting-edge Gen AI approaches. Skilled in API development More ❯
Employment Type: Permanent, Part Time, Work From Home
Birmingham, Staffordshire, United Kingdom Hybrid / WFH Options
Yelp USA
to the experimentation and development of new ad products at Yelp. Design, build, and maintain efficient data pipelines using large-scale processing tools like Apache Spark to transform ad-related data. Manage high-volume, real-time data streams using Apache Kafka and process them with frameworks like Apache Flink. Estimate timelines for projects, feature enhancements, and bug fixes. Work with large-scale data storage solutions, including Apache Cassandra and various data lake systems. Collaborate with cross-functional teams, including engineers, product managers and data scientists, to understand business requirements and translate them into effective system designs. … a proactive approach to identifying opportunities and recommending scalable, creative solutions. Exposure to some of the following technologies: Python, AWS Redshift, AWS Athena/Apache Presto, Big Data technologies (e.g S3, Hadoop, Hive, Spark, Flink, Kafka etc), NoSQL systems like Cassandra, DBT is nice to have. What you More ❯
Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13 billion. Job Description: ============= Spark - Must have Scala - Must Have Hive & SQL - Must Have Hadoop - Must Have Communication - Must Have Banking/Capital Markets Domain - Good to have Note: Candidate should know Scala/Python … Core) coding language. Pyspark profile will not help here. Scala/Spark • Good Big Data resource with the below Skillset: § Spark § Scala § Hive/HDFS/HQL • Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.) • Experience in Big data technologies, real time data processing platform (Spark Streaming More ❯
london, south east england, united kingdom Hybrid / WFH Options
Kantar Media
a broad IT skill set, including hands-on experience with Linux, AWS, Azure, Oracle 19 (admin), Tomcat, UNIX tools, Bash/sh, SQL, Python, Hive, Hadoop/HDFS, and Spark. Work within a modern cloud DevOps environment using Azure, Git, Airflow, Kubernetes, Helm, and Terraform. Demonstrate solid knowledge of … and network technologies. Experienced in writing and running SQL and Bash scripts to automate tasks and manage data. Skilled in installing, configuring, and managing Hive on Spark with HDFS. Strong analytical skills with the ability to troubleshoot complex issues and analyze large volumes of text or binary data in … Linux or Hive environments. Required You’re enthusiastic and eager to learn, especially in fast-paced, dynamic environments. You enjoy solving problems and take a logical, methodical approach to troubleshooting. Under pressure, you remain calm and focused, effectively prioritizing tasks to meet multiple deadlines. You’re flexible and adaptable More ❯
You're ready to gain the skills and experience needed to grow within your role and advance your career - and we have the perfect software engineering opportunity for you. As a Software Engineer III at JPMorgan Chase within Corporate Technology More ❯
and Solution Architect teams to design the overall solution architecture for end-to-end data flows. Utilize big data technologies such as Cloudera, Hue, Hive, HDFS, and Spark for data processing and storage. Ensure smooth data management for marketing consent and master data management (MDM) systems. Key Skills and … delivery for streamlined development workflows. Azure Data Factory/DataBricks : Experience with these services is a plus for handling complex data processes. Cloudera (Hue, Hive, HDFS, Spark) : Experience with these big data tools is highly desirable for data processing. Azure DevOps, Vault : Core skills for working in Azure cloud More ❯