performance, reliability, and security. - Implement event-driven architectures using Kafka for real-time data processing and communication between microservices. - Utilize Big Data technologies (e.g., ApacheSpark, Hadoop) to process and analyze large volumes of data, extracting valuable insights to drive decision-making. - Design and optimize data pipelines for … principles and best practices. - Experience with Kafka for building event-driven architectures and real-time data processing. - Familiarity with Big Data technologies such as ApacheSpark, Hadoop, or similar frameworks. - Proven track record of delivering scalable and reliable software solutions in a fast-paced environment. - Excellent communication skills more »
Data Engineer 6 Month Contract Inside IR35 £450/day Hiring Immediately Job Description (Apache Iceberg, Spark, Big Data) Job Details Overview: Overall IT experience of 5+ years of total experience with strong programming skills Excellent skill in Apache Iceberg, Spark, Big Data 3+ years of … Big Data project development experience Hands on experience in working areas like Apache Iceberg & Spark, Hadoop, Hive Must have knowledge in any Database Ex: Postgres, Oracle, MongoDB Excellent in SDLC Processes and DevOps knowledge (Jira, Jenkins pipeline) Working in Agile POD and with team collaboration Ability to participate more »
Google Cloud Composer, AWS Step Functions, Azure Data Factory, UC4, Control-M). * Experience with data quality and validation. * Experience querying massive datasets using Spark, Presto, Hive, Impala, etc. * Experience in optimization of computer-vision applications. * Experience in building highly-scalable performant data pipelines * Experience with Data Modeling. Morgan more »
working with AWS technologies such as Lambda, ECS Fargate, API Gateway, RDS, DynamoDB, EMR building customer-facing applications and APIs building data pipelines using Spark + Scala that process Tb of data per day working with customers to understand the business context of new features participating in design reviews more »
City of London, London, United Kingdom Hybrid / WFH Options
ECS Resource Group
Technical Architect - Java/Spark Job type - Contract (Inside IR35) 100% Remote working Join our client's team as a Technical Architect! Are you ready to take the lead in delivering cutting-edge technical data solutions? We're seeking a talented individual to join our client's team as … a pivotal role in shaping the technological landscape of our projects. Key Responsibilities: Architect end-to-end solutions, leveraging your expertise in Python, Java, Spark, Trino, Airflow, and Hadoop. Maintain a hands-on approach, ensuring that your architectural vision translates seamlessly into implementation. Collaborate with cross-functional teams to more »
scripting. * Minimum of 1 year experience in investment banking or the financial sector. * Performance Tuning of Oracle/MySQL/Hive SQL Queries/Spark SQL Statements. * Experience in working with large databases - multi terabytes (3+ Terabytes). * Minimum of 5 years' experience in Big Data Space (Hive, Impala … Spark Sql, HDFS etc). * Any cloud experience (AWS/Azure/Google/Oracle). * Solid experience with Oracle objects(Packages,Procedure,functions) * Very clear concepts on Oracle architecture. * Very strong debugging skills. * Proficient in query tuning. * Detail oriented * Strong written and verbal communication skills * Ability to work more »
