Azure SQL Data Warehouse, Azure Data Lake,AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e.Hadoop Hive, Pig, Impala or similar Have experience with Open Source non-relational or NoSQL data repositories such as:MongoDB, Cassandra, Neo4J or similar Be confident more »
Azure SQL Data Warehouse, Azure Data Lake,AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e.Hadoop Hive, Pig, Impala or similar Have experience with Open Source non-relational or NoSQL data repositories such as:MongoDB, Cassandra, Neo4J or similar Be confident more »
Azure SQL Data Warehouse, Azure Data Lake,AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e.Hadoop Hive, Pig, Impala or similar Have experience with Open Source non-relational or NoSQL data repositories such as:MongoDB, Cassandra, Neo4J or similar Be confident more »
SQL Data Warehouse, Azure Data Lake, AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e. Hadoop Hive, Pig, Impala or similar Have experience with Open Source non-relational or NoSQL data repositories such as: MongoDB, Cassandra, Neo4J or similar Be confident more »
SQL Data Warehouse, Azure Data Lake, AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e. Hadoop Hive, Pig, Impala or similar Have experience with Open Source non-relational or NoSQL data repositories such as: MongoDB, Cassandra, Neo4J or similar Be confident more »
source-to-target mappings) to testing and service optimisation.) Good familiarity with our developing key services/applications - AmazonRDS, Amazon DynamoDB, AWS Glue, MapReduce, Hive, Spark, YARN, Airflow. Ability to work with a range of structured, semi-structured and unstructured file formats including Parquet, json, csv, pdf, jpg. Accomplished more »
and analytical role Experience of Data Lake/Hadoop platform implementation Hands-on experience in implementation and performance tuning Hadoop/Spark implementations Experience Apache Hadoop and the Hadoop ecosystem Experience with one or more relevant tools (Sqoop, Flume, Kafka, Oozie, Hue, Zookeeper, Hcatalog, Solr, Avro) Experience with one … or more SQL-on-Hadoop technology (Hive, Impala, Spark SQL, Presto) Experience developing software code in one or more programming languages (Java, Python, etc.) Preferred Qualifications Masters or PhD in Computer Science, Physics, Engineering or Math Hands on experience leading large-scale global data warehousing and analytics projects Ability more »
london, south east england, United Kingdom Hybrid / WFH Options
Careers at MI5, SIS and GCHQ
delivering moderate-to-complex data flows as part of a development team in collaboration with others. You’ll be confident using technologies such as: Apache Kafka, Apache NiFi, SAS DI Studio, or other data integration platforms. You can implement, deliver, and translate several data models, including unstructured data … and recognised standards to build solutions using various traditional or big data languages such as: SQL, PL/SQL, SAS Macro Language, Python, Scala, Apache Spark, Java, JavaScript etc, using various tools including SAS, Hue (Hive/Impala), Kibana (Elastic Search). Knowledge of data management on Cloud more »
Job Purpose and Primary Objectives We are seeking a highly experienced Kafka Developer with expertise in Kafka Streams and Big Data technologies (Hadoop, Hue, Hive, Impala, Spark). The ideal candidate will have strong knowledge of key-value databases and experience in developing microservices using Spring. The role involves … data services for various 24/7 applications with high-performance requirements. Key Skills/Knowledge Solid experience in Big Data technologies, specifically Hadoop, Hive, Java and Spark/Scala . Advanced SQL knowledge for testing changes and replicating code functionality. Proficient with code repositories like GIT and Maven. more »
business intelligence platform (e.g. Microsoft, Crystal, Qlik, SAP, Tableau). Good understanding of open source, big data, and cloud data platforms (e.g. Hadoop, Spark, Hive, Pentaho, AWS, Azure); given a business problem, you can analyse and evaluate options and recommend solutions. Proven experience in designing, building and maintaining ETL more »
Azure SQL Data Warehouse, Azure Data Lake,AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e.Hadoop Hive, Pig, Impala or similar Have experience with Open Source non-relational or NoSQL data repositories such as:MongoDB, Cassandra, Neo4J or similar Be confident more »
analytic tools like R & Python; & visualization tools like Tableau & Power BI Exposure to cloud platforms and big data systems such as Hadoop HDFS, and Hive is a plus Ability to work with IT and Data Engineering teams to help embed analytic outputs in business processes Graduate in Business Analytics more »
quickly Ability to work independently and be self-directed Bachelor's degree in Computer Science or related Experience with big data analytics: Splunk, ELK, Hive, Redshift, etc. (nice to have) In-depth knowledge of streaming back-ends and formats (nice to have) Experience working with Smart/Digital TV more »
Months Location: Milton Keynes Key responsibilities: experience as an Kafka Developer (Ideally Kafka Streams). on experience in Big data technologies (Hadoop, Hue, Hive, Impala,Spark,.) and stronger within Kafka. and experience using Key value data bases . Experience developing microservices using Spring. and develop cloud-based data … data services for different applications ,usually 24/7 applications with a big performance requirement . Key skills/knowledge/experience: Data Hadoop - Hive and Spark/Scala solid experience advance knowledge - Been able to test changes and issues properly, replicating the code functionality into SQL with Code more »
Elastic search and understanding of Hadoop ecosystem Experience working with large data sets, experience working with distributed computing tools like Map/Reduce, Hadoop, Hive, Pig etc. Advanced use of Excel spread sheets for analytical purposes MSc or PhD in Data Science or an analytical subject (Physics, Mathematics, Computing more »
Elastic search and understanding of Hadoop ecosystem Experience working with large data sets, experience working with distributed computing tools like Map/Reduce, Hadoop, Hive, Pig etc. Advanced use of Excel spread sheets for analytical purposes An MSc or PhD in Data Science or an analytical subject (Physics, Mathematics more »
