Period
to 20 November 2018

The following table provides summary statistics for permanent job vacancies with a requirement for MLlib skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited MLlib over the 6 months to 20 November 2018 with a comparison to the same period in the previous 2 years.

Apache Spark Machine Learning Library (MLlib)
UK
6 months to
20 Nov 2018
Same period 2017 Same period 2016
Rank 1069 1161 1223
Rank change year-on-year +92 +62 +72
Permanent jobs citing MLlib 72 45 12
As % of all permanent IT jobs advertised in the UK 0.044% 0.025% 0.006%
As % of the Libraries, Frameworks & Software Standards category 0.11% 0.062% 0.016%
Number of salaries quoted 55 33 9
UK median annual salary £72,500 £60,000 £75,000
Median salary % change year-on-year +20.83% -20.00% -5.66%
10th Percentile £42,500 £25,750 £38,500
90th Percentile £99,000 £107,000 £80,000
UK excluding London median annual salary £35,000 £50,000 £50,000
% change year-on-year -30.00% - -37.11%

MLlib is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
UK
Permanent vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 64,955 72,267 73,070
As % of all permanent IT jobs advertised in the UK 39.82% 40.17% 39.06%
Number of salaries quoted 53,648 60,067 63,381
UK median annual salary £52,500 £50,000 £47,500
Median salary % change year-on-year +5.00% +5.26% +5.56%
10th Percentile £31,250 £29,750 £28,750
90th Percentile £83,750 £77,500 £76,250
UK excluding London median annual salary £45,000 £42,500 £42,500
% change year-on-year +5.88% - +6.25%

MLlib
Job Vacancy Trend

Job postings citing MLlib as a percentage of all IT jobs advertised.

Job vacancy trend for MLlib in the UK

MLlib
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing MLlib.

Salary trend for MLlib in the UK

MLlib
Salary Histogram

The salary distribution of IT jobs citing MLlib over the 6 months to 20 November 2018.

Salary histogram for MLlib in the UK

MLlib
Top 10 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing MLlib within the UK over the 6 months to 20 November 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +83 72 £72,500 +20.83% 2
London +94 50 £95,000 +5.56% 2
UK excluding London +57 22 £35,000 -30.00%
South East +27 14 - -
South West -6 5 £35,000 -30.00%
East of England +26 1 £45,000 -10.00%
Yorkshire - 1 £47,500 -
East Midlands - 1 £100,000 -
Midlands - 1 £100,000 -
North of England - 1 £47,500 -

For the 6 months to 20 November 2018, IT jobs citing MLlib also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for MLlib.

1 67 (93.06%) Machine Learning
2 64 (88.89%) Python
3 59 (81.94%) Apache Spark
4 56 (77.78%) Hadoop
5 51 (70.83%) Data Science
6 49 (68.06%) Deep Learning
6 49 (68.06%) Apache Hive
7 46 (63.89%) TensorFlow
8 44 (61.11%) Scala
9 35 (48.61%) Big Data
9 35 (48.61%) scikit-learn
10 32 (44.44%) Java
11 29 (40.28%) Pandas
12 27 (37.50%) Analytics
13 26 (36.11%) SQL
13 26 (36.11%) Data Analysis
13 26 (36.11%) Artificial Intelligence
14 25 (34.72%) Visualisation
14 25 (34.72%) Data Visualisation
14 25 (34.72%) Natural Language Processing
14 25 (34.72%) Tableau
15 24 (33.33%) R
15 24 (33.33%) Algorithms
15 24 (33.33%) Theano
15 24 (33.33%) PySpark
16 23 (31.94%) HDFS
17 21 (29.17%) HBase
18 20 (27.78%) Degree
19 19 (26.39%) Hortonworks
19 19 (26.39%) Kafka

MLlib
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 59 (81.94%) Apache Spark
2 8 (11.11%) Apache Pig
3 6 (8.33%) SAS
4 4 (5.56%) nginx
5 2 (2.78%) Apache
5 2 (2.78%) Apache Solr
5 2 (2.78%) Elasticsearch
6 1 (1.39%) Confluence
Cloud Services
1 12 (16.67%) Amazon AWS
2 9 (12.50%) Google Cloud Platform
3 7 (9.72%) Amazon SageMaker
4 4 (5.56%) Microsoft Azure
5 2 (2.78%) Amazon EMR
5 2 (2.78%) SaaS
6 1 (1.39%) Cloud Computing
6 1 (1.39%) CloudBees
6 1 (1.39%) PaaS
Database & Business Intelligence
1 56 (77.78%) Hadoop
2 49 (68.06%) Apache Hive
3 35 (48.61%) Big Data
4 21 (29.17%) HBase
5 19 (26.39%) QlikView
6 18 (25.00%) Text Mining
7 16 (22.22%) Impala
8 15 (20.83%) Data Warehouse
9 14 (19.44%) Sqoop
10 10 (13.89%) PostgreSQL
11 9 (12.50%) NoSQL
12 4 (5.56%) Apache Zeppelin
12 4 (5.56%) Couchbase
13 3 (4.17%) Apache Cassandra
13 3 (4.17%) MongoDB
13 3 (4.17%) Relational Database
14 2 (2.78%) Data Lake
14 2 (2.78%) Data Mining
14 2 (2.78%) Parquet
14 2 (2.78%) RDBMS
Development Applications
1 16 (22.22%) Jenkins
2 15 (20.83%) Git (software)
2 15 (20.83%) TeamCity
3 5 (6.94%) JUnit
4 2 (2.78%) Eclipse
4 2 (2.78%) FindBugs
4 2 (2.78%) IntelliJ
4 2 (2.78%) Maven
4 2 (2.78%) RStudio
5 1 (1.39%) JIRA
5 1 (1.39%) Selenium
5 1 (1.39%) SonarQube
General
1 8 (11.11%) Finance
1 8 (11.11%) Retail
2 6 (8.33%) Law
3 5 (6.94%) Banking
3 5 (6.94%) Marketing
3 5 (6.94%) Military
4 2 (2.78%) Games
4 2 (2.78%) German Language
5 1 (1.39%) Aerospace
Job Titles
1 40 (55.56%) Data Scientist
2 30 (41.67%) Senior Data Scientist
3 20 (27.78%) Data Engineer
4 18 (25.00%) Hadoop Engineer
4 18 (25.00%) Machine Learning Scientist
5 17 (23.61%) Big Data Engineer
6 10 (13.89%) Senior Data Engineer
6 10 (13.89%) Senior Hadoop Engineer
7 4 (5.56%) Developer
7 4 (5.56%) Java Developer
7 4 (5.56%) Java Engineer
7 4 (5.56%) Senior Developer
7 4 (5.56%) Senior Java Developer
7 4 (5.56%) Senior Java Engineer
8 2 (2.78%) Architect
8 2 (2.78%) Data Manager
8 2 (2.78%) Junior
8 2 (2.78%) Python Engineer
8 2 (2.78%) Senior Architect
8 2 (2.78%) Senior Data Architect
Libraries, Frameworks & Software Standards
1 46 (63.89%) TensorFlow
2 35 (48.61%) scikit-learn
3 29 (40.28%) Pandas
4 25 (34.72%) Natural Language Processing
5 24 (33.33%) PySpark
5 24 (33.33%) Theano
6 19 (26.39%) Kafka
7 18 (25.00%) NumPy
7 18 (25.00%) SciPy
8 15 (20.83%) Keras
9 6 (8.33%) XGBoost
10 5 (6.94%) MXNet
10 5 (6.94%) RHadoop
11 4 (5.56%) Spring
12 2 (2.78%) Apache Avro
12 2 (2.78%) SparkR
13 1 (1.39%) Apache Flex
13 1 (1.39%) D3.js
13 1 (1.39%) Regular Expression
13 1 (1.39%) Torch
Miscellaneous
1 24 (33.33%) Algorithms
2 17 (23.61%) Computer Science
3 13 (18.06%) Analytical Skills
4 12 (16.67%) Self-Motivation
5 10 (13.89%) User Experience
6 5 (6.94%) Digital Media
7 4 (5.56%) Data Structures
8 3 (4.17%) Greenfield Project
9 2 (2.78%) Mainframe
10 1 (1.39%) Cyberthreat
10 1 (1.39%) Fintech
10 1 (1.39%) GPGPU
10 1 (1.39%) Management Information System
10 1 (1.39%) Social Media
Operating Systems
1 16 (22.22%) Linux
2 2 (2.78%) Ubuntu
2 2 (2.78%) Unix
3 1 (1.39%) Red Hat Enterprise Linux
Processes & Methodologies
1 67 (93.06%) Machine Learning
2 51 (70.83%) Data Science
3 49 (68.06%) Deep Learning
4 27 (37.50%) Analytics
5 26 (36.11%) Artificial Intelligence
5 26 (36.11%) Data Analysis
6 25 (34.72%) Data Visualisation
6 25 (34.72%) Visualisation
7 17 (23.61%) Cybersecurity
8 16 (22.22%) Continuous Integration
8 16 (22.22%) Mathematics
9 15 (20.83%) Agile Software Development
9 15 (20.83%) Data Engineering
10 14 (19.44%) MapReduce
11 13 (18.06%) Physics
12 12 (16.67%) Data Modelling
13 10 (13.89%) Operations Research
13 10 (13.89%) Software Delivery
13 10 (13.89%) Statistics
14 8 (11.11%) DevOps
Programming Languages
1 64 (88.89%) Python
2 44 (61.11%) Scala
3 32 (44.44%) Java
4 26 (36.11%) SQL
5 24 (33.33%) R
6 16 (22.22%) Shell Script
7 13 (18.06%) Go
8 11 (15.28%) Ruby
9 8 (11.11%) C++
10 6 (8.33%) C
11 5 (6.94%) C#
11 5 (6.94%) MATLAB
12 2 (2.78%) VBA
13 1 (1.39%) Bash Shell
13 1 (1.39%) HiveQL
Qualifications
1 20 (27.78%) Degree
2 13 (18.06%) PhD
3 3 (4.17%) Master's Degree
3 3 (4.17%) Numerate Degree
4 2 (2.78%) Computer Science Degree
5 1 (1.39%) DV Cleared
5 1 (1.39%) Security Cleared
Quality Assurance & Compliance
1 2 (2.78%) QA
System Software
1 23 (31.94%) HDFS
2 13 (18.06%) Apache Flume
3 7 (9.72%) Docker
Systems Management
1 13 (18.06%) Hadoop YARN
2 1 (1.39%) Kibana
Vendors
1 25 (34.72%) Tableau
2 19 (26.39%) Hortonworks
3 18 (25.00%) Cloudera
4 8 (11.11%) Google
4 8 (11.11%) MapR
5 5 (6.94%) Microsoft
6 2 (2.78%) Cognizant
6 2 (2.78%) Salesforce.com
6 2 (2.78%) SAP
7 1 (1.39%) Alteryx
7 1 (1.39%) Atlassian
7 1 (1.39%) Databricks
7 1 (1.39%) Datameer
7 1 (1.39%) Informatica
7 1 (1.39%) Paxata
7 1 (1.39%) Splunk
7 1 (1.39%) Talend