Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/thiyangt/dsjobtracker
What skills and qualifications are required for a data scientist?
https://github.com/thiyangt/dsjobtracker
dataset qualifications skills statistics tidy
Last synced: 2 months ago
JSON representation
What skills and qualifications are required for a data scientist?
- Host: GitHub
- URL: https://github.com/thiyangt/dsjobtracker
- Owner: thiyangt
- Created: 2020-08-28T06:18:56.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-02-16T11:28:54.000Z (11 months ago)
- Last Synced: 2024-10-28T17:27:41.993Z (2 months ago)
- Topics: dataset, qualifications, skills, statistics, tidy
- Language: R
- Homepage: https://thiyangt.github.io/DSjobtracker/
- Size: 7.99 MB
- Stars: 3
- Watchers: 4
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
[](https://CRAN.R-project.org/package=DSjobtracker)
# DSjobtracker
## What skills and qualifications are required for data science related jobs?
The DSjobtraker package compiles data from 1172 job advertisements in
data science and statistics, making it an invaluable tool for learning
key skills required in these areas. Here is a word cloud of 97 requested
skills mentioned in the job advertisements. It provides insights into
the most sought-after skills in the field.
# Installation
You can install the **development** version from
[GitHub](https://github.com/thiyangt/DSjobtracker):
``` r
#install.packages("devtools")
devtools::install_github("thiyangt/DSjobtracker")
library(DSjobtracker)
```
# Glimpse of tidy data
``` r
tibble::glimpse(DStidy)
```
Rows: 1,172
Columns: 109
$ ID 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, …
$ Consultant "Thiyanga", "Jayani", "Jayani", "Ja…
$ DateRetrieved 2020-08-05, 2020-08-07, 2020-08-07…
$ DatePublished NA, 2020-07-31, 2020-08-06, 2020-0…
$ Job_title NA, "Junior Data Scientist", "Engin…
$ Company NA, "Dialog Axiata PLC", "London St…
$ R 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0,…
$ SAS 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0,…
$ SPSS 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0,…
$ Python 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0,…
$ MAtlab 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Scala 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `C#` 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `MS Word` 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1,…
$ `Ms Excel` 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1,…
$ `OLE/DB` 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `Ms Access` 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1,…
$ `Ms PowerPoint` 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0,…
$ Spreadsheets 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Data_visualization 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0,…
$ Presentation_Skills 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Communication 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ BigData 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0,…
$ Data_warehouse 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ cloud_storage 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Google_Cloud 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ AWS 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Machine_Learning 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0,…
$ `Deep Learning` 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Computer_vision 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Java 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0,…
$ `C++` 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ C 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ `Linux/Unix` 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ SQL 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0,…
$ NoSQL 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ RDBMS 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Oracle 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,…
$ MySQL 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0,…
$ PHP 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,…
$ SPL 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ web_design_and_development_tools 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,…
$ AI 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `Natural_Language_Processing(NLP)` 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0,…
$ `Microsoft Power BI` 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Google_Analytics 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ graphics_and_design_skills 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,…
$ Data_marketing 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ SEO 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Content_Management 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Tableau 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,…
$ D3 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,…
$ Alteryx 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ KNIME 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Spotfire 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Spark 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0,…
$ S3 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ Redshift 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ DigitalOcean 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ Javascript 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ Kafka 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Storm 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Bash 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Hadoop 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
$ Data_Pipelines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ MPP_Platforms 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Qlik 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Pig 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Hive 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
$ Tensorflow 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `Map/Reduce` 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Impala 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Solr 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Teradata 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ MongoDB 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Elasticsearch 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ YOLO 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `agile execution` 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,…
$ Data_management 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ pyspark 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Data_mining 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ Data_science 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0,…
$ Web_Analytic_tools 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ IOT 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Numerical_Analysis 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Economic 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Finance_Knowledge 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Investment_Knowledge 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Problem_Solving 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Team_Handling 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Debtor_reconcilation 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Payroll_management 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Bayesian 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Optimization 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ Knowledge_in NA, NA, "Elasticsearch, Logstash, K…
$ City NA, "Colombo", "Colombo", "Colombo"…
$ Educational_qualifications NA, "Degree in Engineering / IT or …
$ Salary NA, NA, NA, NA, NA, NA, NA, NA, NA,…
$ URL NA, "https://www.google.com/search?…
$ Search_Term NA, "Data Analysis Jobs in Sri Lank…
$ Job_Category "Unimportant", "Data Science", "Dat…
$ Experience_Category "More than 2 and less than 5 years"…
$ Location NA, "Sri Lanka", "Sri Lanka", "Sri …
$ `Payment Frequency` "unspecified", "unspecified", "unsp…
$ BSc_needed 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1,…
$ MSc_needed 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
$ PhD_needed 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
$ `English Needed` 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,…
$ year 2020, 2020, 2020, 2020, 2020, 2020,…
# 2021 survey data
``` r
df2021 <- get_data(2021)
df2021
```
# A tibble: 382 × 115
ID Consultant URL Search_Term DateRetrieved DatePublished Job_Field
1 1 Abhishanya https://w… statistics… 11/10/2021 11/10/2021 Informat…
2 2 Abhishanya https://w… statistics… 11/11/2021 11/11/2021 Education
3 3 Abhishanya https://w… Data scien… 11/11/2021 11/9/2021 Other
4 4 Abhishanya https://w… Data scien… 11/11/2021 10/13/2021 Other
5 5 Abhishanya https://w… Statistics… 11/11/2021 10/14/2021 Informat…
6 6 Abhishanya https://w… Data Scien… 11/11/2021 10/14/2020 Health
7 7 Abhishanya https://c… Data engin… 11/11/2021 11/8/2021 Other
8 8 Abhishanya https://l… Data engin… 11/11/2021 11/9/2021 Other
9 9 Abhishanya https://w… Data engin… 11/11/2021 10/20/2021 Informat…
10 10 Abhishanya https://w… Data engin… 11/11/2021 10/21/2021 Finance
# ℹ 372 more rows
# ℹ 108 more variables: Job_title , Company , Knowledge_in ,
# `Minimum Experience in Years` , City , Location ,
# Educational_qualifications , `Payment Frequency` ,
# Currency , Salary , `English Needed` ,
# `English proficiency description` , Additional_languages ,
# AI , `Natural_Language_Processing(NLP)` , Data_Pipelines , …
# Preview of the tidy version of the dataset
``` r
head(DStidy)
```
# A tibble: 6 × 109
ID Consultant DateRetrieved DatePublished Job_title Company
1 1 Thiyanga 2020-08-05 00:00:00 NA
2 2 Jayani 2020-08-07 00:00:00 2020-07-31 00:00:00 Junior Data … Dialog…
3 3 Jayani 2020-08-07 00:00:00 2020-08-06 00:00:00 Engineer, An… London…
4 4 Jayani 2020-08-07 00:00:00 2020-07-24 00:00:00 CI-Statistic… E.D. B…
5 5 Jayani 2020-08-07 00:00:00 2020-07-24 00:00:00 DA-Data Anal… E.D. B…
6 6 Jayani 2020-08-07 00:00:00 2020-08-13 00:00:00 Data Scienti… Emirat…
# ℹ 103 more variables: R , SAS , SPSS , Python ,
# MAtlab , Scala , `C#` , `MS Word` , `Ms Excel` ,
# `OLE/DB` , `Ms Access` , `Ms PowerPoint` ,
# Spreadsheets , Data_visualization , Presentation_Skills ,
# Communication , BigData , Data_warehouse ,
# cloud_storage , Google_Cloud , AWS , Machine_Learning ,
# `Deep Learning` , Computer_vision , Java , `C++` , …
# Preview of the untidy version of the dataset
``` r
head(DSraw)
```
# A tibble: 6 × 152
ID Consultant DateRetrieved DatePublished Job_title Company R SAS
1 1 Thiyanga 05/08/2020 1 1
2 2 Jayani 07/08/2020 31/07/2020 Junior Data … Dialog… 1 0
3 3 Jayani 07/08/2020 06/08/20 Engineer, An… London… 0 0
4 4 Jayani 07/08/2020 24/07/2020 CI-Statistic… E.D. B… 1 1
5 5 Jayani 07/08/2020 24/07/2020 DA-Data Anal… E.D. B… 0 1
6 6 Jayani 07/08/2020 13/08/2020 Data Scienti… Emirat… 1 0
# ℹ 144 more variables: SPSS , Python , MAtlab , Scala ,
# `C#` , `MS Word` , `Ms Excel` , `OLE/DB` ,
# `Ms Access` , `Ms PowerPoint` , Spreadsheets ,
# Data_visualization , Presentation_Skills , Communication ,
# BigData , Data_warehouse , cloud_storage ,
# Google_Cloud , AWS , Machine_Learning ,
# `Deep Learning` , Computer_vision , Java , `C++` , …