{"id":19710840,"url":"https://github.com/rootstrap/ai-job-title-area-classification","last_synced_at":"2026-03-02T22:32:09.683Z","repository":{"id":40973290,"uuid":"223251400","full_name":"rootstrap/ai-job-title-area-classification","owner":"rootstrap","description":"Classification of job titles into categories, using different ML techniques","archived":false,"fork":false,"pushed_at":"2022-09-30T18:50:05.000Z","size":48094,"stargazers_count":45,"open_issues_count":8,"forks_count":6,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-06-12T01:05:04.297Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rootstrap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-21T19:41:29.000Z","updated_at":"2025-05-24T05:32:26.000Z","dependencies_parsed_at":"2023-01-18T10:05:15.620Z","dependency_job_id":null,"html_url":"https://github.com/rootstrap/ai-job-title-area-classification","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rootstrap/ai-job-title-area-classification","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootstrap%2Fai-job-title-area-classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootstrap%2Fai-job-title-area-classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootstrap%2Fai-job-title-area-classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootstrap%2Fai-job-title-area-classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rootstrap","download_url":"https://codeload.github.com/rootstrap/ai-job-title-area-classification/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootstrap%2Fai-job-title-area-classification/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30022929,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-02T22:30:10.381Z","status":"ssl_error","status_checked_at":"2026-03-02T22:23:34.650Z","response_time":60,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2024-11-11T22:08:36.448Z","updated_at":"2026-03-02T22:32:09.654Z","avatar_url":"https://github.com/rootstrap.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Classification of job positions by area\nThis is a project to classify job positions using machine learning, more specifically, supervised learning. The main goal is to get a classifier\nthat receives a job position in the form of a sentence, written in natural language, for example `CEO and Founder` and returns the job area for that position. The different areas (labels for the classification) are:  \n* Business\n* Technical\n* Sales-Marketing\n* Other\n\nIn the example, `CEO and Founder` would return `Business`.\n\nThere is an analogous project but it classifies according to the level of the position: [Classification of job positions by level](https://github.com/rootstrap/ai-job-title-level-classification).\n\n\n## Implementation\nThis project is programed using the [Python language](https://www.python.org). The trained classifiers are implemented in the [Scikit Learn library](https://scikit-learn.org), a set of tools for machine learning in Python.\nTwo classification classes are studied:  \n* [SGDClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html?highlight=sgdcl#sklearn.linear_model.SGDClassifier)\n* [MLPClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html?highlight=mlpclassifier#sklearn.neural_network.MLPClassifier)\n\n## Needed libraries\nIf you use pip and virtual environments, you can install easily the needed libraries, such as [pandas](https://pandas.pydata.org/) and scikit learn:  \n`pip install -r requirements.txt`.\n\n## Process data\nSince both classifiers belong to supervised learning, they are trained using manually classified data, that you can see on `data_process/data_sets/classified_titles.tsv`. That is a tab-separated-values file, that has two columns in the form:  \n`\u003cjob position\u003e | \u003cclassification for the job position\u003e`.  \nThe script `data_process/tsv_to_dataframe.py` takes the `tsv` file and:\n1. Generates a dataframe that represents the job positions and corresponding classifications.\n2. Split the dataframe into train(X) and test(y) set.\n3. Normalizes the dataframe according to a defined criteria.\n4. Stores X_train, X_test, y_train, y_test sets.\n\n## Training and tuning\nA classifier has:  \n* parameters: values that corresponds to the mathematical model, that are adjusted after the training.\n* hyper-parameters: values related to the way of training, that are adjusted using a selected part of the training set.  \n\nIt's possible to use a `fit` function to train and adjust the params. Besides, Scikit learn provides a tool named\n [`Pipeline`](https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html), and [`GridSearchCV`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html) in order to make exhaustive search to achieve the hyperparams that optimize the results.\n\n## Script execution\nThe steps are the same as the classification by level:\n1. Run `data_process/tsv_file_to_dataframe.py` to extract the data from the tsv file and split the dataset.\n2. Run `\u003cclf name\u003e_fit_tune_classifier.py` to fit and tune the classifier. `fit` is to learn and fit the model to the train set. `tune` is to search for the optimal combination of the hyperparams, the ones that achieves better results (tuning may take a while).\n3. Run `\u003cclf name\u003e_test_classifier.py` to test the trained classifiers and show the results. Besides, a classified example set is stored in `test_data/\u003cclf name\u003e_results.tsv`.\n\nNotes:\n* `\u003cclf name\u003e` can be `mlp` or `sgd`, depending on the classifier.\n* `mlp_fit_tune_classifier.py` can take days to complete the tuning.\n\n## Results\nThese are the results of each classification class:  \n\n### MLP\n```\n                  precision    recall  f1-score   support\n\n       Business       0.91      0.92      0.91        63\n          Other       0.79      0.86      0.83        44\nSales-Marketing       0.90      0.90      0.90        10\n      Technical       0.86      0.73      0.79        33\n\n       accuracy                           0.86       150\n      macro avg       0.86      0.85      0.86       150\n   weighted avg       0.86      0.86      0.86       150\n```\n\n### SGD\n```\n                  precision    recall  f1-score   support\n\n       Business       0.97      0.97      0.97        63\n          Other       0.95      0.95      0.95        44\nSales-Marketing       0.89      0.80      0.84        10\n      Technical       0.97      1.00      0.99        33\n\n       accuracy                           0.96       150\n      macro avg       0.95      0.93      0.94       150\n   weighted avg       0.96      0.96      0.96       150\n```\n\nSGD has better average than MLP. SGD uses SVM model, a good model for text classification.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frootstrap%2Fai-job-title-area-classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frootstrap%2Fai-job-title-area-classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frootstrap%2Fai-job-title-area-classification/lists"}