{"id":20556510,"url":"https://github.com/somjit101/nlp-stackeroverflow-tag-prediction","last_synced_at":"2026-06-05T17:31:17.132Z","repository":{"id":179926702,"uuid":"387259613","full_name":"somjit101/NLP-StackerOverflow-Tag-Prediction","owner":"somjit101","description":"A multi-class classification problem where the objective is to read a question posted on the popular reference website, StackOverflow and predict the primary topics it deals with, i.e. tags which the question will be associated with. ","archived":false,"fork":false,"pushed_at":"2021-07-19T17:42:19.000Z","size":981,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-16T18:26:28.310Z","etag":null,"topics":["bag-of-words","countvectorizer","logistic-regression","multi-class-classification","multiclass-logistic-regression","natural-language-processing","nlp","one-vs-rest","onevsrestclassifier","stackoverflow-tags","stemming","text-mining","tf-idf","tfidf-vectorizer","word-cloud"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/somjit101.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-18T20:03:42.000Z","updated_at":"2021-07-20T20:50:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"9070597d-8837-462e-b0c8-3ad4209aa59b","html_url":"https://github.com/somjit101/NLP-StackerOverflow-Tag-Prediction","commit_stats":null,"previous_names":["somjit101/nlp-stackeroverflow-tag-prediction"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somjit101%2FNLP-StackerOverflow-Tag-Prediction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somjit101%2FNLP-StackerOverflow-Tag-Prediction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somjit101%2FNLP-StackerOverflow-Tag-Prediction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somjit101%2FNLP-StackerOverflow-Tag-Prediction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/somjit101","download_url":"https://codeload.github.com/somjit101/NLP-StackerOverflow-Tag-Prediction/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242163854,"owners_count":20082224,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bag-of-words","countvectorizer","logistic-regression","multi-class-classification","multiclass-logistic-regression","natural-language-processing","nlp","one-vs-rest","onevsrestclassifier","stackoverflow-tags","stemming","text-mining","tf-idf","tfidf-vectorizer","word-cloud"],"created_at":"2024-11-16T03:28:40.905Z","updated_at":"2026-06-05T17:31:17.045Z","avatar_url":"https://github.com/somjit101.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# StackerOverflow Question Tag Prediction\n\nA multi-class classification problem where the objective is to read a question posted on the popular reference website, StackOverflow and predict the primary topics it deals with, i.e. tags which the question will be associated with. These tags are used to group and search for relevant questionn, provide useful suggestions, map new question to already existing and hence improve the user experience in overall. \n[Kaggle Problem](https://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/)\n\n## Business Problem\n\n### Description\n\n\u003cp\u003e\nStack Overflow is the largest, most trusted online community for developers to learn, share their programming knowledge, and build their careers.\u003cbr /\u003e\n\u003cbr /\u003e\nStack Overflow is something which every programmer use one way or another. Each month, over 50 million developers come to Stack Overflow to learn, share their knowledge, and build their careers. It features questions and answers on a wide range of topics in computer programming. The website serves as a platform for users to ask and answer questions, and, through membership and active participation, to vote questions and answers up or down and edit questions and answers in a fashion similar to a wiki or Digg. As of April 2014 Stack Overflow has over 4,000,000 registered users, and it exceeded 10,000,000 questions in late August 2015. Based on the type of tags assigned to questions, the top eight most discussed topics on the site are: Java, JavaScript, C#, PHP, Android, jQuery, Python and HTML.\u003cbr /\u003e\n\u003cbr /\u003e\n\u003c/p\u003e\n\n### Problem Statement\n\nSuggest the tags based on the content that was there in the question posted on Stackoverflow.\n[Kaggle Problem Page](https://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/)\n\n### References and Useful Links\n* [Kaggle Dataset](https://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/data)\n* [YouTube Video](https://youtu.be/nNDqbUhtIRg)\n* [Automatic Tag Recommendation Algorithms for Social Recommender Systems - Research Paper by Microsoft](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tagging-1.pdf)\n* [A Hybrid Auto-tagging System for StackOverflow Forum Questions - ACM Research Paper](https://dl.acm.org/citation.cfm?id=2660970\u0026dl=ACM\u0026coll=DL)\n\n### Real World Objectives and Constraints\n1. Predict as many tags as possible with high precision and recall.\n2. Incorrect tags could impact customer experience on StackOverflow.\n3. No strict latency constraints.\n\n\n## Machine Learning Problem\n\n### Data\n\n#### Data Overview\n\n[Dataset Link](https://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/data)\n\u003cbr\u003e\nAll of the data is in 2 files: Train and Test.\u003cbr /\u003e\n\u003cpre\u003e\n\u003cb\u003eTrain.csv\u003c/b\u003e contains 4 columns: Id,Title,Body,Tags.\u003cbr /\u003e\n\u003cb\u003eTest.csv\u003c/b\u003e contains the same columns but without the Tags, which you are to predict.\u003cbr /\u003e\n\u003cb\u003eSize of Train.csv\u003c/b\u003e - 6.75GB\u003cbr /\u003e\n\u003cb\u003eSize of Test.csv\u003c/b\u003e - 2GB\u003cbr /\u003e\n\u003cb\u003eNumber of rows in Train.csv\u003c/b\u003e = 6034195\u003cbr /\u003e\n\u003c/pre\u003e\nThe questions are randomized and contains a mix of verbose text sites as well as sites related to math and programming. The number of questions from each site may vary, and no filtering has been performed on the questions (such as closed questions).\u003cbr /\u003e\n\u003cbr /\u003e\n\n__Data Field Explanation__\n\nDataset contains 6,034,195 rows. The columns in the table are:\u003cbr /\u003e\n\u003cpre\u003e\n\u003cb\u003eId\u003c/b\u003e - Unique identifier for each question\u003cbr /\u003e\n\u003cb\u003eTitle\u003c/b\u003e - The question's title\u003cbr /\u003e\n\u003cb\u003eBody\u003c/b\u003e - The body of the question\u003cbr /\u003e\n\u003cb\u003eTags\u003c/b\u003e - The tags associated with the question in a space-seperated format (all lowercase, should not contain tabs '\\t' or ampersands '\u0026')\u003cbr /\u003e\n\u003c/pre\u003e\n\n\u003cbr /\u003e\n\n#### Example Data point\n\n\u003cpre\u003e\n\u003cb\u003eTitle\u003c/b\u003e:  Implementing Boundary Value Analysis of Software Testing in a C++ program?\n\u003cb\u003eBody \u003c/b\u003e: \u003cpre\u003e\u003ccode\u003e\n        #include\u0026lt;\n        iostream\u0026gt;\\n\n        #include\u0026lt;\n        stdlib.h\u0026gt;\\n\\n\n        using namespace std;\\n\\n\n        int main()\\n\n        {\\n\n                 int n,a[n],x,c,u[n],m[n],e[n][4];\\n         \n                 cout\u0026lt;\u0026lt;\"Enter the number of variables\";\\n         cin\u0026gt;\u0026gt;n;\\n\\n         \n                 cout\u0026lt;\u0026lt;\"Enter the Lower, and Upper Limits of the variables\";\\n         \n                 for(int y=1; y\u0026lt;n+1; y++)\\n         \n                 {\\n                 \n                    cin\u0026gt;\u0026gt;m[y];\\n                 \n                    cin\u0026gt;\u0026gt;u[y];\\n         \n                 }\\n         \n                 for(x=1; x\u0026lt;n+1; x++)\\n         \n                 {\\n                 \n                    a[x] = (m[x] + u[x])/2;\\n         \n                 }\\n         \n                 c=(n*4)-4;\\n         \n                 for(int a1=1; a1\u0026lt;n+1; a1++)\\n         \n                 {\\n\\n             \n                    e[a1][0] = m[a1];\\n             \n                    e[a1][1] = m[a1]+1;\\n             \n                    e[a1][2] = u[a1]-1;\\n             \n                    e[a1][3] = u[a1];\\n         \n                 }\\n         \n                 for(int i=1; i\u0026lt;n+1; i++)\\n         \n                 {\\n            \n                    for(int l=1; l\u0026lt;=i; l++)\\n            \n                    {\\n                 \n                        if(l!=1)\\n                 \n                        {\\n                    \n                            cout\u0026lt;\u0026lt;a[l]\u0026lt;\u0026lt;\"\\\\t\";\\n                 \n                        }\\n            \n                    }\\n            \n                    for(int j=0; j\u0026lt;4; j++)\\n            \n                    {\\n                \n                        cout\u0026lt;\u0026lt;e[i][j];\\n                \n                        for(int k=0; k\u0026lt;n-(i+1); k++)\\n                \n                        {\\n                    \n                            cout\u0026lt;\u0026lt;a[k]\u0026lt;\u0026lt;\"\\\\t\";\\n               \n                        }\\n                \n                        cout\u0026lt;\u0026lt;\"\\\\n\";\\n            \n                    }\\n        \n                 }    \\n\\n        \n                 system(\"PAUSE\");\\n        \n                 return 0;    \\n\n        }\\n\n        \u003c/code\u003e\u003c/pre\u003e\\n\\n\n        \u003cp\u003eThe answer should come in the form of a table like\u003c/p\u003e\\n\\n\n        \u003cpre\u003e\u003ccode\u003e       \n        1            50              50\\n       \n        2            50              50\\n       \n        99           50              50\\n       \n        100          50              50\\n       \n        50           1               50\\n       \n        50           2               50\\n       \n        50           99              50\\n       \n        50           100             50\\n       \n        50           50              1\\n       \n        50           50              2\\n       \n        50           50              99\\n       \n        50           50              100\\n\n        \u003c/code\u003e\u003c/pre\u003e\\n\\n\n        \u003cp\u003eif the no of inputs is 3 and their ranges are\\n\n        1,100\\n\n        1,100\\n\n        1,100\\n\n        (could be varied too)\u003c/p\u003e\\n\\n\n        \u003cp\u003eThe output is not coming,can anyone correct the code or tell me what\\'s wrong?\u003c/p\u003e\\n'\n\u003cb\u003eTags \u003c/b\u003e: 'c++ c'\n\u003c/pre\u003e\n\n### Mapping the real-world problem to a Machine Learning Problem\n\n\u003cp\u003e **It is a multi-label classification problem**\n\n\u003cbr\u003e\n\u003cb\u003eMulti-label Classification\u003c/b\u003e: Multilabel classification assigns to each sample a set of target labels. This can be thought as predicting properties of a data-point that are not mutually exclusive, such as topics that are relevant for a document. A question on Stackoverflow might be about any of C, Pointers, FileIO and/or memory-management at the same time or none of these. \u003cbr\u003e\n\u003c/p\u003e\n\n### Performance Metric \n\nThe [F1 score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html) can be interpreted as a weighted average of the precision and recall, where an F1 score reaches its best value at 1 and worst score at 0. The relative contribution of precision and recall to the F1 score are equal. The formula for the F1 score is:\n\n\u003ci\u003eF1 = 2 * (precision * recall) / (precision + recall)\u003c/i\u003e\u003cbr\u003e\n\nIn the multi-class and multi-label case, this is the weighted average of the F1 score of each class. \n\n* **'Micro f1 score':** \u003c/b\u003e\u003cbr\u003e\nCalculate metrics globally by counting the total true positives, false negatives and false positives. This is a better metric when we have class imbalance.\n\n* **'Macro f1 score':** \u003c/b\u003e\u003cbr\u003e\nCalculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.\n\n* [Hamming loss](https://www.kaggle.com/wiki/HammingLoss) \u003c/b\u003e: The Hamming loss is the fraction of labels that are incorrectly predicted.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsomjit101%2Fnlp-stackeroverflow-tag-prediction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsomjit101%2Fnlp-stackeroverflow-tag-prediction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsomjit101%2Fnlp-stackeroverflow-tag-prediction/lists"}