{"id":13696788,"url":"https://github.com/xiaohuiyan/BurstyBTM","last_synced_at":"2025-05-03T17:32:24.608Z","repository":{"id":25621062,"uuid":"29056279","full_name":"xiaohuiyan/BurstyBTM","owner":"xiaohuiyan","description":"Code for Bursty Biterm Topic Model (published in AAAI2015)","archived":false,"fork":false,"pushed_at":"2019-02-23T07:59:57.000Z","size":10788,"stargazers_count":53,"open_issues_count":2,"forks_count":27,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-08-03T18:21:15.688Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/xiaohuiyan/xiaohuiyan.github.io/blob/master/paper/BBTM-AAAI15.pdf","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xiaohuiyan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-10T12:17:18.000Z","updated_at":"2024-03-07T13:45:48.000Z","dependencies_parsed_at":"2022-07-14T00:40:34.352Z","dependency_job_id":null,"html_url":"https://github.com/xiaohuiyan/BurstyBTM","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaohuiyan%2FBurstyBTM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaohuiyan%2FBurstyBTM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaohuiyan%2FBurstyBTM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaohuiyan%2FBurstyBTM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xiaohuiyan","download_url":"https://codeload.github.com/xiaohuiyan/BurstyBTM/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224369751,"owners_count":17299950,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T18:00:47.149Z","updated_at":"2024-11-13T00:31:01.573Z","avatar_url":"https://github.com/xiaohuiyan.png","language":"C++","funding_links":[],"categories":["Models"],"sub_categories":["Topic Models for short documents"],"readme":"# Code of Bursty Biterm Topic Model\r\n\r\n\r\nBursty biterm topic model (BurstyBTM) is a topic model for bursty discovery in short text streams such as microblogs. \r\n\r\nMore detail can be referred to the following paper:\r\n\r\n\u003e Xiaohui Yan, Jiafeng Guo, Yanyan Lan, Jun Xu, and Xueqi Cheng. A Probabilistic Model for Bursty Topic Discovery in Microblogs. AAAI2015.\r\n\r\n## Usage ##\r\n\r\nThe code includes a runnable example, you can run it by:\r\n\t\r\n\t   $ cd script\r\n       $ ./runExample.sh\r\n\r\nIt trains BTM over the documents in *sample-data/0.txt, 1.txt, ...* and output the topics. The n.txt contains the training documents in day *n*, where each line represents one document with words separated by space as:\r\n\u003e word1 word2 word3 ....\r\n\r\n(*Note: the sample data is only used for illustration of the usage of the code. It is not the data set used in the paper.*)\r\n\r\nYou can change the paths of data files and parameters in *script/runExample.sh* to run over your own data. \r\n\r\nIndeed, the *runExample.sh* processes the input documents in 4 steps.\r\n\r\n**1. Index the words in the documents**   \r\n   To simplify the main code, we provide a python script to map each word to a unique ID (starts from 0) in the documents. \r\n\r\n     $ python script/indexDocs.py \u003cdoc_pt\u003e \u003cdwid_pt\u003e \u003cvoca_pt\u003e\r\n      doc_ptinput docs to be indexed, each line is a doc with the format \"word word ...\"\r\n      dwid_pt   output docs after indexing, each line is a doc with the format \"wordId wordId ...\"\r\n      voca_pt   output vocabulary file, each line is a word with the format \"wordId word\"\r\n\r\n**2. Statistic daily frequencies for each biterm**\r\n\r\n    $ python bitermDayFreq.py \u003cdwid_dir\u003cres_dir\u003e\r\n      dwid_pt   input docs, each line is a doc with the format \"word word ...\"\r\n      res_pt   output the frequencies of the biterms in the format \"w1 w2freq\"\r\n\r\n**3. Compute eta (bursty probability) in each day for biterms**\r\n\r\n    $ python eta.py \u003cn_day\u003e \u003cbf_dir\u003e \u003cres_dir\u003e\r\n\t  n_day   number of days to be processed (count from 0)\r\n\t  bf_pt   input docs, each line is a biterm with its daily frequency. Line format: \"w1 w2    day:freq day:freq ...\"\r\n\t  res_dir   output the eta of the biterms. Line format: \"w1 w2    eta\"\r\n\r\n\r\n**4. Topic learning**  \r\n   The next step is to train the model using the documents represented by word ids.    \r\n\r\n    $./src/bbtm \u003cK\u003e \u003cW\u003e \u003calpha\u003e \u003cbeta\u003e \u003cn_iter\u003e \u003cbiterm_pt\u003e \u003cmodel_dir\u003e \u003cfix_b\u003e\r\n\t type\t's' means simplified BurstyBTM, 'n' means normal BurstyBTM\r\n\t K\tint, number of topics, like 20\r\n\t W\tint, the size of vocabulary\r\n\t alpha\tdouble, Symmetric Dirichlet prior of P(z), like 1\r\n\t beta\tdouble, Symmetric Dirichlet prior of P(w|z), like 0.01\r\n\t n_iter\tint, number of iterations of Gibbs sampling\r\n\t biterm_pt\tstring, path of training biterms, each line is a biterm with the format 'wi wj eta'\r\n\t model_dir\tstring, output directory\r\n\t fix_b\t'y' means fixing the background word distribution to the empirical word distribution\r\n\r\n \r\n   The results will be written into the directory \"model\\_dir\":   \r\n   - k20.day1.type-n.iter100.pw_z: a K*M matrix for P(w|z), suppose K=20, day=1, iter=100, and did not use the simplified model   \r\n   - k20.day1.type-n.iter100.pz:   a K*1 matrix for P(z), suppose K=20\r\n\r\n  \r\n**5. Results display**    \r\n   Finally, we also provide a python script to illustrate the top words of the topics and their proportions in the collection. \r\n\r\n    $ python topicDisplay.py \u003cmodel_dir\u003e \u003cvoca_pt\u003e\r\n\t  model_dir    the output dir of BTM\r\n\t  K    the number of topics\r\n\t  voca_pt    the vocabulary file\r\n\r\n## Related codes ##\r\n- [Basic BTM](https://github.com/xiaohuiyan/BTM)\r\n- [Online BTM](https://github.com/xiaohuiyan/OnlineBTM)\r\n\r\n## History ##\r\n- 2015-01-13, v0.5, clean up\r\n- 2014-09-25, v0.1\r\n\r\nIf there is any question, feel free to contact: [Xiaohui Yan](http://shortext.org \"Xiaohui Yan\")(xhcloud@gmail.com).\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiaohuiyan%2FBurstyBTM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxiaohuiyan%2FBurstyBTM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiaohuiyan%2FBurstyBTM/lists"}