{"id":50350534,"url":"https://github.com/kifi/ReactiveLDA","last_synced_at":"2026-06-15T12:01:43.691Z","repository":{"id":18177130,"uuid":"21290623","full_name":"kifi/ReactiveLDA","owner":"kifi","description":"ReactiveLDA is a fast, lightweight implementation of the Latent Dirichlet Allocation (LDA) algorithm, using a parallel vanilla Gibbs sampling algorithm.","archived":false,"fork":false,"pushed_at":"2015-07-13T16:48:52.000Z","size":365,"stargazers_count":62,"open_issues_count":0,"forks_count":12,"subscribers_count":22,"default_branch":"master","last_synced_at":"2024-03-26T18:27:55.115Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.kifi.com/","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kifi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-06-27T22:07:10.000Z","updated_at":"2024-03-26T18:27:55.115Z","dependencies_parsed_at":"2022-09-14T13:00:31.308Z","dependency_job_id":null,"html_url":"https://github.com/kifi/ReactiveLDA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kifi/ReactiveLDA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kifi%2FReactiveLDA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kifi%2FReactiveLDA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kifi%2FReactiveLDA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kifi%2FReactiveLDA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kifi","download_url":"https://codeload.github.com/kifi/ReactiveLDA/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kifi%2FReactiveLDA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34361403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-15T02:00:07.085Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-29T21:00:23.928Z","updated_at":"2026-06-15T12:01:43.675Z","avatar_url":"https://github.com/kifi.png","language":"Scala","funding_links":[],"categories":["人工智能"],"sub_categories":["机器学习"],"readme":"ReactiveLDA\n===========\n\n__ReactiveLDA__ is a lightweight implementation of the [Latent Dirichlet Allocation](http://en.wikipedia.org/wiki/Latent_Dirichlet_allocation) (LDA) algorithm. The main ingredient is a parallel vanilla Gibbs sampling algorithm, and the parallelism is done via Akka actors (hence the naming 'reactive'). This approach is very different from a more popular algorithm based on collapsed Gibbs sampler, which is difficult to be parallelized, unless one assumes weak dependencies in sampling latent variables. \n\n# Introduction\n\n## The LDA Model\nThe LDA model assumes each word in a document is generated in two steps:\n- Sample a topic from the document-topic distribution (each document has its own document-topic distribution);\n- From the sampled topic, sample a word from the topic-word distribution (each topic has its own topic-word distribution). \n \nBoth the document-topic distribution and topic-word distribution are multinomial distributions. The model assumes that these distributions themselves are sampled from Dirichlet distributions. The objective of LDA learning algorithm is to infer the topic-word distribution, given observed words in a corpus. Such topic-word distribution can later give vector representation of words and documents. In addition to text analysis, it has been reported that LDA model can be applied to perform collabarative filtering.\n\n## The Gibbs Sampler\nIn this implementation, we use the vanilla Gibbs sampler. We sample word topics, docuemnt-topic distribution, and topic-word distribution alternatively. The sampling iteration goes like this:\n- Given topic-word distributions and the document-topic distribution for a document, we sample topic variables for words in the document.\n- After sampling topic variables for the document, we can sample (update) document-topic distribution for that document.\n- After we have sampled topic variables for all words in the corpus, we can sample topic-word distributions.\n\nTo kick off the iteration, in the first iteration, we perform uniform sampling for topic variables for all words in the corpus.\n\n\n\n## The Actor System\nThere are 3 types of actors:\n- `BetaActor` (master)\n  - Retrieve jobs from `MiniBatchActor`\n  - Distribute jobs to `DocSamplingActor`s\n  - Update `beta`, which gives topic-word distributions\n- `DocSamplingActor` (worker)\n  - Sample latent topic variable for each word in the document\n  - Sample document's topic distribution\n- `MiniBatchActor`\n  - Send batches of documents to `BetaActor`\n\n## Performance\nReactiveLDA has the following features:\n- Highly scalable Map-Reduce like job distribution. It would be interesting to extend the work by using remote Akka actors.\n- Memory friendly: no need to hold the entire corpus in memory. The major part of memory footprint is from model variables: topic-word distributions and document-topic distributions. We also provide an in-memory option, which can save I/O when the corpus can fit into memory. \n- Good speedup: empirical results suggest that ReactiveLDA achieves near-perfect parallel speedup (we have only performed tests up to 32 CPU cores).\n- Good speed: we have done some experiments with English wikipedia corpus (3M documents, 100K vocabulary size, 1.5 billion tokens, after filtering out redirected articles and low frequencey words). We train a topic model with 512 dimensions on an Amazon instance with 32 virtual CPUs, one iteration of Gibbs sampling takes about 10 minutes. A total number of 50 iterations usually gives reasonably good model. That is less than half a day (with a strong machine)! \n\n \n# How to Use the Library\n\n## Build the Jar\nAssuming you are at the project's root directory. Start `sbt` console and enter `assembly`. You should have `LDA.Jar` built under the folder `/target/scala-x.xx`.\n\n## Use the Jar\nYou can run the jar with minimal required arguments like this:\n```\njava -jar LDA.jar -nw 32 -t 100 -voc 123456 -iter 50 -b 10000 -in trainFile.txt -betaFile betaFile.bin\n```\n\nThe parameters aboves are:\n- -nw: number of workers\n- -t: number of topics\n- -voc: vocabulary size of your corpus\n- -iter: number of iterations of Gibbs sampling\n- -b: This defines a 'mini-batch' size. This is because corpus may not fit into memory. In this case, use an appropriate mini-batch size helps to hold a small portion of corpus in memory and perform sampling just for this part. If this value is too low, I/O overhead may reduce performance. \n- -in: path to the training file (training file format is explained below)\n- -betaFile: the file name to store the `beta` parameter. `beta` gives topic-word distributions. \n\nFor a comprehensive arguments list, please see documentations in the source code, or type `java -jar LDA.jar` for usage. \n\n## Training File Format\nFor simplicity and efficiency reasons, we assume the input file has the following format:\n- The file represents the whole corpus.\n- Each line represents a document in the corpus.\n- Each line is a sequence of `word-id`s, seperated by space. \n- word-ids form the integer set {0, 1, 2, ..., V-1}, where `V` is the vocabulary size. \n\nWe provide a util class to convert regular txt file corpus into this format. See `CorpusUtil.scala`. It should be straigtforward to use the class from `sbt` console. \n\n## Sample Run\nWe provide a toy training file `trivial.test.txt` under the `test` folder. This represents a corpus with 6 documents and 4 words. Word 0 and 1 form a topic, 2 and 3 form another topic. \n\n```\n0 0 0 0 1 1 1 1\n1 1 1 1 1 0 0 0\n1 1 1 0 0 0 0 0\n2 2 2 2 2 3 3 3\n3 2 2 2 3 3 3 2\n3 3 3 2 2 3 3 2\n```\n\nYou can train an LDA model on this simple corpus like this:\n```\njava -jar LDA.jar -nw 10 -t 2 -voc 4 -iter 50 -b 3 -in trivial.txt -betaFile trivial_beta.bin -verbose true\n```\nAt the end of training, you would see some console outputs like this\n```\nDEBUG: sampling dirichlet with 12.1 12.1 0.1 0.1\nDEBUG: sampled beta for topic 0: 0.45371073 0.5395452 2.3765434E-4 0.006506452\nDEBUG: sampling dirichlet with 0.1 0.1 12.1 12.1\nDEBUG: sampled beta for topic 1: 0.0011156916 1.17734814E-7 0.46746284 0.53142136\n```\n\nThat is, the final model consists of two topic-word distributions:\n```\ntopic 0: 0.45371073 0.5395452 2.3765434E-4 0.006506452\ntopic 1: 0.0011156916 1.17734814E-7 0.46746284 0.53142136\n```\nSo, topic 0 is defined by word 0 and word 1, and topic 1 is defined by word 2 and 3. \n\nOf course, this is an overly simplified example, and we know the correct number of topics a priori. In practice, one has to try a few different topic sizes and evaluate the quality of the model (e.g. by computing perplexity, or examine if similar words have similar topic distributions, etc). \n\n## Use the Trained Model\nWe provide a simple util class `ModelReader` to read the trained `beta` file. With that util class you can do the following:\n- Examine topic-word distributions, e.g. top words in a topic.\n- Examine word-topic distributions. This helps to evaluate model qualtiy, e.g. simialr words should have similar topic distribution.\n- Text classification: One can use the model to generate a low dimensional representation of a document. The vector representation is a probability distribution over topics. We provide two methods:\n  - A naive summation of word vectors. This is fast, but we ignore context information.\n  - An EM style inference. This takes account of word context. It's slightly slower than the first method, yet it potentailly gives better classification result. The iterative algorithm usually converges in a few steps. In fact, the naive summation alogrithm is just a special case of EM inference, with only one iteration.\n\nExamples (in `sbt` console):\n```\nimport com.kifi.lda.ModelReader\n\nval beta = ModelReader.parseBeta(\"beta.bin\")                    // generated by running the Jar\nval word2id = ModelReader.parseWord2id(\"word2id.json\")          // can be generated by CorpusUtil\nval reader = new ModelReader(beta, w2id)\n\n\n// show top 20 words in topic 1\nreader.showTopics(topic = 1, topK = 20)\n\n// show the top 10 topics associated with the word \"akka\"\nreader.showWordTopic(\"akka\", topK = 10)\n\n// write topic and topic words to a file\nval topics = reader.getAllTopics(topK: Int = 100)\nscala.tools.nsc.io.File(\"topic_words.txt\").writeAll(topics)\n\n// classification\nreader.classify(\"akka actor documentation\", topK = 3)\nreader.EM_inference(\"akka actor documentation\", topK = 3)\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkifi%2FReactiveLDA","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkifi%2FReactiveLDA","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkifi%2FReactiveLDA/lists"}