{"id":20310030,"url":"https://github.com/abeusher/malletwrapper","last_synced_at":"2026-02-01T21:02:01.115Z","repository":{"id":151391811,"uuid":"260779107","full_name":"abeusher/MalletWrapper","owner":"abeusher","description":"A Python wrapper for MALLET topic modeling","archived":false,"fork":false,"pushed_at":"2018-02-11T22:34:22.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-25T20:07:55.876Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abeusher.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-02T21:34:32.000Z","updated_at":"2020-06-17T21:03:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"fa98f91f-3cb5-4a0d-9154-185f87ac7917","html_url":"https://github.com/abeusher/MalletWrapper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/abeusher/MalletWrapper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abeusher%2FMalletWrapper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abeusher%2FMalletWrapper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abeusher%2FMalletWrapper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abeusher%2FMalletWrapper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abeusher","download_url":"https://codeload.github.com/abeusher/MalletWrapper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abeusher%2FMalletWrapper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28990682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T20:57:35.821Z","status":"ssl_error","status_checked_at":"2026-02-01T20:57:29.580Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T17:29:44.133Z","updated_at":"2026-02-01T21:02:01.098Z","avatar_url":"https://github.com/abeusher.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# MalletWrapper\nA Python wrapper for [MALLET](http://mallet.cs.umass.edu/about.php). Topic modeling only, for now.\n\n## Mallet Installation Instructions (Mac)\n1. [Download](http://mallet.cs.umass.edu/download.php) the Mallet directory\n2. [Download](http://www.oracle.com/technetwork/java/javase/downloads/index.html) JDK\n3. [Install](https://brew.sh) Homebrew\n4. Install Ant via ```brew install ant```\n```\ncd /dir/containing/mallet-2.x.x\nant\n```\n\n## MalletWrapper Usage\n\n```python\nfrom MalletWrapper import Mallet\n\nmodel = Mallet('/Users/mikeronayne/mallet-2.0.8/')\nmodel.import_dir(input='/Users/mikeronayne/mallet-2.0.8/sample-data/web/en')\nmodel.train_topics()\n\nprint(model.topic_keys) # see output_topic_keys parameter in Train Topics documentation\nprint(model.doc_topics) # see output_doc_topics parameter in Train Topics documentation\nprint(model.word_weights) # see topic_word_weights_file parameter in Train Topics documentation\n```\n\n```python\n# {topic # (int): {dirichlet parameter: float, words: list}, ... }\n{0: {'dirichlet': 0.5, 'words': ['rings', 'are', 'were', 'norway', 'ring', 'dust', 'only', 'number', 'may', 'moons', 'narrow', 'uranian', 'particles', 'discovered', 'uranus', 'survived', 'some', 'saga', 'including', 'system']}, ... }\n\n# {document # (int): {document name: e.g. file path (str), topics: {topic # (int): weight (float), ... }}, ... }\n{0: {'name': 'file:/Users/mikeronayne/mallet-2.0.8/sample-data/web/en/elizabeth_needham.txt', 'topics': {7: 0.3105263157894737, 6: 0.3, 1: 0.19473684210526315, 8: 0.07894736842105263, 0: 0.03684210526315789, 9: 0.02631578947368421, 2: 0.02631578947368421, 3: 0.015789473684210527, 5: 0.005263157894736842, 4: 0.005263157894736842}}, ... }\n\n# {topic # (int): {word (str): weight (float)}, ... }\n{0: {'elizabeth': 0.01, 'needham': 0.01, 'died': 0.01, 'may': 3.01, 'also': 0.01, 'known': 0.01, 'mother': 0.01, 'was': 0.01, 'english': 0.01, 'procuress': 0.01, 'and': 0.01, 'brothel-keeper': 1.01, 'th-century': 0.01, 'london': 0.01, ... }, ... }\n```\n\n## MalletWraper Documentation\n\n### Constructor\n\n```python\nMallet(mallet_dir, memory=1)\n```\n\nParameter | Type | Description | Default\n--- | --- | --- | ---\nmallet_dir | str | File path of Mallet-2.x.x directory\nmemory | int, float | Maximum gigabytes of memory to allocate to Mallet | 1\n\n### Importing Data\n\n#### Via Directory\n\n```python\nimport_dir(**kwargs)\n```\n\nParameter | Type | Description | Default\n--- | --- | --- | ---\ninput | str, list | The directories containing text files to be classified, one directory per class | null\npreserve_case | bool | If true, do not force all strings to lowercase. | False\nreplacement_files | str, list | Files containing string replacements, one per line: 'A B [tab] C' replaces A B with C; 'A B' replaces A B with A_B | null\ndeletion_files | str, list | Files containing strings to delete after replacements but before tokenization (ie multiword stop terms) | null\nremove_stopwords | bool | If true, remove a default list of common English \"stop words\" from the text. | False\nstoplist_file | str | Instead of the default list, read stop words from a file, one per line. Implies ```remove_stopwords``` | null\nextra_stopwords | str | Read whitespace-separated words from this file, and add them to either the default English stoplist or the list specified by ```stoplist_file```. | null\nstop_pattern_file | str | Read regular expressions from a file, one per line. Tokens matching these regexps will be removed. | null\nskip_header | bool | If true, in each document, remove text occurring before a blank line. This is useful for removing email or UseNet header | False\nskip_html | bool | If true, remove text occurring inside \u003c...\u003e, as in HTML or SGML. | False\ngram_sizes | int, str | Include among the features all n-grams of sizes specified. For example, to get all unigrams and bigrams, use ```gram_sizes='1,2'```. This option occurs after the removal of stop words, if removed. | 1\nencoding | str | Character encoding for input file | UTF-8\ntoken_regex | str | Regular expression used for tokenization. Example: ```[\\p{L}\\p{N}_]+\\|[\\p{P}]+``` (unicode letters, numbers and underscore OR all punctuation) | ```\\p{L}[\\p{L}\\p{P}]+\\p{L}```\nprint_output | bool | If true, print a representation of the processed data to standard output. This option is intended for debugging. | False\n\n#### Via File\n\n```python\nimport_file(**kwargs)\n```\n\nParameter | Type | Description | Default\n--- | --- | --- | ---\ninput | str | The file containing data to be classified, one instance per line | null\nline_regex | str | Regular expression containing regex-groups for label, name and data. | ```^(\\S*)[\\s,]*(\\S*)[\\s,]*(.*)$```\nname | int | The index of the group containing the instance name. Use 0 to indicate that the name field is not used. | 1\ndata |int | The index of the group containing the data. | 3\nremove_stopwords | bool | If true, remove a default list of common English \"stop words\" from the text. | False\nreplacement_files | str, list | Files containing string replacements, one per line: 'A B [tab] C' replaces A B with C; 'A B' replaces A B with A_B | null\ndeletion_files | str, list | Files containing strings to delete after replacements but before tokenization (ie multiword stop terms) | null\nstoplist_file | str | Instead of the default list, read stop words from a file, one per line. Implies ```remove_stopwords``` | null\nextra_stopwords | str | Read whitespace-separated words from this file, and add them to either the default English stoplist or the list specified by ```stoplist_file```. | null\nstop_pattern_file | str | Read regular expressions from a file, one per line. Tokens matching these regexps will be removed. | null\npreserve_case | bool | If true, do not force all strings to lowercase. | False\nencoding | str | Character encoding for input file | UTF-8\ntoken_regex | str | Regular expression used for tokenization. Example: ```[\\p{L}\\p{N}_]+\\|[\\p{P}]+``` (unicode letters, numbers and underscore OR all punctuation) | ```\\p{L}[\\p{L}\\p{P}]+\\p{L}```\nprint_output | bool | If true, print a representation of the processed data to standard output. This option is intended for debugging. | False\n\n### Train Topics\n\n```python\ntrain_topics(**kwargs)\n```\n\nParameter | Type | Description | Default\n--- | --- | --- | ---\ninput_model | str | The filename from which to read the binary topic model. The ```input``` option is ignored. By default this is null, indicating that no file will be read. | null\ninput_state | str | The filename from which to read the gzipped Gibbs sampling state created by ```output_state```. The original input file must be included, using ```input```. By default this is null, indicating that no file will be read. | null\noutput_model | str | The filename in which to write the binary topic model at the end of the iterations. By default this is null, indicating that no file will be written. | null\noutput_state | str | The filename in which to write the Gibbs sampling state after at the end of the iterations. By default this is null, indicating that no file will be written. | null\noutput_model_interval | int | The number of iterations between writing the model (and its Gibbs sampling state) to a binary file. You must also set the output_model to use this option, whose argument will be the prefix of the filenames. | 0\noutput_state_interval | int | The number of iterations between writing the sampling state to a text file. You must also set the ```output_state``` to use this option, whose argument will be the prefix of the filenames. | 0\ninferencer_filename | str | A topic inferencer applies a previously trained topic model to new documents. By default this is null, indicating that no file will be written. | null\nevaluator_filename | str | A held-out likelihood evaluator for new documents. By default this is null, indicating that no file will be written. | null\noutput_topic_keys | str | The filename in which to write the top words for each topic and any Dirichlet parameters. By default this is null, indicating that no file will be written. | null\nnum_top_words | int | The number of most probable words to print for each topic after model estimation. | 20\nshow_topics_interval | int | The number of iterations between printing a brief summary of the topics so far. | 50\ntopic_word_weights_file | str | The filename in which to write unnormalized weights for every topic and word type. By default this is null, indicating that no file will be written. | null\nword_topic_counts_file | str | The filename in which to write a sparse representation of topic-word assignments. By default this is null, indicating that no file will be written. | null\ndiagnostics_file | str | The filename in which to write measures of topic quality, in XML format. By default this is null, indicating that no file will be written. | null\nxml_topic_report | str | The filename in which to write the top words for each topic and any Dirichlet parameters in XML format. By default this is null, indicating that no file will be written. | null\nxml_topic_phrase_report | str | The filename in which to write the top words and phrases for each topic and any Dirichlet parameters in XML format. By default this is null, indicating that no file will be written. | null\noutput_topic_docs | str | The filename in which to write the most prominent documents for each topic, at the end of the iterations. By default this is null, indicating that no file will be written. | null\nnum_top_docs | int | When writing topic documents with ```output_topic_docs```, report this number of top documents. | 100\noutput_doc_topics | str | The filename in which to write the topic proportions per document, at the end of the iterations. By default this is null, indicating that no file will be written. | null\ndoc_topics_threshold | float | When writing topic proportions per document with ```output_doc_topics```, do not print topics with proportions less than this threshold value. | 0.0\ndoc_topics_max | int | When writing topic proportions per document with ```output_doc_topics```, do not print more than INTEGER number of topics. A negative value indicates that all topics should be printed. | -1\nnum_topics | int | The number of topics to fit. | 10\nnum_threads | int | The number of threads for parallel training. | 1\nnum_iterations | int | The number of iterations of Gibbs sampling. | 1000\nnum_icm_iterations | int | The number of iterations of iterated conditional modes (topic maximization). | 0\nno_inference | bool | Do not perform inference, just load a saved model and create a report. Equivalent to ```num_iterations``` 0. | False\nrandom_seed | int | The random seed for the Gibbs sampler. Default is 0, which will use the clock. | 0\noptimize_interval | int | The number of iterations between reestimating dirichlet hyperparameters. | 0\noptimize_burn_in | int | The number of iterations to run before first estimating dirichlet hyperparameters. | 200\nuse_symmetric_alpha | bool | Only optimize the concentration parameter of the prior over document-topic distributions. This may reduce the number of very small, poorly estimated topics, but may disperse common words over several topics. | False\nalpha | float | SumAlpha parameter: sum over topics of smoothing over doc-topic distributions. alpha_k = [this value] / [num topics] | 5.0\nbeta | float | Beta parameter: smoothing parameter for each topic-word. beta_w = [this value] | 0.01\n\n## Future Improvements\n\n * Provide interface to move away from file reading (e.g. no extra stopwords file)\n * Better error handling, especially checking for bad inputs\n * Classification\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabeusher%2Fmalletwrapper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabeusher%2Fmalletwrapper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabeusher%2Fmalletwrapper/lists"}