{"id":15576415,"url":"https://github.com/njsmith/pysrilm","last_synced_at":"2025-07-18T13:39:36.729Z","repository":{"id":145902210,"uuid":"5651850","full_name":"njsmith/pysrilm","owner":"njsmith","description":"An extremely simple Python wrapper for the SRI Language Modeling toolkit","archived":false,"fork":false,"pushed_at":"2014-10-04T02:30:28.000Z","size":130,"stargazers_count":70,"open_issues_count":8,"forks_count":20,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-12-30T01:42:02.376Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/njsmith.png","metadata":{"files":{"readme":"README","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-09-02T21:02:52.000Z","updated_at":"2024-03-26T14:59:58.000Z","dependencies_parsed_at":"2023-03-23T21:43:06.865Z","dependency_job_id":null,"html_url":"https://github.com/njsmith/pysrilm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fpysrilm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fpysrilm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fpysrilm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/njsmith%2Fpysrilm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/njsmith","download_url":"https://codeload.github.com/njsmith/pysrilm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232059243,"owners_count":18466730,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T18:48:53.967Z","updated_at":"2025-01-01T07:33:53.579Z","avatar_url":"https://github.com/njsmith.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This is an extremely simple Python wrapper for SRILM:\n  http://www.speech.sri.com/projects/srilm/\n\nBasically it lets you load a SRILM-format ngram model into memory, and\nthen query it directly from Python.\n\nRight now this is extremely bare-bones, just enough to do what I\nneeded, no fancy infrastructure at all. Feel free to send patches\nthough if you extend it!\n\nRequirements:\n  - SRILM\n  - Cython\n\nInstallation:\n  - Edit setup.py so that it can find your SRILM build files.\n  - To install in your Python environment, use:\n       python setup.py install\n    To just build the interface module:\n       python setup.py build_ext --inplace\n    which will produce srilm.so, which can be placed on your\n    PYTHONPATH and accessed as 'import srilm'.\n    \nUsage:\n\nfrom srilm import LM\n\n# Use lower=True if you passed -lower to ngram-count. lower=False is\n# default.\nlm = LM(\"path/to/model/from/ngram-count\", lower=True)\n\n# Compute log10(P(brown | the quick))\n#\n# Note that the context tokens are in *reverse* order, as per SRILM's\n# internal convention. I can't decide if this is a bug or not. If you\n# have a model of order N, and you pass more than (N-1) words, then\n# the first (N-1) entries in the list will be used. (I.e., the most\n# recent (N-1) context words.)\nlm.logprob_strings(\"brown\", [\"quick\", \"the\"])\n\n# We can also compute the probability of a sentence (this is just\n# a convenience wrapper):\n#   log10 P(The | \u003cs\u003e)\n#   + log10 P(quick | \u003cs\u003e The)\n#   + log10 P(brown | \u003cs\u003e The quick)\nlm.total_logprob_strings([\"The\", \"quick\", \"brown\"])\n\n# Internally, SRILM interns tokens to integers. You can convert back\n# and forth using the .vocab attribute on an LM object:\nidx = lm.vocab.intern(\"brown\")\nprint idx\nassert lm.vocab.extern(idx) == \"brown\"\n# .extern() returns None if an idx is unused for some reason.\n\n# There's a variant of .logprob_strings that takes these directly,\n# which is probably not really any faster, but sometimes is more\n# convenient if you're working with interned tokens anyway:\nlm.logprob(lm.vocab.intern(\"brown\"),\n           [lm.vocab.intern(\"quick\"),\n            lm.vocab.intern(\"the\"),\n           ])\n\n# There are detect \"magic\" tokens that don't actually represent anything\n# in the input stream, like \u003cs\u003e and \u003cunk\u003e. You can detect them like\nassert lm.vocab.is_non_word(lm.intern(\"\u003cs\u003e\"))\nassert not lm.vocab.is_non_word(lm.intern(\"brown\"))\n\n# Sometimes it's handy to have two models use the same indices for the\n# same words, i.e., share a vocab table. This can be done like:\nlm2 = LM(\"other/model\", vocab=lm.vocab)\n\n# This gives the index of the highest vocabulary word, useful for\n# iterating over the whole vocabulary. Unlike the Python convention\n# for describing ranges, this is the *inclusive* maximum:\nlm.vocab.max_interned()\n\n# And finally, let's put it together with an example of how to find\n# the max-probability continuation:\n#   argmax_w P(w | the quick)\n# by querying each word in the vocabulary in turn:\ncontext = [lm.vocab.intern(w) for w in [\"quick\", \"the\"]]\nbest_idx = None\nbest_logprob = -1e100\n# Don't forget the +1, because Python and SRILM disagree about how\n# ranges should work...\nfor i in xrange(lm.vocab.max_interned() + 1):\n    logprob = lm.logprob(i, context)\n    if logprob \u003e best_logprob:\n        best_idx = i\n        best_logprob = logprob\nbest_word = lm.vocab.extern(best_idx)\nprint \"Max prob continuation: %s (%s)\" % (best_word, best_logprob)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnjsmith%2Fpysrilm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnjsmith%2Fpysrilm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnjsmith%2Fpysrilm/lists"}