{"id":13816761,"url":"https://github.com/jzbjyb/LPAQA","last_synced_at":"2025-05-15T18:32:29.492Z","repository":{"id":44402011,"uuid":"224398629","full_name":"jzbjyb/LPAQA","owner":"jzbjyb","description":"Language model Prompt And Query Archive","archived":false,"fork":false,"pushed_at":"2021-05-11T04:49:20.000Z","size":890,"stargazers_count":157,"open_issues_count":5,"forks_count":14,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-11-19T14:42:20.635Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jzbjyb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-27T09:54:39.000Z","updated_at":"2024-10-21T22:33:54.000Z","dependencies_parsed_at":"2022-07-15T06:47:08.203Z","dependency_job_id":null,"html_url":"https://github.com/jzbjyb/LPAQA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jzbjyb%2FLPAQA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jzbjyb%2FLPAQA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jzbjyb%2FLPAQA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jzbjyb%2FLPAQA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jzbjyb","download_url":"https://codeload.github.com/jzbjyb/LPAQA/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254397911,"owners_count":22064581,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T06:00:20.310Z","updated_at":"2025-05-15T18:32:29.154Z","avatar_url":"https://github.com/jzbjyb.png","language":"Shell","funding_links":[],"categories":["Factual Knowledge Probes","Papers"],"sub_categories":["Language Models"],"readme":"# LPAQA: Language model Prompt And Query Archive\n\nThis repository contains data and code for the paper [How Can We Know What Language Models Know?](https://arxiv.org/abs/1911.12543)\n\n## Install\nOur repository is based on [LAMA](https://github.com/facebookresearch/LAMA). Please download our fork\n from [here](https://github.com/jzbjyb/LAMA) and follow the instructions to set up the environment and download pre-trained language models.\n```bash\n# clone LPAQA\ngit clone https://github.com/jzbjyb/LPAQA.git LPAQA\n# clone LAMA\ngit clone https://github.com/jzbjyb/LAMA.git LAMA\npushd LAMA \u0026\u0026 git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 \u0026\u0026 rm -rf .git \u0026\u0026 popd\nmv LAMA/* LPAQA/ \u0026\u0026 rm -rf LAMA\n# follow the instructions to install\ncd LPAQA\n./setup.sh\n```\n\n## Retrieve factual knowledge from LMs\n\nFor example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (*x is owned by y*):\n```bash\npython lama/eval_generation.py --lm bert --t \"MSN is owned by [MASK].\"\n```\n\u003cimg align=\"middle\" src=\"demo/manual.png\" height=\"200\" alt=\"manual\"/\u003e\n\nor use LPAQA that ensembles a diversity of prompts:\n```bash\n# mined prompts\npython lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine\n# paraphrased prompts\npython lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase\n```\n\u003cimg align=\"middle\" src=\"demo/mine.png\" height=\"200\" alt=\"mine\"/\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjzbjyb%2FLPAQA","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjzbjyb%2FLPAQA","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjzbjyb%2FLPAQA/lists"}