{"id":20530643,"url":"https://github.com/juandspy/llm-assert","last_synced_at":"2025-10-06T02:58:31.414Z","repository":{"id":226386302,"uuid":"767713975","full_name":"juandspy/llm-assert","owner":"juandspy","description":"Use any LLM model to assert unit tests for you","archived":false,"fork":false,"pushed_at":"2024-03-07T09:26:15.000Z","size":7,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-02T13:10:47.474Z","etag":null,"topics":["ai","chatgpt","llm","lmstudio","openai","poc","pytest","testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/juandspy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-05T19:10:41.000Z","updated_at":"2025-01-06T03:06:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"2f129fc2-e806-413c-bf55-8cbc046149a5","html_url":"https://github.com/juandspy/llm-assert","commit_stats":null,"previous_names":["juandspy/llm-assert"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/juandspy/llm-assert","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juandspy%2Fllm-assert","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juandspy%2Fllm-assert/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juandspy%2Fllm-assert/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juandspy%2Fllm-assert/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/juandspy","download_url":"https://codeload.github.com/juandspy/llm-assert/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juandspy%2Fllm-assert/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278551496,"owners_count":26005389,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatgpt","llm","lmstudio","openai","poc","pytest","testing"],"created_at":"2024-11-15T23:39:33.196Z","updated_at":"2025-10-06T02:58:31.399Z","avatar_url":"https://github.com/juandspy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Asserting with LLMs\n\nA tool for lazy people that don't want to spend time on writing tests.\n\n(Just a project that I wrote for fun)\n\n## How to run\n\nInstall the required packages with `pip install .`\n\nLoad any model on [LM Studio](https://lmstudio.ai) and start the \"Local Inference Server\".\nAssuming it's configured to expose the port `1234` on localhost, you can use this tool\non your unit tests like this:\n\n```\n\u003e\u003e\u003e from src.main import llm_assert\n\u003e\u003e\u003e from src.engines.lmstudio import LMStudioEngine\n\u003e\u003e\u003e engine = LMStudioEngine(\"http://localhost:1234/v1\")\n\u003e\u003e\u003e llm_assert(engine, \"5\", \"this is a number\")\n\u003e\u003e\u003e llm_assert(engine, \"5\", \"this is not a number\")\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"/Users/jdiazsua/Documents/Projects/PoCs/assert-llm/src/main.py\", line 49, in llm_assert\n    assert answer == \"True\", f\"'{sut}' doesn't match the prompt '{prompt}'\"\n           ^^^^^^^^^^^^^^^^\nAssertionError: '5' doesn't match the prompt 'this is not a number'\n```\n\nThis looks like an overkill, but it can be useful for lazy checks like:\n\n```\n\u003e\u003e\u003e input_text = \"\"\"\nLorem ipsum dolor sit amet, consectetur adipiscing elit.\n123 Elm Street, Springfield, IL 62704.\nVivamus aliquet, augue id semper varius, ex tellus luctus justo, nec viverra metus lectus nec magna.\n\"\"\"\n\u003e\u003e\u003e llm_assert(engine, input_text, \"it contains an address\")\n\u003e\u003e\u003e llm_assert(engine, input_text, \"it contains a telephone number\")\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"/Users/jdiazsua/Documents/Projects/PoCs/assert-llm/src/main.py\", line 49, in llm_assert\n    assert answer == \"True\", f\"'{sut}' doesn't match the prompt '{prompt}'\"\n           ^^^^^^^^^^^^^^^^\nAssertionError: '\nLorem ipsum dolor sit amet, consectetur adipiscing elit.\n123 Elm Street, Springfield, IL 62704.\nVivamus aliquet, augue id semper varius, ex tellus luctus justo, nec viverra metus lectus nec magna.\n' doesn't match the prompt 'it contains a telephone number'\n```\n\n## Contributing\n\nFeel free to add new engines to the [engines](src/engines/) package. The only requirement is that\nthey must implement the `Engine` interface, as it will be used by the `llm_assert` function.\n\nYou can also contribute by adding new functions with different `system_message` and `prompt`\nor calling different APIs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuandspy%2Fllm-assert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuandspy%2Fllm-assert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuandspy%2Fllm-assert/lists"}