{"id":13576728,"url":"https://github.com/robocorp/llmfoo","last_synced_at":"2025-04-27T19:33:10.551Z","repository":{"id":210521036,"uuid":"726765659","full_name":"robocorp/llmfoo","owner":"robocorp","description":"Code with the flow of a river, refactor with the grace of a breeze, and commit like the thunder — always moving, gently improving, and powerfully finalizing.","archived":false,"fork":false,"pushed_at":"2024-07-06T01:34:32.000Z","size":1605,"stargazers_count":14,"open_issues_count":3,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-05T03:03:10.275Z","etag":null,"topics":["genai","llm","openai","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robocorp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-03T10:39:46.000Z","updated_at":"2024-07-27T06:55:13.000Z","dependencies_parsed_at":"2024-05-03T23:23:46.764Z","dependency_job_id":"dc0ee396-f92d-40fb-9632-ebec4f188fca","html_url":"https://github.com/robocorp/llmfoo","commit_stats":null,"previous_names":["robocorp/llmfoo"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robocorp%2Fllmfoo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robocorp%2Fllmfoo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robocorp%2Fllmfoo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robocorp%2Fllmfoo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robocorp","download_url":"https://codeload.github.com/robocorp/llmfoo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251196471,"owners_count":21550958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["genai","llm","openai","python"],"created_at":"2024-08-01T15:01:13.236Z","updated_at":"2025-04-27T19:33:09.723Z","avatar_url":"https://github.com/robocorp.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# LLM FOO\r\n\r\n[![Version](https://img.shields.io/pypi/v/llmfoo.svg)](https://pypi.python.org/pypi/llmfoo)\r\n[![Downloads](http://pepy.tech/badge/llmfoo)](http://pepy.tech/project/llmfoo)\r\n\r\n## Overview\r\nLLM FOO is a cutting-edge project blending the art of Kung Fu with the science of Large Language Models... or \r\nactually this is about automatically making the OpenAI tool JSON Schema, parsing call and constructing the\r\nresult to the chat model.\r\nAnd then there is a second utility `is_statement_true` that uses [genius logit_bias trick](https://twitter.com/AAAzzam/status/1669753721574633473)\r\nthat only uses one output token.\r\n\r\nBut hey I hope this will become a set of small useful LLM helper functions that will make building stuff easier\r\nbecause current bleeding edge APIs are a bit of a mess and I think we can do better.\r\n\r\n![](/llmfoo.webp)\r\n\r\n## Installation\r\n```bash\r\npip install llmfoo\r\n```\r\n\r\n## Usage\r\n\r\n* You need to have OPENAI_API_KEY in env and ability to call `gpt-4-1106-preview` model\r\n\r\n* `is_statement_true` should be easy to understand.\r\nMake some natural language statement, and check it against criteria or general truthfulness. You get back boolean.\r\n\r\n### Converting PDF Documents to Markdown\r\n\r\nNewly introduced, `pdf2md` functionality allows for the conversion of PDF documents into Markdown format, making them easier to process and integrate with LLM-based systems. This feature is particularly useful for extracting text and tables from PDFs and transforming them into a more manageable format.\r\n\r\n#### Prerequisites\r\n\r\n- `pdftocairo` must be installed on your system for converting PDF pages to images.\r\n\r\n#### Example Usage\r\n\r\n```python\r\nfrom llmfoo.pdf2md import process_pdf\r\nfrom pathlib import Path\r\n\r\npdf_path = Path(\"path/to/your/document.pdf\")\r\noutput_dir = Path(\"path/to/output/directory\")\r\n\r\n# Process the PDF and generate Markdown\r\nmarkdown_file = process_pdf(pdf_path, output_dir)\r\n```\r\n\r\nThis function will process each page of the PDF, attempting to extract text, figures and tables, and convert them into a Markdown file in the specified output directory.\r\n\r\n\r\n### For the LLM FOO tool:\r\n\r\n1. Add `@tool` annotation.\r\n2. llmfoo will generate the json schema to YOURFILE.tool.json with GPT-4-Turbo - \"Never send a human to do a machine's job\" .. like who wants to write boilerplate docs for Machines???\r\n3. Annotated functions have helpers:\r\n   - `openai_schema` to return the schema (You can edit it from the json if your not happy with what the machines did)\r\n   - `openai_tool_call` to make the tool call and return the result in chat API message format\r\n   - `openai_tool_output` to make the tool call and return the result in assistant API tool output format\r\n\r\n```python\r\nfrom time import sleep\r\n\r\nfrom openai import OpenAI\r\n\r\nfrom llmfoo.functions import tool\r\nfrom llmfoo import is_statement_true\r\n\r\n\r\ndef test_is_statement_true_with_default_criteria():\r\n    assert is_statement_true(\"Earth is a planet.\")\r\n    assert not is_statement_true(\"1 + 2 = 5\")\r\n\r\n\r\ndef test_is_statement_true_with_own_criteria():\r\n    assert not is_statement_true(\"Temperature outside is -2 degrees celsius\",\r\n                                 criteria=\"Temperature above 0 degrees celsius\")\r\n    assert is_statement_true(\"1984 was written by George Orwell\",\r\n                             criteria=\"George Orwell is the author of 1984\")\r\n\r\n\r\ndef test_is_statement_true_criteria_can_change_truth_value():\r\n    assert is_statement_true(\"Earth is 3rd planet from the Sun\")\r\n    assert not is_statement_true(\"Earth is 3rd planet from the Sun\",\r\n                                 criteria=\"Earth is stated to be 5th planet from the Sun\")\r\n\r\n\r\n@tool\r\ndef adder(x: int, y: int) -\u003e int:\r\n    return x + y\r\n\r\n\r\n@tool\r\ndef multiplier(x: int, y: int) -\u003e int:\r\n    return x * y\r\n\r\n\r\nclient = OpenAI()\r\n\r\n\r\ndef test_chat_completion_with_adder():\r\n    number1 = 3267182746\r\n    number2 = 798472847\r\n    messages = [\r\n        {\r\n            \"role\": \"user\",\r\n            \"content\": f\"What is {number1} + {number2}?\"\r\n        }\r\n    ]\r\n    response = client.chat.completions.create(\r\n        model=\"gpt-4-1106-preview\",\r\n        messages=messages,\r\n        tools=[adder.openai_schema]\r\n    )\r\n    messages.append(response.choices[0].message)\r\n    messages.append(adder.openai_tool_call(response.choices[0].message.tool_calls[0]))\r\n    response2 = client.chat.completions.create(\r\n        model=\"gpt-4-1106-preview\",\r\n        messages=messages,\r\n        tools=[adder.openai_schema]\r\n    )\r\n    assert str(adder(number1, number2)) in response2.choices[0].message.content.replace(\",\", \"\")\r\n\r\n\r\ndef test_assistant_with_multiplier():\r\n    number1 = 1238763428176\r\n    number2 = 172388743612\r\n    assistant = client.beta.assistants.create(\r\n        name=\"The Calc Machina\",\r\n        instructions=\"You are a calculator with a funny pirate accent.\",\r\n        tools=[multiplier.openai_schema],\r\n        model=\"gpt-4-1106-preview\"\r\n    )\r\n    thread = client.beta.threads.create(messages=[\r\n        {\r\n            \"role\":\"user\",\r\n            \"content\":f\"What is {number1} * {number2}?\"\r\n        }\r\n    ])\r\n    run = client.beta.threads.runs.create(\r\n        thread_id=thread.id,\r\n        assistant_id=assistant.id\r\n    )\r\n    while True:\r\n        run_state = client.beta.threads.runs.retrieve(\r\n            run_id=run.id,\r\n            thread_id=thread.id,\r\n        )\r\n        if run_state.status not in ['in_progress', 'requires_action']:\r\n            break\r\n        if run_state.status == 'requires_action':\r\n            tool_call = run_state.required_action.submit_tool_outputs.tool_calls[0]\r\n            run = client.beta.threads.runs.submit_tool_outputs(\r\n                thread_id=thread.id,\r\n                run_id=run.id,\r\n                tool_outputs=[\r\n                    multiplier.openai_tool_output(tool_call)\r\n                ]\r\n            )\r\n            sleep(1)\r\n        sleep(0.1)\r\n    messages = client.beta.threads.messages.list(thread_id=thread.id)\r\n    assert str(multiplier(number1, number2)) in messages.data[0].content[0].text.value.replace(\",\", \"\")\r\n\r\n```\r\n\r\n## Contributing\r\nInterested in contributing? Loved to get your help to make this project better!\r\nThe APIs under are changing and system is still very much first version.\r\n\r\n## License\r\nThis project is licensed under the [MIT License](LICENSE).\r\n\r\n## Acknowledgements\r\n- Thanks to all the contributors and maintainers.\r\n- Special thanks to the Kung Fu masters such as Bruce Lee who inspired this project.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobocorp%2Fllmfoo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobocorp%2Fllmfoo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobocorp%2Fllmfoo/lists"}