{"id":13407318,"url":"https://github.com/aiplanethub/beyondllm","last_synced_at":"2025-03-14T11:31:17.854Z","repository":{"id":234498830,"uuid":"769026721","full_name":"aiplanethub/beyondllm","owner":"aiplanethub","description":"Build, evaluate and observe LLM apps","archived":false,"fork":false,"pushed_at":"2025-01-27T05:32:27.000Z","size":2770,"stargazers_count":279,"open_issues_count":11,"forks_count":43,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-07T02:37:01.825Z","etag":null,"topics":["ai","artificial-intelligence","embeddings","evaluate-llm","genai","generative-ai","hacktoberfest","hacktoberfest-accepted","hacktoberfest2024","large-language-models","llm","llms","rag"],"latest_commit_sha":null,"homepage":"https://beyondllm.aiplanet.com/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aiplanethub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-08T07:33:31.000Z","updated_at":"2025-02-28T12:43:25.000Z","dependencies_parsed_at":"2024-05-28T16:29:35.426Z","dependency_job_id":"be469ca9-7b71-4fc8-bfff-c5c9cefad5cb","html_url":"https://github.com/aiplanethub/beyondllm","commit_stats":{"total_commits":91,"total_committers":18,"mean_commits":5.055555555555555,"dds":0.8131868131868132,"last_synced_commit":"90c49c4fe526dd302abdef42f43c4c514f1a27bb"},"previous_names":["aiplanethub/beyondllm"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiplanethub%2Fbeyondllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiplanethub%2Fbeyondllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiplanethub%2Fbeyondllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiplanethub%2Fbeyondllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aiplanethub","download_url":"https://codeload.github.com/aiplanethub/beyondllm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243569455,"owners_count":20312429,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","embeddings","evaluate-llm","genai","generative-ai","hacktoberfest","hacktoberfest-accepted","hacktoberfest2024","large-language-models","llm","llms","rag"],"created_at":"2024-07-30T20:00:36.480Z","updated_at":"2025-03-14T11:31:14.788Z","avatar_url":"https://github.com/aiplanethub.png","language":"Jupyter Notebook","funding_links":[],"categories":["Libraries","Jupyter Notebook","Neural Search and Retrieval"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ch1 align=\"center\"\u003eBeyondLLM\u003c/h1\u003e\n\u003ch2 align=\"center\"\u003eBuild - Rapid Experiment - Evaluate - Observability\u003c/h2\u003e\n\n![Thumbnails](https://github.com/aiplanethub/beyondllm/assets/132284203/4b8492f1-88ce-47a6-b17e-9e925e9fe54e)\n\n\u003ca href=\"https://img.shields.io/badge/Python-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11-3776AB.svg?style=flat\u0026logo=python\u0026logoColor=white\"\u003e\u003cimg src=\"https://img.shields.io/badge/Python-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11-3776AB.svg?style=flat\u0026logo=python\u0026logoColor=white\" alt=\"Python Versions\"\u003e\u003c/a\u003e\n\u003ca href=\"https://discord.gg/4aWV7He2QU\"\u003e\u003cimg src=\"https://dcbadge.vercel.app/api/server/4aWV7He2QU?style=flat\" alt=\"Discord\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://twitter.com/aiplanethub\"\u003e\u003cimg src=\"https://img.shields.io/twitter/follow/aiplanethub\" alt=\"Twitter\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://colab.research.google.com/drive/1S1UL2uCahHkfJsurRA3f7dcR6IHjg-IM?usp=sharing\" target=\"_blank\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Try an example\"/\u003e\u003c/a\u003e\n\n[![Contributors](https://img.shields.io/github/contributors/aiplanethub/beyondllm.svg?style=flat-square)](https://github.com/aiplanethub/beyondllm/graphs/contributors)\n[![Code of Conduct](https://img.shields.io/badge/code%20of%20conduct-contributor%20covenant-green.svg?style=flat-square)](CODE_OF_CONDUCT.md)\n\n\u003cp\u003eBeyond LLM offers an all-in-one toolkit for experimentation, evaluation, and deployment of Retrieval-Augmented Generation (RAG) systems, simplifying the process with automated integration, customizable evaluation metrics, and support for various Large Language Models (LLMs) tailored to specific needs, ultimately aiming to reduce LLM hallucination risks and enhance reliability.\u003c/p\u003e\n\u003ci\u003e\u003ca href=\"https://discord.gg/4aWV7He2QU\"\u003e👉 Join our Discord community!\u003c/a\u003e\u003c/i\u003e\n\u003c/div\u003e\n\nTry out a quick demo on Google Colab:\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1S1UL2uCahHkfJsurRA3f7dcR6IHjg-IM?usp=sharing)\n\n## Quick install\n\n```bash\npip install beyondllm\n```\n\n## Quickstart Guide- Chat with YouTube Video\n\nIn this quick start guide, we'll demonstrate how to create a Chat with YouTube video RAG application using Beyond LLM with less than 8 lines of code. This 8 lines of code includes:\n* Getting custom data source\n* Retrieving documents\n* Generating LLM responses\n* Evaluating embeddings\n* Evaluating LLM responses\n\n### Approach-1: Using Default LLM and Embeddings\n\nBuild customised RAG in less than ``5 lines of code`` using Beyond LLM. \n\n```python\nfrom beyondllm import source,retrieve,generator\nimport os\nos.environ['GOOGLE_API_KEY'] = \"Your Google API Key:\"\n\ndata = source.fit(\"https://www.youtube.com/watch?v=oJJyTztI_6g\",dtype=\"youtube\",chunk_size=512,chunk_overlap=50)\nretriever = retrieve.auto_retriever(data,type=\"normal\",top_k=3)\npipeline = generator.Generate(question=\"what tool is video mentioning about?\",retriever=retriever)\n\nprint(pipeline.call())\n```\n\n### Approach-2: With Custom LLM and Embeddings\n\nBeyond LLM support various Embeddings and LLMs that are two very important components in Retrieval Augmented Generation. \n\n```python\nfrom beyondllm import source,retrieve,embeddings,llms,generator\nimport os\nfrom getpass import getpass\nos.environ['OPENAI_API_KEY'] = getpass(\"Your OpenAI API Key:\")\n\ndata = source.fit(\"https://www.youtube.com/watch?v=oJJyTztI_6g\",dtype=\"youtube\",chunk_size=1024,chunk_overlap=0)\nembed_model = embeddings.OpenAIEmbeddings()\nretriever = retrieve.auto_retriever(data,embed_model,type=\"normal\",top_k=4)\nllm = llms.ChatOpenAIModel()\npipeline = generator.Generate(question=\"what tool is video mentioning about?\",retriever=retriever,llm=llm)\n\nprint(pipeline.call()) #AI response\nprint(retriever.evaluate(llm=llm)) #evaluate embeddings\nprint(pipeline.get_rag_triad_evals()) #evaluate LLM response\n```\n\n##### Output\n\n```bash\nThe tool mentioned in the context is called Jupiter, which is an AI Guru designed to simplify the learning of complex data science topics. Users can access Jupiter by logging into AI Planet, accessing any course for free, and then requesting explanations of topics from Jupiter in various styles, such as in the form of a movie plot. Jupiter aims to make AI education more accessible and interactive for everyone.\n\nHit_rate:1.0\nMRR:1.0\n\nContext relevancy Score: 8.0\nAnswer relevancy Score: 7.0\nGroundness score: 7.666666666666667\n```\n\n## Observability\n\n``Observability`` helps to keep track of the closed source models on the latency and the cost monitor tracking. BeyondLLM provides ``Observer`` that currently monitors the OpenAI LLM model performance. \n\n```python\nfrom beyondllm import source,retrieve,generator, llms, embeddings\nfrom beyondllm.observe import Observer\nimport os\n\nos.environ['OPENAI_API_KEY'] = 'sk-****'\n\nObserve = Observer()\nObserve.run()\n\nllm=llms.ChatOpenAIModel()\nembed_model = embeddings.OpenAIEmbeddings()\n\ndata = source.fit(\"https://www.youtube.com/watch?v=oJJyTztI_6g\",dtype=\"youtube\",chunk_size=512,chunk_overlap=50)\nretriever = retrieve.auto_retriever(data,embed_model,type=\"normal\",top_k=4)\n\npipeline = generator.Generate(question=\"what tool is video mentioning about?\",retriever=retriever, llm=llm)\npipeline = generator.Generate(question=\"What is the tool used for?\",retriever=retriever, llm=llm)\npipeline = generator.Generate(question=\"How can i use the tool for my own use?\",retriever=retriever, llm=llm)\n```\n\n## Documentation\n\nSee the [beyondllm.aiplanet.com](https://beyondllm.aiplanet.com/) for complete documentation.\n\n\n## Contribution guidelines\n\nBeyond LLM thrives in the rapidly evolving landscape of open-source projects. We wholeheartedly welcome contributions in various capacities, be it through innovative features, enhanced infrastructure, or refined documentation.\n\nSee [Contributing guide](https://github.com/aiplanethub/beyondllm/blob/main/CONTRIBUTING.md) for more information on contributing to the BeyondLLM library. \n\n## Acknowledgements\n\n* [HuggingFace](https://github.com/huggingface)\n* [LlamaIndex](https://github.com/jerryjliu/llama_index)\n* [OpenAI](https://github.com/openai)\n* [Google Gemini](https://ai.google.dev/)\n  \nand the entire OpenSource community.\n\n## License\n\nThe contents of this repository are licensed under the [Apache License, version 2.0](https://github.com/aiplanethub/beyondllm/blob/main/LICENSE).\n\n## Get in Touch\n\nYou can schedule a 1:1 meeting with our Team to get started with GenAI Stack, OpenAGI, AI Planet Open Source LLMs(Buddhi, effi and Panda Coder) and Beyond LLM. Schedule the call here: [https://calendly.com/jaintarun](https://calendly.com/jaintarun)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiplanethub%2Fbeyondllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faiplanethub%2Fbeyondllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiplanethub%2Fbeyondllm/lists"}