{"id":25126792,"url":"https://github.com/joshprk/agentrec","last_synced_at":"2025-04-23T13:10:12.964Z","repository":{"id":273876075,"uuid":"909870480","full_name":"joshprk/agentrec","owner":"joshprk","description":"A recommendation system that picks out the best AI agents for a given natural language prompt","archived":false,"fork":false,"pushed_at":"2025-02-02T22:55:17.000Z","size":2820,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-23T13:09:50.370Z","etag":null,"topics":["agentic-ai","artificial-intelligence","machine-learning","multiagent-systems","natural-language-processing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joshprk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-29T23:56:32.000Z","updated_at":"2025-03-07T22:21:59.000Z","dependencies_parsed_at":"2025-01-23T14:19:52.506Z","dependency_job_id":"f04271bb-4378-473c-bff1-ea95f90a56d3","html_url":"https://github.com/joshprk/agentrec","commit_stats":null,"previous_names":["joshprk/agentrec"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshprk%2Fagentrec","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshprk%2Fagentrec/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshprk%2Fagentrec/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshprk%2Fagentrec/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joshprk","download_url":"https://codeload.github.com/joshprk/agentrec/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250439288,"owners_count":21430823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-ai","artificial-intelligence","machine-learning","multiagent-systems","natural-language-processing"],"created_at":"2025-02-08T10:37:41.592Z","updated_at":"2025-04-23T13:10:12.919Z","avatar_url":"https://github.com/joshprk.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AgentRec\n\nAgentRec is a library for building small end-to-end models for agent\nrecommendation in multiagent systems. It contains functions for training,\ntesting, and evaluating robust agent recommendation systems. Unlike traditional\nclassification techniques, it is adaptive to new classes as the computation of\nprior embeddings need not change. In the original paper, this library was able to\nproduce a model with a top-1 accuracy of 92.2% with a \u003e=300 ms evaluation time.\n\n## Implementation\n\nTo recommend an agent, it is important to find a knowledge representation that\nallows us to compare and contrast the capabilities of LLM agents numerically.\nOur research found that we can achieve this by encoding agent capabilities as\na corpus of many single-sentence user prompts which we know that a given LLM\nagent can answer.\n\nAfterwards, an unseen prompt is evaluated by comparing it to the many prompts\nalready labeled in the agent capabilities corpora. A score function which\nproduces a rank out of hundreds of comparisons then generates the top-k agent\nrecommendations.\n\n![A picture of the architecture](./figures/architecture.png)\n\n## Usage\n\nIt is important to first install the dependencies. This repository was written\nusing Python 3.12. It is recommended to use a venv with the following commands:\n\n```bash\npython -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\nA LLM is also necessary, which must produce the input and output of an OpenAI\nformatted context. More details are available in the documentation.\n\nA synthetic dataset can be generated using the `agentrec.datasets.PromptPool`\nclass. A list of agent names must be given, which can have optional\ndescriptions and examples tied to them. This dataset can then be saved through\nserializing the `PromptPool`. A usage example is given in `generatedata.py`.\nNote that a Huggingface API key must be inserted in `.env`.\n\nAfterwards, one can finetune the model by using `finetune.py`. After finetuning,\nit is possible to try out the agent recommendation system by using `test.py`.\nOtherwise, pre-trained model weights are available in the releases.\n\n## References\n\nIf you find this repository helpful, please feel free to cite our work.\n\n```\n@misc{park2025agentrecagentrecommendationusing,\n  title={AgentRec: Agent Recommendation Using Sentence Embeddings Aligned to Human Feedback}, \n  author={Joshua Park and Yongfeng Zhang},\n  year={2025},\n  eprint={2501.13333},\n  archivePrefix={arXiv},\n  primaryClass={cs.LG},\n  url={https://arxiv.org/abs/2501.13333}, \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoshprk%2Fagentrec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoshprk%2Fagentrec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoshprk%2Fagentrec/lists"}