{"id":23029715,"url":"https://github.com/antononcube/raku-www-llama","last_synced_at":"2025-04-02T20:25:51.309Z","repository":{"id":225158650,"uuid":"764868159","full_name":"antononcube/Raku-WWW-LLaMA","owner":"antononcube","description":"Raku package that provides access to the algorithms/models of (the executable LLM) llamafile.","archived":false,"fork":false,"pushed_at":"2024-03-16T21:28:33.000Z","size":1260,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-08T11:13:21.632Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Raku","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"artistic-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antononcube.png","metadata":{"files":{"readme":"README-work.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-28T21:20:54.000Z","updated_at":"2024-03-19T15:35:43.000Z","dependencies_parsed_at":"2024-03-08T14:43:52.905Z","dependency_job_id":"89b51c19-7c81-493c-be3d-5606332cf293","html_url":"https://github.com/antononcube/Raku-WWW-LLaMA","commit_stats":null,"previous_names":["antononcube/raku-www-llamafile","antononcube/raku-www-llama"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-WWW-LLaMA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-WWW-LLaMA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-WWW-LLaMA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antononcube%2FRaku-WWW-LLaMA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antononcube","download_url":"https://codeload.github.com/antononcube/Raku-WWW-LLaMA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246886513,"owners_count":20849881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-15T14:16:55.878Z","updated_at":"2025-04-02T20:25:51.280Z","avatar_url":"https://github.com/antononcube.png","language":"Raku","funding_links":[],"categories":[],"sub_categories":[],"readme":"# WWW::LLaMA\n\n## In brief\n\nThis Raku package provides access to the machine learning service [llamafile](https://github.com/mozilla-Ocho/llamafile), [MO1].\nFor more details of the llamafile's API usage see [the documentation](https://github.com/mozilla-Ocho/llamafile?tab=readme-ov-file), [MO2].\n\nThis package is very similar to the packages \n[\"WWW::OpenAI\"](https://github.com/antononcube/Raku-WWW-OpenAI), [AAp1], and \n[\"WWW::MistralAI\"](https://github.com/antononcube/Raku-WWW-MistralAI), [AAp2]. \n\n\"WWW::LLaMA\" can be used with (is integrated with) \n[\"LLM::Functions\"](https://github.com/antononcube/Raku-LLM-Functions), [AAp3], and\n[\"Jupyter::Chatbook\"](https://github.com/antononcube/Raku-Jupyter-Chatbook), [AAp5].\n\nAlso, of course, prompts from \n[\"LLM::Prompts\"](https://github.com/antononcube/Raku-LLM-Prompts), [AAp4],\ncan be used with LLaMA's functions.\n\n**Remark:** The package \"WWW::OpenAI\" can be also used to access \n[\"llamafile\" chat completions](https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/server/README.md#api-endpoints).\nThat is done by specifying appropriate base URL to the `openai-chat-completion` function.\n\n-----\n\n## Installation\n\nPackage installations from both sources use [zef installer](https://github.com/ugexe/zef)\n(which should be bundled with the \"standard\" Rakudo installation file.)\n\nTo install the package from [Zef ecosystem](https://raku.land/) use the shell command:\n\n```\nzef install WWW::LLaMA\n```\n\nTo install the package from the GitHub repository use the shell command:\n\n```\nzef install https://github.com/antononcube/Raku-WWW-LLaMA.git\n```\n\n----\n\n## Install and run LLaMA server\n\nIn order to use the package access to LLaMA server is required.\n\nSince the package follows closely the Web API of [\"llamafile\"](https://github.com/Mozilla-Ocho/llamafile/), [MO1],\nit is advised to follow first the installation steps in the section of [\"Quickstart\"](https://github.com/Mozilla-Ocho/llamafile/tree/main#quickstart)\nof [MO1] before trying the functions of the package.\n\n----\n\n## Usage examples\n\n**Remark:** When the authorization key, `auth-key`, is specified to be `Whatever`\nthen it is assigned the string `sk-no-key-required`.\nIf an authorization key is required then the env variable `LLAMA_API_KEY` can be also used.\n\n### Universal \"front-end\"\n\nThe package has an universal \"front-end\" function `llama-playground` for the \n[different functionalities provided by llamafile](https://github.com/Mozilla-Ocho/llamafile/blob/main/README.md).\n\nHere is a simple call for a \"chat completion\":\n\n```perl6\nuse WWW::LLaMA;\nllama-playground('What is the speed of a rocket leaving Earth?');\n```\n\nAnother one using Bulgarian:\n\n```perl6\nllama-playground('Колко групи могат да се намерят в този облак от точки.', max-tokens =\u003e 300, random-seed =\u003e 234232, format =\u003e 'values');\n```\n\n**Remark:** The functions `llama-chat-completion` or `llama-completion` can be used instead in the examples above.\n(The latter is synonym of the former.)\n\n\n### Models\n\nThe current LLaMA model can be found with the function `llama-model`:\n\n```perl6\nllama-model;\n```\n\n**Remark:** Since there is no dedicated API endpoint for getting the model(s),\nthe current model is obtained via \"simple\" (non-chat) completion.\n\n### Code generation\n\nThere are two types of completions : text and chat. Let us illustrate the differences\nof their usage by Raku code generation. Here is a text completion:\n\n```perl6\nllama-text-completion(\n        'generate Raku code for making a loop over a list',\n        max-tokens =\u003e 120,\n        format =\u003e 'values');\n```\n\nHere is a chat completion:\n\n```perl6\nllama-completion(\n        'generate Raku code for making a loop over a list',\n        max-tokens =\u003e 120,\n        format =\u003e 'values');\n```\n\n\n### Embeddings\n\nEmbeddings can be obtained with the function `llama-embedding`. Here is an example of finding the embedding vectors\nfor each of the elements of an array of strings:\n\n```perl6\nmy @queries = [\n    'make a classifier with the method RandomForeset over the data dfTitanic',\n    'show precision and accuracy',\n    'plot True Positive Rate vs Positive Predictive Value',\n    'what is a good meat and potatoes recipe'\n];\n\nmy $embs = llama-embedding(@queries, format =\u003e 'values', method =\u003e 'tiny');\n$embs.elems;\n```\n\nHere we show:\n- That the result is an array of four vectors each with length 1536\n- The distributions of the values of each vector\n\n```perl6\nuse Data::Reshapers;\nuse Data::Summarizers;\n\nsay \"\\$embs.elems : { $embs.elems }\";\nsay \"\\$embs\u003e\u003e.elems : { $embs\u003e\u003e.elems }\";\nrecords-summary($embs.kv.Hash.\u0026transpose);\n```\n\nHere we find the corresponding dot products and (cross-)tabulate them:\n\n```perl6\nuse Data::Reshapers;\nuse Data::Summarizers;\nmy @ct = (^$embs.elems X ^$embs.elems).map({ %( i =\u003e $_[0], j =\u003e $_[1], dot =\u003e sum($embs[$_[0]] \u003e\u003e*\u003c\u003c $embs[$_[1]])) }).Array;\n\nsay to-pretty-table(cross-tabulate(@ct, 'i', 'j', 'dot'), field-names =\u003e (^$embs.elems)\u003e\u003e.Str);\n````\n\n**Remark:** Note that the fourth element (the cooking recipe request) is an outlier.\n(Judging by the table with dot products.)\n\n### Tokenizing and de-tokenizing\n\nHere we tokenize some text:\n\n```perl6\nmy $txt = @queries.head;\nmy $res = llama-tokenize($txt, format =\u003e 'values');\n```\n\nHere we get the original text be de-tokenizing:\n\n```perl6\nllama-detokenize($res);\n```\n\n### Chat completions with engineered prompts\n\nHere is a prompt for \"emojification\" (see the\n[Wolfram Prompt Repository](https://resources.wolframcloud.com/PromptRepository/)\nentry\n[\"Emojify\"](https://resources.wolframcloud.com/PromptRepository/resources/Emojify/)):\n\n```perl6\nmy $preEmojify = q:to/END/;\nRewrite the following text and convert some of it into emojis.\nThe emojis are all related to whatever is in the text.\nKeep a lot of the text, but convert key words into emojis.\nDo not modify the text except to add emoji.\nRespond only with the modified text, do not include any summary or explanation.\nDo not respond with only emoji, most of the text should remain as normal words.\nEND\n```\n\nHere is an example of chat completion with emojification:\n\n```perl6\nllama-chat-completion([ system =\u003e $preEmojify, user =\u003e 'Python sucks, Raku rocks, and Perl is annoying'], max-tokens =\u003e 200, format =\u003e 'values')\n```\n\n-------\n\n## Command Line Interface\n\n### Playground access\n\nThe package provides a Command Line Interface (CLI) script:\n\n```shell\nllama-playground --help\n```\n\n**Remark:** When the authorization key, `auth-key`, is specified to be `Whatever`\nthen it is assigned the string `sk-no-key-required`.\nIf an authorization key is required then the env variable `LLAMA_API_KEY` can be also used.\n\n--------\n\n## Mermaid diagram\n\nThe following flowchart corresponds to the steps in the package function `llama-playground`:\n\n```mermaid\ngraph TD\n\tUI[/Some natural language text/]\n\tTO[/\"LLaMA\u003cbr/\u003eProcessed output\"/]\n\tWR[[Web request]]\n\tLLaMA{{http://127.0.0.1:8080}}\n\tPJ[Parse JSON]\n\tQ{Return\u003cbr\u003ehash?}\n\tMSTC[Compose query]\n\tMURL[[Make URL]]\n\tTTC[Process]\n\tQAK{Auth key\u003cbr\u003esupplied?}\n\tEAK[[\"Try to find\u003cbr\u003eLLAMA_API_KEY\u003cbr\u003ein %*ENV\"]]\n\tQEAF{Auth key\u003cbr\u003efound?}\n\tNAK[[\"Use 'sk-no-key-required'\"]]\n\tUI --\u003e QAK\n\tQAK --\u003e |yes|MSTC\n\tQAK --\u003e |no|EAK\n\tEAK --\u003e QEAF\n\tMSTC --\u003e TTC\n\tQEAF --\u003e |no|NAK\n\tQEAF --\u003e |yes|TTC\n\tTTC -.-\u003e MURL -.-\u003e WR -.-\u003e TTC\n\tWR -.-\u003e |URL|LLaMA \n\tLLaMA -.-\u003e |JSON|WR\n\tTTC --\u003e Q \n\tQ --\u003e |yes|PJ\n\tQ --\u003e |no|TO\n\tPJ --\u003e TO\n```\n\n--------\n\n## References\n\n### Packages\n\n[AAp1] Anton Antonov,\n[WWW::OpenAI Raku package](https://github.com/antononcube/Raku-WWW-OpenAI),\n(2023-2024),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp2] Anton Antonov,\n[WWW::MistralAI Raku package](https://github.com/antononcube/Raku-WWW-MistralAI),\n(2023-2024),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp3] Anton Antonov,\n[LLM::Functions Raku package](https://github.com/antononcube/Raku-LLM-Functions),\n(2023-2024),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp4] Anton Antonov,\n[LLM::Prompts Raku package](https://github.com/antononcube/Raku-LLM-Prompts),\n(2023-2024),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[AAp5] Anton Antonov,\n[Jupyter::Chatbook Raku package](https://github.com/antononcube/Raku-Jupyter-Chatbook),\n(2023),\n[GitHub/antononcube](https://github.com/antononcube).\n\n[MO1] Mozilla Ocho, [llamafile](https://github.com/mozilla-Ocho/llamafile).\n\n[MO2] Mozilla Ocho, [llamafile documentation](https://github.com/Mozilla-Ocho/llamafile/blob/main/README.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-www-llama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantononcube%2Fraku-www-llama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantononcube%2Fraku-www-llama/lists"}