{"id":14964521,"url":"https://github.com/ariya/gamal","last_synced_at":"2025-09-15T03:32:22.860Z","repository":{"id":247298386,"uuid":"822645183","full_name":"ariya/gamal","owner":"ariya","description":"Research tool leveraging LLM for answers","archived":false,"fork":false,"pushed_at":"2024-11-15T02:10:19.000Z","size":206,"stargazers_count":30,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-08T06:46:52.357Z","etag":null,"topics":["copilot","deepinfra","gemma","groq","llama","llm","mistral","openai","perplexity","telegram-bot"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ariya.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-01T14:32:20.000Z","updated_at":"2024-12-30T16:17:00.000Z","dependencies_parsed_at":"2024-09-07T20:12:36.696Z","dependency_job_id":"9b0c923b-8c3e-4c5c-a070-93572ff2451a","html_url":"https://github.com/ariya/gamal","commit_stats":{"total_commits":231,"total_committers":1,"mean_commits":231.0,"dds":0.0,"last_synced_commit":"473a65b2d4d9136b593e1fc1a3035ba7de1dbaeb"},"previous_names":["ariya/gamal"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ariya%2Fgamal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ariya%2Fgamal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ariya%2Fgamal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ariya%2Fgamal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ariya","download_url":"https://codeload.github.com/ariya/gamal/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233073852,"owners_count":18621028,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["copilot","deepinfra","gemma","groq","llama","llm","mistral","openai","perplexity","telegram-bot"],"created_at":"2024-09-24T13:33:18.343Z","updated_at":"2025-01-08T17:49:26.970Z","avatar_url":"https://github.com/ariya.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gamal\n\nGamal is a simple, zero-dependency tool designed to quickly provide answers to questions. It finds relevant web pages and uses an LLM to summarize the content, delivering concise answers.  Gamal is accessible via the terminal (as a CLI tool), through its minimalist web interface, or as a Telegram bot.\n\n[![asciicast](https://asciinema.org/a/668554.svg)](https://asciinema.org/a/668554)\n\nGamal utilizes [SearXNG](https://searxng.org) for web searches and requires an LLM to generate responses based on search results. By default, Gamal integrates with [OpenRouter](https://openrouter.ai) as its LLM service, requiring the configuration of an API key in the `LLM_API_KEY` environment variable. Please continue reading for detailed instructions on configuring Gamal to use either a local LLM ([llama.cpp](https://github.com/ggerganov/llama.cpp), [Jan](https://jan.ai), and [Ollama](https://ollama.com)) or other managed LLM services (offering over half a dozen options, including [OpenAI](https://platform.openai.com), [Fireworks](https://fireworks.ai), and [Groq](https://groq.com)).\n\nTo execute Gamal as a CLI tool, run it with [Node.js](https://nodejs.org) (version \u003e= 18) or [Bun](https://bun.sh):\n\n```bash\n./gamal.js\n```\n\nFor instant answers, pipe the questions directly into Gamal:\n\n```bash\necho \"List 5 Indonesia's best travel destinations\" | ./gamal.js\n```\n\nGamal also includes a minimalist front-end web interface. To launch it, specify the environment variable `GAMAL_HTTP_PORT`, for example:\n\n```bash\nGAMAL_HTTP_PORT=5000 ./gamal.js\n```\n\nThen, open a web browser and go to `localhost:5000`.\n\nGamal is capable of functioning as a [Telegram bot](https://core.telegram.org/bots). Obtain a token (refer to [Telegram documentation](https://core.telegram.org/bots/tutorial#obtain-your-bot-token) for details) and set it as the environment variable `GAMAL_TELEGRAM_TOKEN` before launching Gamal. Note that conversation history in Telegram chats is stored in memory and not persisted to disk.\n\n## Multi-language Support\n\nGamal can converse in many languages besides English. It always tries to respond in the same language as the question. You can freely switch languages between questions, as shown in the following example:\n\n```\n\u003e\u003e Which planet in our solar system is the biggest?\nJupiter is the largest planet in our solar system [1].\n[1] https://science.nasa.gov/jupiter/\n\n\u003e\u003e ¿Y el más caliente?\nVenus es el planeta más caliente, con hasta 475°C. [1].\n[1] https://www.redastronomy.com/sistema-solar/el-planeta-venus/\n```\n\nGamal's continuous integration workflows include evaluation tests in English, Spanish, German, French, Italian, and Indonesian.\n\n[![English tests](https://github.com/ariya/gamal/actions/workflows/english.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/english.yml)\n[![Spanish tests](https://github.com/ariya/gamal/actions/workflows/spanish.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/spanish.yml)\n[![French tests](https://github.com/ariya/gamal/actions/workflows/french.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/french.yml)\n[![German tests](https://github.com/ariya/gamal/actions/workflows/german.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/german.yml)\n[![Italian tests](https://github.com/ariya/gamal/actions/workflows/italian.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/italian.yml)\n[![Indonesian tests](https://github.com/ariya/gamal/actions/workflows/indonesian.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/indonesian.yml)\n[![Language switch](https://github.com/ariya/gamal/actions/workflows/lang-switch.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/lang-switch.yml)\n\n## Conversational Interface\n\nWith the integration of third-party tools, Gamal can engage in conversations using voice (both input and output) rather than just text.\n\nFor automatic speech recognition (ASR), also known as speech-to-text (STT), Gamal leverages the streaming tool from [whisper.cpp](https://github.com/ggerganov/whisper.cpp). Ensure that `whisper-cpp-stream`, or the custom executable specified in the `WHISPER_STREAM` environment variable, is available in your system's path. Whisper requires a GGML model, which can be downloaded from [Hugging Face](https://huggingface.co/ggerganov/whisper.cpp). The [base model](https://huggingface.co/ggerganov/whisper.cpp/blob/main/ggml-base.en-q5_1.bin) (60 MB) is generally a good balance between accuracy and speed for most modern computers. Set the `WHISPER_MODEL` environment variable to the full path of the downloaded model.\n\nTo enable Gamal to respond with voice instead of just text, install [Piper](https://github.com/rhasspy/piper) for text-to-speech (TTS) conversion. Piper can be installed via Nixpkg (the `piper-tts` package). Piper also requires a [voice model](https://huggingface.co/rhasspy/piper-voices), which can be downloaded from sources like [ryan-medium](https://huggingface.co/rhasspy/piper-voices/tree/main/en/en_US/ryan/medium). Make sure to download both the ONNX model file (63 MB) and the corresponding config JSON. Before running Gamal, set the `PIPER_MODEL` environment variable to the full path of the voice model.\n\nThe synthesized audio will be played back through the speaker or other audio output using the `play` utility from the [SOX (Sound eXchange project)](https://sourceforge.net/projects/sox/). Ensure that SOX is installed and available in your system's path.\n\n## Using Other LLM Services\n\nGamal is designed to be used with OpenRouter by default, but it can also be configured to work with other LLM services by adjusting some environment variables. The correct API key and a suitable model are required.\n\nCompatible LLM services include [Deep Infra](https://deepinfra.com), [Fireworks](https://fireworks.ai), [Gemini](https://ai.google.dev/gemini-api), [Groq](https://groq.com), [Hyperbolic](https://www.hyperbolic.xyz), [Lepton](https://lepton.ai), [Novita](https://novita.ai), [OpenAI](https://platform.openai.com), and [Together](https://www.together.ai).\n\n[![Test on DeepInfra](https://github.com/ariya/gamal/actions/workflows/test-deepinfra.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-deepinfra.yml)\n[![Test on Fireworks](https://github.com/ariya/gamal/actions/workflows/test-fireworks.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-fireworks.yml)\n[![Test on Gemini](https://github.com/ariya/gamal/actions/workflows/test-gemini.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-gemini.yml)\n[![Test on Groq](https://github.com/ariya/gamal/actions/workflows/test-groq.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-groq.yml)\n[![Test on Hyperbolic](https://github.com/ariya/gamal/actions/workflows/test-hyperbolic.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-hyperbolic.yml)\n[![Test on Lepton](https://github.com/ariya/gamal/actions/workflows/test-lepton.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-lepton.yml)\n[![Test on Novita](https://github.com/ariya/gamal/actions/workflows/test-novita.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-novita.yml)\n[![Test on OpenAI](https://github.com/ariya/gamal/actions/workflows/test-openai.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-openai.yml)\n[![Test on Together](https://github.com/ariya/gamal/actions/workflows/test-together.yml/badge.svg)](https://github.com/ariya/gamal/actions/workflows/test-together.yml)\n\nRefer to the relevant section for configuration details. The example provided is for Llama-3.1 8B, though any LLM with 7B parameters should also work, such as Mistral 7B, Qwen-2 7B, or Gemma-2 9B.\n\n\u003cdetails\u003e\u003csummary\u003eDeep Infra\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.deepinfra.com/v1/openai\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"meta-llama/Meta-Llama-3.1-8B-Instruct\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eFireworks\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.fireworks.ai/inference/v1\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"accounts/fireworks/models/llama-v3p1-8b-instruct\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eGoogle Gemini\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://generativelanguage.googleapis.com/v1beta\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"gemini-1.5-flash-8b\"\nexport LLM_JSON_SCHEMA=1\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eGroq\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.groq.com/openai/v1\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"llama-3.1-8b-instant\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eHyperbolic\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.hyperbolic.xyz/v1\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"meta-llama/Meta-Llama-3.1-8B-Instruct\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eLepton\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://llama3-1-8b.lepton.run/api/v1\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"llama3-1-8b\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eNovita\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.novita.ai/v3/openai\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"meta-llama/llama-3.1-8b-instruct\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eOpenAI\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.openai.com/v1\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"gpt-4o-mini\"\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eTogether\u003c/summary\u003e\n\n```bash\nexport LLM_API_BASE_URL=https://api.together.xyz/v1\nexport LLM_API_KEY=\"yourownapikey\"\nexport LLM_CHAT_MODEL=\"meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo\"\n```\n\u003c/details\u003e\n\n\n## Using Local LLM Servers\n\nGamal is compatible with local LLM inference tools such as [llama.cpp](https://github.com/ggerganov/llama.cpp), [Jan](https://jan.ai), and [Ollama](https://ollama.com). Refer to the relevant section for configuration details.\n\nThe example provided uses Llama-3.1 8B. For optimal performance, an instruction-following LLM with 7B parameters or more is recommended. Suitable models include Mistral 7B, Qwen-2 7B, and Gemma-2 9B.\n\n\u003cdetails\u003e\u003csummary\u003ellama.cpp\u003c/summary\u003e\n\nFirst, load a quantized model such as [Llama-3.1 8B](https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF). Then, adjust the `LLM_API_BASE_URL` environment variable accordingly:\n\n```bash\n/path/to/llama-server -m Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf\nexport LLM_API_BASE_URL=http://127.0.0.1:8080/v1\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eJan\u003c/summary\u003e\n\nRefer to [the documentation](https://jan.ai/docs/local-api) and load a model like [Llama-3.1 8B](https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF). Then, set the environment variable:\n\n```bash\nexport LLM_API_BASE_URL=http://127.0.0.1:1337/v1\nexport LLM_CHAT_MODEL='llama3.1'\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eOllama\u003c/summary\u003e\n\nLoad a model and configure the environment variables:\n\n```bash\nollama pull llama3.1\nexport LLM_API_BASE_URL=http://127.0.0.1:11434/v1\nexport LLM_CHAT_MODEL='llama3.1'\n```\n\u003c/details\u003e\n\n\n## Evaluating Questions\n\nGamal includes a built-in evaluation tool. For instance, if a text file named `qa.txt` contains pairs of `User` and `Assistant` messages:\n\n```\nUser: Which planet is the largest?\nAssistant: The largest planet is /Jupiter/.\n\nUser: and the smallest?\nAssistant: The smallest planet is /Mercury/.\n```\n\nexecuting the following command will sequentially search for these questions and verify the answers using regular expressions:\n```bash\n./gamal.js qa.txt\n```\n\nAdditional examples can be found in the `tests/` subdirectory.\n\nTwo environment variables can modify the behavior:\n\n* `LLM_DEBUG_FAIL_EXIT`: When set, Gamal will exit immediately upon encountering an incorrect answer, and subsequent questions in the file will not be processed.\n\n* `LLM_DEBUG_PIPELINE`: When set, if the expected regular expression does not match the answer, the internal LLM pipeline will be printed to stdout.\n\n\n## Improving Search Quality\n\nBy default, Gamal uses the [public SearXNG instance](https://searx.space/). However, public instances typically apply aggressive rate limiting. To avoid this, it is recommended to run a local SearXNG instance, such as via a Docker container, following the steps in [its documentation](https://docs.searxng.org/admin/installation-docker.html). Ensure that the local instance is configured to support JSON format by including the following block in the `settings.yml` file:\n\n```yaml\nsearch:\n  formats:\n    - html\n    - json\n```\n\nBefore starting Gamal, the `SEARXNG_URL` environment variable should be set to the URL of the local SearXNG instance (e.g., `localhost:8080` if running locally).\n\nAdditionally, connecting Gamal to a private SearXNG instance allows integration with [custom data sources](https://docs.searxng.org/dev/engines/offline/search-indexer-engines.html), enabling enhanced search capabilities.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fariya%2Fgamal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fariya%2Fgamal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fariya%2Fgamal/lists"}