{"id":22338043,"url":"https://github.com/shettysach/candlemist","last_synced_at":"2025-10-25T23:37:15.971Z","repository":{"id":243986293,"uuid":"813975330","full_name":"ShettySach/CandleMist","owner":"ShettySach","description":"Fullstack chatbot built using Rust. Made using Candle, Leptos, Actix, Tokio and Tailwind. Uses quantized Mistral 7B Instruct v0.1 GGUF models.","archived":false,"fork":false,"pushed_at":"2024-08-08T13:51:47.000Z","size":544,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-08T16:16:27.605Z","etag":null,"topics":["actix","backend","candle","chatbot","dolphin","fullstack","huggingface","huggingface-candle","leptos","mistral-7b","mistral-7b-instruct","tailwind","tailwindcss","tokyonight"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ShettySach.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-12T05:27:19.000Z","updated_at":"2024-08-08T13:51:50.000Z","dependencies_parsed_at":"2024-06-29T12:22:40.118Z","dependency_job_id":"80224f33-5db9-4434-bb49-48266ba528fd","html_url":"https://github.com/ShettySach/CandleMist","commit_stats":null,"previous_names":["shettysach/candlemist"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShettySach%2FCandleMist","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShettySach%2FCandleMist/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShettySach%2FCandleMist/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShettySach%2FCandleMist/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ShettySach","download_url":"https://codeload.github.com/ShettySach/CandleMist/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228054438,"owners_count":17862129,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actix","backend","candle","chatbot","dolphin","fullstack","huggingface","huggingface-candle","leptos","mistral-7b","mistral-7b-instruct","tailwind","tailwindcss","tokyonight"],"created_at":"2024-12-04T06:12:53.837Z","updated_at":"2025-10-25T23:37:15.636Z","avatar_url":"https://github.com/ShettySach.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"## CandleMist\n\n\u003cdiv \"align: center;\"\u003e\n    \u003cimg src=\"assets/image.png\" width=\"45%\"\u003e\n    \u0026nbsp; \u0026nbsp;\n    \u003cimg src=\"assets/image1.png\" width=\"45%\"\u003e\n\u003c/div\u003e\n\n- A simple chatbot built using Rust, in the frontend and the backend.\n- Made using `candle`, `leptos`, `actix`, `tokio` and TailwindCSS.\n- Uses quantized Mistral 7B Instruct v0.1 GGUF models.\n\n  #### Credits\n\n  - This is a fork of [MoonKraken/rusty_llama](https://github.com/MoonKraken/rusty_llama) by [Code to the Moon](https://www.youtube.com/watch?v=vAjle3c9Xqc).\n  - This chatbot uses Mistral GGUF models and the [`huggingface/candle`](https://github.com/huggingface/candle) framework, which includes the `candle-transformers` crate, whereas the original uses GGML models and the [`rustformers/llm`](https://github.com/rustformers/llm) crate.\n  - The frontend has some aesthetic changes, but the overall structure is the same.\n  - Colours are from the Tokyo Night colorscheme.\n\n## Setup Instructions\n\n### Rust Toolchain\n\nYou'll need to use the nightly Rust toolchain, and install the `wasm32-unknown-unknown` target as well as the `trunk` and `cargo-leptos` tools:\n\n```\nrustup toolchain install nightly\nrustup target add wasm32-unknown-unknown\ncargo install trunk cargo-leptos\n```\n\n### Hardware\n\n- For CUDA, add the `cuda` feature for candle_core in Cargo.toml.\n```\ncandle-core = { git = \"https://github.com/huggingface/candle.git\", version = \"0.6.0\", optional = true, features = [\"cuda\"] }\n```\n- For Metal, add the `metal` feature for candle_core in Cargo.toml.\n- For Intel's oneAPI Math Kernel Library, add the `mkl` feature for candle_core in Cargo.toml.\n- For Accelerate, add the `accelerate` feature for candle_core in Cargo.toml.\n\n### Model\n\n- Download any Mistral 7B Instruct v0.1 GGUF model and set the environment variable `MODEL_PATH` in `.env`.\n\n  #### Tested Models\n\n  - [mistral-7b-instruct-v0.1.Q4_K_M.gguf](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/tree/main)\n  - [dolphin-2.6-mistral-7b.Q4_K_M.gguf](https://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-GGUF/tree/main)\n\n- Download tokenizer.json and set the environment variable in `TOKENIZER_PATH` in `.env`.\n\n  #### Tokenizer\n\n  - [Mistral-7B-v0.1/tokenizer.json](https://huggingface.co/mistralai/Mistral-7B-v0.1/blob/main/tokenizer.json)\n\n### TailwindCSS\n\n- Install TailwindCSS with `npm install -D tailwindcss`.\n\n### Run\n\n1.\n\n```\ngit clone https://github.com/ShettySach/CandleMist.git\ncd CandleMist\n```\n\n2.\n\n```\nnpx tailwindcss -i ./input.css -o ./style/output.css\n```\n\n3.\n\n```\ncargo leptos serve --release\n```\n\n4. In your browser, navigate to [http://localhost:3000/?](http://localhost:3000/?)\n\n##### NOTE -\n\n- You can modify parameters such as temperature, seed, top-k, top-p, max history and max response length, and also modify the chat template in 'src/api.rs'.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshettysach%2Fcandlemist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshettysach%2Fcandlemist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshettysach%2Fcandlemist/lists"}