{"id":18903632,"url":"https://github.com/mobinx/greetup","last_synced_at":"2026-06-30T16:32:08.164Z","repository":{"id":113707480,"uuid":"518009193","full_name":"MobinX/greetup","owner":"MobinX","description":"A video conference app ","archived":false,"fork":false,"pushed_at":"2024-07-06T01:09:11.000Z","size":8811,"stargazers_count":0,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-14T20:34:52.411Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MobinX.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-26T10:09:46.000Z","updated_at":"2024-02-01T13:49:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"bddf073d-f35c-4a9b-a390-0374b77bf872","html_url":"https://github.com/MobinX/greetup","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MobinX/greetup","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobinX%2Fgreetup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobinX%2Fgreetup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobinX%2Fgreetup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobinX%2Fgreetup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MobinX","download_url":"https://codeload.github.com/MobinX/greetup/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobinX%2Fgreetup/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34975669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-30T02:00:05.919Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T09:05:59.840Z","updated_at":"2026-06-30T16:32:08.127Z","avatar_url":"https://github.com/MobinX.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# candle\n[![discord server](https://dcbadge.vercel.app/api/server/hugging-face-879548962464493619)](https://discord.gg/hugging-face-879548962464493619)\n[![Latest version](https://img.shields.io/crates/v/candle-core.svg)](https://crates.io/crates/candle-core)\n[![Documentation](https://docs.rs/candle-core/badge.svg)](https://docs.rs/candle-core)\n![License](https://img.shields.io/crates/l/candle-core.svg)\n\nCandle is a minimalist ML framework for Rust with a focus on performance (including GPU support) \nand ease of use. Try our online demos: \n[whisper](https://huggingface.co/spaces/lmz/candle-whisper),\n[LLaMA2](https://huggingface.co/spaces/lmz/candle-llama2),\n[T5](https://huggingface.co/spaces/radames/Candle-T5-Generation-Wasm),\n[yolo](https://huggingface.co/spaces/lmz/candle-yolo),\n[Segment\nAnything](https://huggingface.co/spaces/radames/candle-segment-anything-wasm).\n\n## Get started\n\nMake sure that you have [`candle-core`](https://github.com/huggingface/candle/tree/main/candle-core) correctly installed as described in [**Installation**](https://huggingface.github.io/candle/guide/installation.html).\n\nLet's see how to run a simple matrix multiplication.\nWrite the following to your `myapp/src/main.rs` file:\n```rust\nuse candle_core::{Device, Tensor};\n\nfn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let device = Device::Cpu;\n\n    let a = Tensor::randn(0f32, 1., (2, 3), \u0026device)?;\n    let b = Tensor::randn(0f32, 1., (3, 4), \u0026device)?;\n\n    let c = a.matmul(\u0026b)?;\n    println!(\"{c}\");\n    Ok(())\n}\n```\n\n`cargo run` should display a tensor of shape `Tensor[[2, 4], f32]`.\n\n\nHaving installed `candle` with Cuda support, simply define the `device` to be on GPU:\n\n```diff\n- let device = Device::Cpu;\n+ let device = Device::new_cuda(0)?;\n```\n\nFor more advanced examples, please have a look at the following section.\n\n## Check out our examples\n\nThese online demos run entirely in your browser:\n- [yolo](https://huggingface.co/spaces/lmz/candle-yolo): pose estimation and\n  object recognition.\n- [whisper](https://huggingface.co/spaces/lmz/candle-whisper): speech recognition.\n- [LLaMA2](https://huggingface.co/spaces/lmz/candle-llama2): text generation.\n- [T5](https://huggingface.co/spaces/radames/Candle-T5-Generation-Wasm): text generation.\n- [Phi-1.5, and Phi-2](https://huggingface.co/spaces/radames/Candle-Phi-1.5-Wasm): text generation.\n- [Segment Anything Model](https://huggingface.co/spaces/radames/candle-segment-anything-wasm): Image segmentation.\n- [BLIP](https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning): image captioning.\n\nWe also provide a some command line based examples using state of the art models:\n\n- [LLaMA and LLaMA-v2](./candle-examples/examples/llama/): general LLM, includes\n  the SOLAR-10.7B variant.\n- [Falcon](./candle-examples/examples/falcon/): general LLM.\n- [Phi-1, Phi-1.5, and Phi-2](./candle-examples/examples/phi/): 1.3b and 2.7b general LLMs with performance on par with LLaMA-v2 7b.\n- [StableLM-3B-4E1T](./candle-examples/examples/stable-lm/): a 3b general LLM\n  pre-trained on 1T tokens of English and code datasets.\n- [Minimal Mamba](./candle-examples/examples/mamba-minimal/): a minimal\n  implementation of the Mamba state space model.\n- [Mistral7b-v0.1](./candle-examples/examples/mistral/): a 7b general LLM with\n  better performance than all publicly available 13b models as of 2023-09-28.\n- [Mixtral8x7b-v0.1](./candle-examples/examples/mixtral/): a sparse mixture of\n  experts 8x7b general LLM with better performance than a Llama 2 70B model with\n  much faster inference.\n- [StarCoder](./candle-examples/examples/bigcode/): LLM specialized to code generation.\n- [Replit-code-v1.5](./candle-examples/examples/replit-code/): a 3.3b LLM specialized for code completion.\n- [Yi-6B / Yi-34B](./candle-examples/examples/yi/): two bilingual\n  (English/Chinese) general LLMs with 6b and 34b parameters.\n- [Quantized LLaMA](./candle-examples/examples/quantized/): quantized version of\n  the LLaMA model using the same quantization techniques as\n  [llama.cpp](https://github.com/ggerganov/llama.cpp).\n\n\u003cimg src=\"https://github.com/huggingface/candle/raw/main/candle-examples/examples/quantized/assets/aoc.gif\" width=\"600\"\u003e\n  \n- [Stable Diffusion](./candle-examples/examples/stable-diffusion/): text to\n  image generative model, support for the 1.5, 2.1, SDXL 1.0 and Turbo versions.\n\n\u003cimg src=\"https://github.com/huggingface/candle/raw/main/candle-examples/examples/stable-diffusion/assets/stable-diffusion-xl.jpg\" width=\"200\"\u003e\n\n- [Wuerstchen](./candle-examples/examples/wuerstchen/): another text to\n  image generative model.\n\n\u003cimg src=\"https://github.com/huggingface/candle/raw/main/candle-examples/examples/wuerstchen/assets/cat.jpg\" width=\"200\"\u003e\n\n- [yolo-v3](./candle-examples/examples/yolo-v3/) and\n  [yolo-v8](./candle-examples/examples/yolo-v8/): object detection and pose\n  estimation models.\n\n\u003cimg src=\"https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.od.jpg\" width=\"200\"\u003e\u003cimg src=\"https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.pose.jpg\" width=\"200\"\u003e\n- [segment-anything](./candle-examples/examples/segment-anything/): image\n  segmentation model with prompt.\n\n\u003cimg src=\"https://github.com/huggingface/candle/raw/main/candle-examples/examples/segment-anything/assets/sam_merged.jpg\" width=\"200\"\u003e\n\n- [Whisper](./candle-examples/examples/whisper/): speech recognition model.\n- [T5](./candle-examples/examples/t5), [Bert](./candle-examples/examples/bert/),\n  [JinaBert](./candle-examples/examples/jina-bert/) : useful for sentence embeddings.\n- [DINOv2](./candle-examples/examples/dinov2/): computer vision model trained\n  using self-supervision (can be used for imagenet classification, depth\n  evaluation, segmentation).\n- [VGG](./candle-examples/examples/vgg/),\n  [RepVGG](./candle-examples/examples/repvgg): computer vision models.\n- [BLIP](./candle-examples/examples/blip/): image to text model, can be used to\n- [BLIP](./candle-examples/examples/blip/): image to text model, can be used to\n  generate captions for an image.\n- [Marian-MT](./candle-examples/examples/marian-mt/): neural machine translation\n  model, generates the translated text from the input text.\n\nRun them using commands like:\n```\ncargo run --example quantized --release\n```\n\nIn order to use **CUDA** add `--features cuda` to the example command line. If\nyou have cuDNN installed, use `--features cudnn` for even more speedups.\n\nThere are also some wasm examples for whisper and\n[llama2.c](https://github.com/karpathy/llama2.c). You can either build them with\n`trunk` or try them online:\n[whisper](https://huggingface.co/spaces/lmz/candle-whisper),\n[llama2](https://huggingface.co/spaces/lmz/candle-llama2),\n[T5](https://huggingface.co/spaces/radames/Candle-T5-Generation-Wasm),\n[Phi-1.5, and Phi-2](https://huggingface.co/spaces/radames/Candle-Phi-1.5-Wasm),\n[Segment Anything Model](https://huggingface.co/spaces/radames/candle-segment-anything-wasm).\n\nFor LLaMA2, run the following command to retrieve the weight files and start a\ntest server:\n```bash\ncd candle-wasm-examples/llama2-c\nwget https://huggingface.co/spaces/lmz/candle-llama2/resolve/main/model.bin\nwget https://huggingface.co/spaces/lmz/candle-llama2/resolve/main/tokenizer.json\ntrunk serve --release --port 8081\n```\nAnd then head over to\n[http://localhost:8081/](http://localhost:8081/).\n\n\u003c!--- ANCHOR: useful_libraries ---\u003e\n\n## Useful External Resources\n- [`candle-tutorial`](https://github.com/ToluClassics/candle-tutorial): A\n  very detailed tutorial showing how to convert a PyTorch model to Candle.\n- [`candle-lora`](https://github.com/EricLBuehler/candle-lora): Efficient and\n  ergonomic LoRA implementation for Candle. `candle-lora` has      \n  out-of-the-box LoRA support for many models from Candle, which can be found\n  [here](https://github.com/EricLBuehler/candle-lora/tree/master/candle-lora-transformers/examples).\n- [`optimisers`](https://github.com/KGrewal1/optimisers): A collection of optimisers\n  including SGD with momentum, AdaGrad, AdaDelta, AdaMax, NAdam, RAdam, and RMSprop.\n- [`candle-vllm`](https://github.com/EricLBuehler/candle-vllm): Efficient platform for inference and\n  serving local LLMs including an OpenAI compatible API server.\n- [`candle-ext`](https://github.com/mokeyish/candle-ext): An extension library to Candle that provides PyTorch functions not currently available in Candle.\n- [`kalosm`](https://github.com/floneum/floneum/tree/master/interfaces/kalosm): A multi-modal meta-framework in Rust for interfacing with local pre-trained models with support for controlled generation, custom samplers, in-memory vector databases, audio transcription, and more.\n- [`candle-sampling`](https://github.com/EricLBuehler/candle-sampling): Sampling techniques for Candle.\n- [`gpt-from-scratch-rs`](https://github.com/jeroenvlek/gpt-from-scratch-rs): A port of Andrej Karpathy's _Let's build GPT_ tutorial on YouTube showcasing the Candle API on a toy problem.\n\nIf you have an addition to this list, please submit a pull request.\n\n\u003c!--- ANCHOR_END: useful_libraries ---\u003e\n\n\u003c!--- ANCHOR: features ---\u003e\n\n## Features\n\n- Simple syntax, looks and feels like PyTorch.\n    - Model training.\n    - Embed user-defined ops/kernels, such as [flash-attention v2](https://github.com/huggingface/candle/blob/89ba005962495f2bfbda286e185e9c3c7f5300a3/candle-flash-attn/src/lib.rs#L152).\n- Backends.\n    - Optimized CPU backend with optional MKL support for x86 and Accelerate for macs.\n    - CUDA backend for efficiently running on GPUs, multiple GPU distribution via NCCL.\n    - WASM support, run your models in a browser.\n- Included models.\n    - Language Models.\n        - LLaMA v1 and v2 with variants such as SOLAR-10.7B.\n        - Falcon.\n        - StarCoder.\n        - Phi 1, 1.5, and 2.\n        - Minimal Mamba\n        - Mistral 7b v0.1.\n        - Mixtral 8x7b v0.1.\n        - StableLM-3B-4E1T.\n        - Replit-code-v1.5-3B.\n        - Bert.\n        - Yi-6B and Yi-34B.\n    - Quantized LLMs.\n        - Llama 7b, 13b, 70b, as well as the chat and code variants.\n        - Mistral 7b, and 7b instruct.\n        - Mixtral 8x7b.\n        - Zephyr 7b a and b (Mistral-7b based).\n        - OpenChat 3.5 (Mistral-7b based).\n    - Text to text.\n        - T5 and its variants: FlanT5, UL2, MADLAD400 (translation), CoEdit (Grammar correction).\n        - Marian MT (Machine Translation).\n    - Whisper (multi-lingual support).\n    - Text to image.\n        - Stable Diffusion v1.5, v2.1, XL v1.0.\n        - Wurstchen v2.\n    - Image to text.\n        - BLIP.\n    - Computer Vision Models.\n        - DINOv2, ConvMixer, EfficientNet, ResNet, ViT, VGG, RepVGG.\n        - yolo-v3, yolo-v8.\n        - Segment-Anything Model (SAM).\n- File formats: load models from safetensors, npz, ggml, or PyTorch files.\n- Serverless (on CPU), small and fast deployments.\n- Quantization support using the llama.cpp quantized types.\n\n\u003c!--- ANCHOR_END: features ---\u003e\n\n## How to use\n\n\u003c!--- ANCHOR: cheatsheet ---\u003e\nCheatsheet:\n\n|            | Using PyTorch                            | Using Candle                                                     |\n|------------|------------------------------------------|------------------------------------------------------------------|\n| Creation   | `torch.Tensor([[1, 2], [3, 4]])`         | `Tensor::new(\u0026[[1f32, 2.], [3., 4.]], \u0026Device::Cpu)?`           |\n| Creation   | `torch.zeros((2, 2))`                    | `Tensor::zeros((2, 2), DType::F32, \u0026Device::Cpu)?`               |\n| Indexing   | `tensor[:, :4]`                          | `tensor.i((.., ..4))?`                                           |\n| Operations | `tensor.view((2, 2))`                    | `tensor.reshape((2, 2))?`                                        |\n| Operations | `a.matmul(b)`                            | `a.matmul(\u0026b)?`                                                  |\n| Arithmetic | `a + b`                                  | `\u0026a + \u0026b`                                                        |\n| Device     | `tensor.to(device=\"cuda\")`               | `tensor.to_device(\u0026Device::new_cuda(0)?)?`                            |\n| Dtype      | `tensor.to(dtype=torch.float16)`         | `tensor.to_dtype(\u0026DType::F16)?`                                  |\n| Saving     | `torch.save({\"A\": A}, \"model.bin\")`      | `candle::safetensors::save(\u0026HashMap::from([(\"A\", A)]), \"model.safetensors\")?` |\n| Loading    | `weights = torch.load(\"model.bin\")`      | `candle::safetensors::load(\"model.safetensors\", \u0026device)`        |\n\n\u003c!--- ANCHOR_END: cheatsheet ---\u003e\n\n\n## Structure\n\n- [candle-core](./candle-core): Core ops, devices, and `Tensor` struct definition\n- [candle-nn](./candle-nn/): Tools to build real models\n- [candle-examples](./candle-examples/): Examples of using the library in realistic settings\n- [candle-kernels](./candle-kernels/): CUDA custom kernels\n- [candle-datasets](./candle-datasets/): Datasets and data loaders.\n- [candle-transformers](./candle-transformers): transformers-related utilities.\n- [candle-flash-attn](./candle-flash-attn): Flash attention v2 layer.\n- [candle-onnx](./candle-onnx/): ONNX model evaluation.\n\n## FAQ\n\n### Why should I use Candle?\n\nCandle's core goal is to *make serverless inference possible*. Full machine learning frameworks like PyTorch\nare very large, which makes creating instances on a cluster slow. Candle allows deployment of lightweight\nbinaries.\n\nSecondly, Candle lets you *remove Python* from production workloads. Python overhead can seriously hurt performance,\nand the [GIL](https://www.backblaze.com/blog/the-python-gil-past-present-and-future/) is a notorious source of headaches.\n\nFinally, Rust is cool! A lot of the HF ecosystem already has Rust crates, like [safetensors](https://github.com/huggingface/safetensors) and [tokenizers](https://github.com/huggingface/tokenizers).\n\n\n### Other ML frameworks\n\n- [dfdx](https://github.com/coreylowman/dfdx) is a formidable crate, with shapes being included\n  in types. This prevents a lot of headaches by getting the compiler to complain about shape mismatches right off the bat.\n  However, we found that some features still require nightly, and writing code can be a bit daunting for non rust experts.\n\n  We're leveraging and contributing to other core crates for the runtime so hopefully both crates can benefit from each\n  other.\n\n- [burn](https://github.com/burn-rs/burn) is a general crate that can leverage multiple backends so you can choose the best\n  engine for your workload.\n\n- [tch-rs](https://github.com/LaurentMazare/tch-rs.git) Bindings to the torch library in Rust. Extremely versatile, but they \n  bring in the entire torch library into the runtime. The main contributor of `tch-rs` is also involved in the development\n  of `candle`.\n\n### Common Errors\n\n#### Missing symbols when compiling with the mkl feature.\n\nIf you get some missing symbols when compiling binaries/tests using the mkl\nor accelerate features, e.g. for mkl you get:\n```\n  = note: /usr/bin/ld: (....o): in function `blas::sgemm':\n          .../blas-0.22.0/src/lib.rs:1944: undefined reference to `sgemm_' collect2: error: ld returned 1 exit status\n\n  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified\n  = note: use the `-l` flag to specify native libraries to link\n  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo\n```\nor for accelerate:\n```\nUndefined symbols for architecture arm64:\n            \"_dgemm_\", referenced from:\n                candle_core::accelerate::dgemm::h1b71a038552bcabe in libcandle_core...\n            \"_sgemm_\", referenced from:\n                candle_core::accelerate::sgemm::h2cf21c592cba3c47 in libcandle_core...\n          ld: symbol(s) not found for architecture arm64\n```\n\nThis is likely due to a missing linker flag that was needed to enable the mkl library. You\ncan try adding the following for mkl at the top of your binary:\n```rust\nextern crate intel_mkl_src;\n```\nor for accelerate:\n```rust\nextern crate accelerate_src;\n```\n\n#### Cannot run the LLaMA examples: access to source requires login credentials\n\n```\nError: request error: https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/tokenizer.json: status code 401\n```\n\nThis is likely because you're not permissioned for the LLaMA-v2 model. To fix\nthis, you have to register on the huggingface-hub, accept the [LLaMA-v2 model\nconditions](https://huggingface.co/meta-llama/Llama-2-7b-hf), and set up your\nauthentication token. See issue\n[#350](https://github.com/huggingface/candle/issues/350) for more details.\n\n#### Missing cute/cutlass headers when compiling flash-attn\n\n```\n  In file included from kernels/flash_fwd_launch_template.h:11:0,\n                   from kernels/flash_fwd_hdim224_fp16_sm80.cu:5:\n  kernels/flash_fwd_kernel.h:8:10: fatal error: cute/algorithm/copy.hpp: No such file or directory\n   #include \u003ccute/algorithm/copy.hpp\u003e\n            ^~~~~~~~~~~~~~~~~~~~~~~~~\n  compilation terminated.\n  Error: nvcc error while compiling:\n```\n[cutlass](https://github.com/NVIDIA/cutlass) is provided as a git submodule so you may want to run the following command to check it in properly.\n```bash\ngit submodule update --init\n```\n\n#### Compiling with flash-attention fails\n\n```\n/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:\n```\n\nThis is a bug in gcc-11 triggered by the Cuda compiler. To fix this, install a different, supported gcc version - for example gcc-10, and specify the path to the compiler in the CANDLE_NVCC_CCBIN environment variable.\n```\nenv CANDLE_NVCC_CCBIN=/usr/lib/gcc/x86_64-linux-gnu/10 cargo ...\n```\n\n#### Linking error on windows when running rustdoc or mdbook tests\n\n```\nCouldn't compile the test.\n---- .\\candle-book\\src\\inference\\hub.md - Using_the_hub::Using_in_a_real_model_ (line 50) stdout ----\nerror: linking with `link.exe` failed: exit code: 1181\n//very long chain of linking\n = note: LINK : fatal error LNK1181: cannot open input file 'windows.0.48.5.lib'\n```\n\nMake sure you link all native libraries that might be located outside a project target, e.g., to run mdbook tests, you should run:\n\n```\nmdbook test candle-book -L .\\target\\debug\\deps\\ `\n-L native=$env:USERPROFILE\\.cargo\\registry\\src\\index.crates.io-6f17d22bba15001f\\windows_x86_64_msvc-0.42.2\\lib `\n-L native=$env:USERPROFILE\\.cargo\\registry\\src\\index.crates.io-6f17d22bba15001f\\windows_x86_64_msvc-0.48.5\\lib\n```\n\n#### Extremely slow model load time with WSL\n\nThis may be caused by the models being loaded from `/mnt/c`, more details on\n[stackoverflow](https://stackoverflow.com/questions/68972448/why-is-wsl-extremely-slow-when-compared-with-native-windows-npm-yarn-processing).\n\n#### Tracking down errors\n\nYou can set `RUST_BACKTRACE=1` to be provided with backtraces when a candle\nerror is generated.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobinx%2Fgreetup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmobinx%2Fgreetup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobinx%2Fgreetup/lists"}