https://github.com/ericlbuehler/candle-lora-macro

Macros for candle-lora.
https://github.com/ericlbuehler/candle-lora-macro

candle-lora rust

Last synced: 12 months ago
JSON representation

Macros for candle-lora.

• Host: GitHub
• URL: https://github.com/ericlbuehler/candle-lora-macro
• Owner: EricLBuehler
• License: mit
• Created: 2023-09-12T23:49:31.000Z (over 2 years ago)
• Default Branch: master
• Last Pushed: 2023-10-28T20:42:21.000Z (over 2 years ago)
• Last Synced: 2025-05-09T01:12:58.648Z (about 1 year ago)
• Topics: candle-lora, rust
• Language: Rust
• Homepage:
• Size: 1.06 MB
• Stars: 7
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# candle-lora-macro

[![MIT License](https://img.shields.io/badge/License-MIT-informational)](LICENSE)

[![Continuous integration](https://github.com/EricLBuehler/candle-lora-macro/actions/workflows/ci.yml/badge.svg)](https://github.com/EricLBuehler/candle-lora-macro/actions/workflows/ci.yml)

[![Documentation](https://github.com/EricLBuehler/candle-lora-macro/actions/workflows/docs.yml/badge.svg)](https://ericlbuehler.github.io/candle-lora-macro/candle_lora_macro/)

## ⚠️ **This crate is deprecated and has been merged with `candle-lora` itself**.

This library makes using [`candle-lora`](https://github.com/EricLBuehler/candle-lora) as simple as adding 2 macros to your model structs and calling a method! It is inspired by the simplicity of the Python `peft` library's `get_peft_model` method. Like `candle-lora`, the supported concrete layer types are `Linear`, `Conv1d`, `Conv2d`, and `Embedding`.

`candle-lora-macro` exports 2 macros: `AutoLoraConvert` and `replace_layer_fields`.

The `AutoLoraConvert` derive macro automatically creates a method `get_lora_model`, when called which selects and swaps all supported layers for their LoRA counterparts. This method is the equivalent of `peft`'s `get_peft_model` method, and modifies the model in place. It expects all

layers of the supported types to be a `dyn` type, that is `Box`.

In addition, `AutoLoraConvert` also defines a method `get_merged_lora_model` which does everything `get_lora_model` does, but also merges the weights of the LoRA layers to improve inference performance.

To further automate the process of using `candle-lora`, `candle-lora-macro` also provides an attribute macro called `replace_layer_fields`.

`replace_layer_fields` swaps out the concrete types for `dyn` types. If this macro is not added to the model structs, be sure to change the member types to `Box`.

`replace_layer_fields` is able to swap:

- `Linear` to `Box`

- `Conv1d` to `Box`

- `Conv2d` to `Box`

- `Embedding` to `Box`

- `Option` to `Option>`

- `Option` to `Option>`

- `Option` to `Option>`

- `Option` to `Option>`