https://github.com/masa3141/japanese-alpaca-lora

A japanese finetuned instruction LLaMA
https://github.com/masa3141/japanese-alpaca-lora

Last synced: 28 days ago
JSON representation

A japanese finetuned instruction LLaMA

• Host: GitHub
• URL: https://github.com/masa3141/japanese-alpaca-lora
• Owner: masa3141
• License: apache-2.0
• Created: 2023-03-20T04:28:10.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-03-20T06:23:54.000Z (almost 2 years ago)
• Last Synced: 2024-11-10T08:35:33.603Z (about 1 month ago)
• Language: Jupyter Notebook
• Size: 7.62 MB
• Stars: 126
• Watchers: 4
• Forks: 19
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• StarryDivineSky - masa3141/japanese-alpaca-lora

README

        
# Japanese-Alpaca-LoRA

A Japanese finetuned instruction LLaMA.

In accordance with other models being referenced, this model is not available for commercial use and can only be used for research purposes.

## References

This project is based on [LLaMA](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/), [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca), [Alpaca LoRA](https://github.com/tloen/alpaca-lora), [cabrita](https://github.com/22-hours/cabrita)

## Data

We translated the [alpaca_data.json](https://github.com/tatsu-lab/stanford_alpaca/blob/main/alpaca_data.json) to japanese using ChatGPT API.

We paid around US $45 to translate the full dataset to japanese. Translated data is available. ([japanese_alpaca_data.json](data/japanese_alpaca_data.json))

## Finetuning

We just followed [Alpaca LoRA](https://github.com/tloen/alpaca-lora) and [cabrita](https://github.com/22-hours/cabrita). We could run finetuning step using Google Colab PRO+. It took 6.5 hours for finetuning.

## Example outputs

### Good Examples

```

Input: 日本の首都はどこですか？

Alpaca-LoRA-Output: The capital of Japan is Tokyo.

Japanese-Alpaca-LoRA-Output: 「日本の首都は東京です。」

```

```

Input: 高齢の方が罹りやすい病気を3つ挙げてください。

Alpaca-LoRA-Output: Three diseases that are more common in the elderly are heart disease, stroke, and dementia.

Japanese-Alpaca-LoRA-Output:「高齢の方が罹りやすい病気には、心臓疾患、精神疾患、そして呼吸器疾患が含まれます。」

```

### Bad Examples

```

Input: 大阪のオススメの食べ物を教えて

Alpaca-LoRA-Output: Osaka's favorite foods include takoyaki, okonomiyaki, kushikatsu, and yakitori.

Japanese-Alpaca-LoRA-Output:「大阪のオススメの食べ物には、オーストラリアの牛肉フルーツバター、オーストラリアの牛肉フルーツバター、オーストラリアの牛肉フルーツバター、

```

```

Input: 「3x + 2= 20」という方程式を解くとxはいくつになりますか

Alpaca-LoRA-Output:  x = 6

Japanese-Alpaca-LoRA-Output:「3x + 2 = 20」という方程式を解くと、xは2です。」

```