Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/yangjianxin1/yangjianxin1

Last synced: 13 days ago
JSON representation

Host: GitHub
URL: https://github.com/yangjianxin1/yangjianxin1
Owner: yangjianxin1
Created: 2022-01-29T13:04:46.000Z (almost 3 years ago)
Default Branch: main
Last Pushed: 2024-03-09T14:36:27.000Z (10 months ago)
Last Synced: 2024-11-07T13:37:47.388Z (2 months ago)
Size: 15.6 KB
Stars: 0
Watchers: 2
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

### Hi there 👋, I'm Yang Jianxin
[![yangjianxin1's GitHub stats](https://github-readme-stats-git-masterorgs-github-readme-stats-team.vercel.app/api?username=yangjianxin1&hide=prs)](https://github.com/anuraghazra/github-readme-stats)

I'm a NLPer interested in Large Language Model and graduated from [SYSU](https://www.sysu.edu.cn/) with a master's degree.

In my free time, I like to write technical blogs on [[Wechat Official Accounts: YeungNLP]]() and [[Zhihu: 红雨瓢泼]](https://www.zhihu.com/people/jian-xin-15-96)

🔭 Experiences:
- **Shopee**, responsible for building NLP algorithm ability about Customer Service. (from 2022-04 to now)
- **Tencent**, responsible for building NLP algorithm ability about Product Understanding. (from 2021-06 to 2022-04)
- **Alibaba**, Internship at Alibaba (from 2020-06 to 2020-09).

⚙ Here are some my public projects:

| Project | Description | Code |
|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------|
| [Firefly](https://github.com/yangjianxin1/Firefly) | One-stop training for LLMs. Some achievements:
1. [firefly-llama2-13b](https://huggingface.co/YeungNLP/firefly-llama2-13b) ranked **3rd** among all 13B models on [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard), only 0.5 points less than 1st.
2. [firefly-llama-30b](https://huggingface.co/YeungNLP/firefly-llama-30b) ranked **10th** among all 30B models on [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) trained with single V100.
3. [firefly-baichuan-13b](https://huggingface.co/YeungNLP/firefly-baichuan-13b) achieves over **1.63 million downloads**.
4. [firefly-qwen1.5-en-7b-dpo](https://huggingface.co/YeungNLP/firefly-qwen1.5-en-7b-dpo-v0.1) improves 7.21 points compared with the official chat model.
5. [firefly-gemma-7b](https://huggingface.co/YeungNLP/firefly-gemma-7b) improves 9.37 points compared with the official chat model.| ![](https://img.shields.io/github/stars/yangjianxin1/Firefly?style=social) |
| [GPT2-chitchat](https://github.com/yangjianxin1/GPT2-chitchat) | Chinese GPT2 for chitchat | ![](https://img.shields.io/github/stars/yangjianxin1/GPT2-chitchat?style=social) |
| [Firefly-LLaMA2-Chinese](https://github.com/yangjianxin1/Firefly-LLaMA2-Chinese) | Chinese Llama2 with efficient and effective training method. | ![](https://img.shields.io/github/stars/yangjianxin1/Firefly-LLaMA2-Chinese?style=social) |
| [LongQLoRA](https://github.com/yangjianxin1/LongQLoRA) | Efficient and Effective method for extending context length of Llama2 to 8192 with single V100. [Technical Report](https://arxiv.org/abs/2311.04879) | ![](https://img.shields.io/github/stars/yangjianxin1/LongQLoRA?style=social) |
| [CPM](https://github.com/yangjianxin1/CPM) | Chinese composition model based on CPM | ![](https://img.shields.io/github/stars/yangjianxin1/CPM?style=social) |
| [CLIP-Chinese](https://github.com/yangjianxin1/CLIP-Chinese) | Chinese CLIP model trained with 1.4 million image-text pairs | ![](https://img.shields.io/github/stars/yangjianxin1/CLIP-Chinese?style=social) |
| [ClipCap-Chinese](https://github.com/yangjianxin1/ClipCap-Chinese) | Chinese image caption model based on clip and mengzi | ![](https://img.shields.io/github/stars/yangjianxin1/Clipcap-Chinese?style=social) |
| [OFA-Chinese](https://github.com/yangjianxin1/OFA-Chinese) | Chinese multi-modal unified pre-training model | ![](https://img.shields.io/github/stars/yangjianxin1/OFA-Chinese?style=social) |
| [LLMPruner](https://github.com/yangjianxin1/LLMPruner) | Prune vocabulary of LLMs to save memory in training. | ![](https://img.shields.io/github/stars/yangjianxin1/LLMPruner?style=social) |

📁 Here are some my technical blogs:
- 📝 [使用Firefly在单卡V100上对Qwen1.5进行SFT和DPO，大幅超越Qwen1.5和Gemma](https://mp.weixin.qq.com/s/fTaGzuIZq3Uig0524GiGPA)
- 📝 [图解大模型推理优化之KV Cache](https://mp.weixin.qq.com/s/7Fm8LbUN9jQ2HqxPbUU7UQ)
- 📝 [Mixtral-8x7B MoE大模型微调实践，超越Llama2-65B](https://mp.weixin.qq.com/s/f24e-Tp-1WyXTbVOzePvhg)
- 📝 [LongQLoRA：单卡高效扩展LLaMA2-13B的上下文长度](https://mp.weixin.qq.com/s/lptWXi9sZXd2MTTXZsDiPw)
- 📝 [详解基于调整RoPE旋转角度的大模型长度外推方法](https://mp.weixin.qq.com/s/RtI95hu-ZLxGkdGuNIkERQ)
- 📝 [图解RoPE旋转位置编码及其特性](https://mp.weixin.qq.com/s/-1xVXjoM0imXMC7DKqo-Gw)
- 📝 [QLoRA轻量级增量预训练方案，及汉化Llama2的实践](https://mp.weixin.qq.com/s/26-Qxma9M2wGoTQgOlKRmQ)
- 📝 [源码解析ChatGLM2多轮对话训练方法的不足，以及改进方法](https://mp.weixin.qq.com/s/nhogoWnzl3nrs_77r38_UA)
- 📝 [微调百川Baichuan-13B保姆式教程，手把手教你训练百亿大模型](https://mp.weixin.qq.com/s/ZBY6kbogHjbCQvZBzNEqag)
- 📝 [QLoRA文章解读 & 单卡高效微调bloom-7b1](https://mp.weixin.qq.com/s/DED7yeiE0DibsVzTmMeDOw)
- 📝 [Firefly(流萤): 中文对话式大语言模型](https://mp.weixin.qq.com/s/TX7wj8IzD_EaMTvk0bjRtA)
- 📝 [LLMPruner：大语言模型裁剪工具](https://mp.weixin.qq.com/s/leVtrwZc1zLput51Nr99lw)