Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tongyx361/awesome-llm-research
Curation of resources for LLM research, screened by @tongyx361 to ensure high quality and accompanied with elaborately-written concise descriptions to help readers get the gist as quickly as possible.
https://github.com/tongyx361/awesome-llm-research
List: awesome-llm-research
llm research
Last synced: about 6 hours ago
JSON representation
Curation of resources for LLM research, screened by @tongyx361 to ensure high quality and accompanied with elaborately-written concise descriptions to help readers get the gist as quickly as possible.
- Host: GitHub
- URL: https://github.com/tongyx361/awesome-llm-research
- Owner: tongyx361
- License: mit
- Created: 2024-04-18T12:46:37.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-07-12T13:36:55.000Z (4 months ago)
- Last Synced: 2024-07-13T09:32:32.987Z (4 months ago)
- Topics: llm, research
- Homepage: https://tongyx361.notion.site/Awesome-LLM-Research-7b999071d476409cb1fbfdd081f87086?pvs=4
- Size: 68.4 KB
- Stars: 25
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Awesome LLM Research
> Curation of resources for **LLM research**, **screened** by @tongyx361 to ensure **high quality** and accompanied with **elaborately-written concise descriptions** to help readers get the gist as quickly as possible.
[](https://github.com/tongyx361/Awesome-LLM-Research) [](https://opensource.org/licenses/MIT)
🐱 [GitHub](https://github.com/tongyx361/Awesome-LLM-Research) | 📝 [Notion (Interactable)](https://tongyx361.notion.site/Awesome-LLM-Research-7b999071d476409cb1fbfdd081f87086) | 🐦 [X(Twitter)](https://twitter.com/tongyx361/status/1780956572384145515) | 🐶 [Zhihu(知乎)](https://zhuanlan.zhihu.com/p/708331040)
✨ Featured by:
- Theory & practice **comprehensive introductory** materials.
- **Classic/high-quality** information sources.
- **Latest hot-spot** information sources.
📊 There is also [an **interactable (i.e. sort / filter / search)** version of the following table](https://tongyx361.notion.site/6958f3f8753a4458813991a709894699?v=af2e57fc6c274a74a1404452c9014bb4).
📥 You can **subscribe to our updates** in the following ways:
- **Follow** the [**X(Twitter) account** @tongyx361](https://x.com/tongyx361),
- **Follow** the [**Zhihu(知乎) account** @天欲雪](https://www.zhihu.com/people/bai-li-tian-he-84),
- **Watch releases in this GitHub repository**: upper right corner→Watch->Custom->Releases.
📢 If you have any **suggestions**, please don't hesitate to
- **comment** in the [**Notion** page](https://www.notion.so/tongyx361/Awesome-LLM-Research-7b999071d476409cb1fbfdd081f87086),
- **reply** to the [**X(Twitter)** thread](https://twitter.com/tongyx361/status/1780956572384145515),
- post an **issue** in the [**GitHub** repository](https://github.com/tongyx361/Awesome-LLM-Research),
- or [**E-mail** *Yuxuan Tong*]([email protected]).
| Link | Abstract | Description | Language | Modality | Update Cycle | Type |
| ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ---------------- | ------------ | ----------- |
| [国立台湾大学: 李宏毅机器学习 - CS自学指南](https://csdiy.wiki/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/LHY/) | **Basic theory and fundamental works** of Deep Learning | Lectures from different years have different focuses, e.g. 2023 focuses on LLM. | EN(Text) ZH(Speech) | Speech Text Code | Year | Basic |
| [Introduction - Hugging Face NLP Course](https://huggingface.co/learn/nlp-course/chapter1/1) | Basic NLP **practice** (based on HuggingFace ecosystem) | *HuggingFace* is so accessible that its success is a given (but this also comes with some hidden price for developers). | EN ZH … | Text Code | Dynamic | Basic |
| [Yao Fu’s Blog](https://yaofu.notion.site/Yao-Fu-s-Blog-b536c3d6912149a395931f1e871370db) | Fundamental research topics **walkthrough** | Such as emergent abilities, reasoning, long-context modeling. | EN | Text | Months | Fundamental |
| [Transformer Math 101 \| EleutherAI Blog](https://blog.eleuther.ai/transformer-math/) | *Transformer*-related math estimation - Basic | Basic arithmetic about *Transformer*-based models. | EN | Text | None | Basic |
| [分析transformer模型的参数量、计算量、中间激活、KV cache - 知乎](https://zhuanlan.zhihu.com/p/624740065) | *Transformer*-related math estimation - Mediate | Detailed analysis of calculations in *Transformer*-based model. | ZH | Text | None | Basic |
| [*紫气东来* - 知乎](https://www.zhihu.com/people/zi-qi-dong-lai-1/posts) | **Specific** engineering details | Such as inference and training frameworks. | ZH | Text | Weeks | Practical |
| [GitHub - liguodongiot/llm-action](https://github.com/liguodongiot/llm-action?tab=readme-ov-file) | Engineering detail **summaries** | Summarizing AI engineering techniques, such as inference, parallel computing, etc. | ZH | Text | Days | Practical |
| 微信公众号：*大猿搬砖简记* | **Illustrated** **source code (e.g. vLLM, CUDA)** and algorithms (e.g. FlashAttention) | | ZH | Text | Weeks | Practical |
| [游凯超 - 知乎](https://www.zhihu.com/people/youkaichao) | **Infrastructure-level** engineering details | Such as *CUDA*, *NCCL*, `torch.compile` and other side infrastructures like *Docker*, etc. | ZH | Text | Days | Practical |
| [Alignment Guidebook - Notion](https://efficient-unicorn-451.notion.site/Alignment-Guidebook-e5c64df77c0a4b528b7951e87337fa78) | Introduction to LLM **Alignment (SFT + RL)** | | EN | Text | Dynamic | Basic |
| [Spinning Up in Deep RL! — Spinning Up documentation](https://spinningup.openai.com/en/latest/) | Basic **Deep RL** | | EN | Text Code
| None | Basic |
| [科学空间\|Scientific Spaces](https://kexue.fm/) | Blogs combining **graceful theories** and solid experiments | Blogs by *Jianlin Su (苏剑林)*, the author of *RoPE* (de facto standard of positional encoding now), versed in math and ML theory while not unfamiliar with experiments and practice. | ZH | Text | Weeks | Fundamental |
| [Research](https://openai.com/research) | ***OpenAI*** research blogs | “We keep re-discovering what *OpenAI* discovered five years ago.” | EN | Text | Months | Fundamental |
| [Research \\ Anthropic](https://www.anthropic.com/research) | ***Anthropic*** research blogs | | EN | Text | Months | Fundamental |
| [Transformer Circuits Thread](https://transformer-circuits.pub/) | Amazingly insightful and **open** ***Anthropic*** **interpretability** team research blogs | | EN | Text | Month | Fundamental |
| E.g. [\[2312.11805\] Gemini: A Family of Highly Capable Multimodal Models](https://arxiv.org/abs/2312.11805) | LLM **technical reports** | Such technical reports, while usually not very detailed, often do reveal some important details of SotA LLMs. | EN | Text | Months | Fundamental |
| [Hazy Research](https://hazyresearch.stanford.edu/blog) | Blogs of **pioneer visions** | Blogs from *Hazy Research* led by *Christopher Ré* @ *Stanford* (one of the best NLP&AI research groups around the world). | EN | Text | Months | Fundamental |
| [Ilya 30u30](https://arc.net/folder/D0472A20-9C20-4D3F-B145-D2865C0A9FEE) | Short reading list to understand **the fundamentals of the AI today**, said to be **from *Ilya***. | Not the most frontier and not the most suitable for research starters, but really fundamental for essential understanding. | EN | Text | None | Fundamental |
| [FAI-Seminar](https://www.fai-seminar.ac.cn/) | High-quality talks (largely contributed by **Yao class alumna**) | | ZH | Speech Text | Week | Trending |
| [Cool Papers - Immersive Paper Discovery](https://papers.cool/) | **Daily *arXiv*** paper & *Kimi* interaction | | EN | Text | Day | Trending |
| [Daily Papers - Hugging Face](https://huggingface.co/papers) | The most popular paper selection on *Twitter*. | | EN | Text | Day | Trending |
| 微信公众号： *SparksofAGI* | Individual paper selection, some of which **common popular paper collections might not notice** | Selected by *Jianbo Dai* (戴建波)* (senior researcher at *Huawei*). | ZH | Text | Weeks | Trending |
| 微信公众号： *AINLP* | **Curations** of other AI 微信公众号：s | | ZH | Text | Day | Trending |
| 中文 AI 媒体四大顶号：*机器之心*、*新智元*、*量子位*、*夕小瑶科技说* | **Popular** paper selection | | ZH | Text | Day | Trending |
| 微信公众号： *arXiv 每日学术速递* | *arXiv* paper from **broader domains** | | ZH | Text | Day | Auxiliary |
| 微信公众号： *AI 前线* | Various AI news **(not limited to research)** | | ZH | Text | Day | Auxiliary |
| Video channel *Song Zhao* ([*YouTube*](https://www.youtube.com/@zhaosong2031) / [*BiliBili*](https://space.bilibili.com/3546587376650961)) | Various **practical academic-relevant affairs** (e.g. paper submission, job choices) | A little “abstract” though … | ZH | Speech Text | Weeks | Auxiliary |