{"id":22528646,"url":"https://github.com/huggingface/smol-course","last_synced_at":"2025-10-14T15:29:06.346Z","repository":{"id":265309882,"uuid":"894113337","full_name":"huggingface/smol-course","owner":"huggingface","description":"A course on aligning smol models.","archived":false,"fork":false,"pushed_at":"2025-10-01T10:19:13.000Z","size":11192,"stargazers_count":6441,"open_issues_count":74,"forks_count":2288,"subscribers_count":45,"default_branch":"main","last_synced_at":"2025-10-12T15:05:46.843Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/huggingface.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-11-25T19:22:43.000Z","updated_at":"2025-10-12T14:42:54.000Z","dependencies_parsed_at":"2024-12-17T13:30:35.755Z","dependency_job_id":"ad71003e-2594-4903-a4f8-33c71e122a73","html_url":"https://github.com/huggingface/smol-course","commit_stats":null,"previous_names":["burtenshaw/smol-course","huggingface/smol-course"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/huggingface/smol-course","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2Fsmol-course","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2Fsmol-course/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2Fsmol-course/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2Fsmol-course/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/huggingface","download_url":"https://codeload.github.com/huggingface/smol-course/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2Fsmol-course/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279019314,"owners_count":26086711,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-07T07:03:04.332Z","updated_at":"2025-10-14T15:29:06.341Z","avatar_url":"https://github.com/huggingface.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook","A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"![smolcourse image](./banner.png)\n\n# a smol course\n\nThis is a practical course on aligning language models for your specific use case. It's a handy way to get started with aligning language models, because everything runs on most local machines. There are minimal GPU requirements and no paid services. The course is built around the [SmolLM3](https://huggingface.co/blog/smollm3) and [SmolVLM2](https://huggingface.co/blog/smolvlm2) models, but the skills you'll learn can be applied to larger models or other small LLMs/VLMs as well.\n\n\u003cdiv style=\"background: linear-gradient(to right, #e0f7fa, #e1bee7, orange); padding: 20px; border-radius: 5px; margin-bottom: 20px; color: purple;\"\u003e\n    \u003ch2\u003esmol course v2 is live!\u003c/h2\u003e\n    \u003cp\u003eThis course is open and peer reviewed. To get involved with the course \u003cstrong\u003eopen a pull request\u003c/strong\u003e and submit your work for review. Here are the steps:\u003c/p\u003e\n    \u003col\u003e\n        \u003cli\u003eFollow the \u003ca href=\"https://huggingface.co/smol-course\"\u003eHugging Face Hub org\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003eRead the material, make changes, do the exercises, add your own examples.\u003c/li\u003e\n        \u003cli\u003eSubmit a model to the leaderboard\u003c/li\u003e\n        \u003cli\u003eClimb the leaderboard\u003c/li\u003e\n    \u003c/ol\u003e\n    \u003cp\u003eThis should help you learn and to build a community-driven course that is always improving.\u003c/p\u003e\n\u003c/div\u003e\n\n\n\u003ca href=\"http://hf.co/join/discord\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/Discord-7289DA?\u0026logo=discord\u0026logoColor=white\"/\u003e\n\u003c/a\u003e\n\n\n\n\n## Future of this course\n\nThis course will soon be re-released on Hugging Face Learn! Stay tuned for updates.\n\n## Course Outline\n\nThis course provides a practical, hands-on approach to working with small language models, from initial training through to production deployment.\n\n| # | Topic | Description | Released |\n| - | ----- | ----------- | -------- |\n| 1 | Instruction Tuning | Supervised fine-tuning, chat templates, instruction following | ✅ |\n| 2 | Evaluation | Benchmarks and custom domain evaluation | ✅ |\n| 3 | Preference Alignment | Aligning models to human preferences with algorithms like DPO. | ✅ |\n| 4 | Vision Language Models | Adapt and use multimodal models | ✅ |\n| 5 | Reinforcement Learning | Optimizing models with based on reinforcement policies. | October 2025 |\n| 6 | Synthetic Data | Generate synthetic datasets for custom domains | November 2025 |\n| 7 | Award Ceremony | Showcase projects and celebrate | December 2025 |\n\n\n## Why Small Language Models?\n\nWhile large language models have shown impressive capabilities, they often require significant computational resources and can be overkill for focused applications. Small language models offer several advantages for domain-specific applications:\n\n- **Efficiency**: Require significantly less computational resources to train and deploy\n- **Customization**: Easier to fine-tune and adapt to specific domains\n- **Control**: Better understanding and control of model behavior\n- **Cost**: Lower operational costs for training and inference\n- **Privacy**: Can be run locally without sending data to external APIs\n- **Green Technology**: Advocates efficient usage of resources with reduced carbon footprint\n- **Easier Academic Research Development**: Provides an easy starter for academic research with cutting-edge LLMs with less logistical constraints\n\n## Prerequisites\n\nBefore starting, ensure you have the following:\n- Basic understanding of machine learning and natural language processing.\n- Familiarity with Python, PyTorch, and the `transformers` library.\n- Access to a pre-trained language model and a labeled dataset.\n\n## v1 of the course\n\nThe first version of the course used GithHub markdown and Jupyter notebooks. You can find it in the [v1](./v1) directory.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuggingface%2Fsmol-course","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhuggingface%2Fsmol-course","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuggingface%2Fsmol-course/lists"}