https://github.com/notnaton/microllm

My own implementation to run inference on local LLM models
https://github.com/notnaton/microllm

chatgpt llm llm-inference

Last synced: over 1 year ago
JSON representation

My own implementation to run inference on local LLM models

• Host: GitHub
• URL: https://github.com/notnaton/microllm
• Owner: Notnaton
• License: agpl-3.0
• Created: 2023-11-09T19:12:29.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-09-03T12:06:55.000Z (almost 2 years ago)
• Last Synced: 2025-02-28T16:11:00.370Z (over 1 year ago)
• Topics: chatgpt, llm, llm-inference
• Language: Python
• Homepage:
• Size: 90.8 KB
• Stars: 7
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Microllm

 just the bare basics to run inference on local hardware.

 currently working:

 - read_gguf.py

Refactor made it faster and more compact

TODO:

fix token generation to generate sensible tokens...