https://github.com/notnaton/microllm

My own implementation to run inference on local LLM models
https://github.com/notnaton/microllm

chatgpt llm llm-inference

Last synced: about 2 months ago
JSON representation

My own implementation to run inference on local LLM models

• Host: GitHub
• URL: https://github.com/notnaton/microllm
• Owner: Notnaton
• License: agpl-3.0
• Created: 2023-11-09T19:12:29.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-09-03T12:06:55.000Z (4 months ago)
• Last Synced: 2024-10-11T09:38:34.603Z (2 months ago)
• Topics: chatgpt, llm, llm-inference
• Language: Python
• Homepage:
• Size: 90.8 KB
• Stars: 9
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Microllm

 just the bare basics to run inference on local hardware.

 currently working:

 - read_gguf.py

Refactor made it faster and more compact

TODO:

fix token generation to generate sensible tokens...