https://github.com/m0dulo/InferSpore

🌱 A fully independent Large Language Model (LLM) inference engine, built leveraging cuBLAS and cub. 🧩
https://github.com/m0dulo/InferSpore

cuda inference-engine llama2 llm

Last synced: about 1 month ago
JSON representation

🌱 A fully independent Large Language Model (LLM) inference engine, built leveraging cuBLAS and cub. 🧩

• Host: GitHub
• URL: https://github.com/m0dulo/InferSpore
• Owner: m0dulo
• License: mit
• Created: 2024-05-26T10:14:17.000Z (12 months ago)
• Default Branch: main
• Last Pushed: 2025-04-16T19:46:10.000Z (about 1 month ago)
• Last Synced: 2025-04-17T00:22:31.489Z (about 1 month ago)
• Topics: cuda, inference-engine, llama2, llm
• Language: Cuda
• Homepage:
• Size: 90.8 KB
• Stars: 17
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-cuda-and-hpc - m0dulo/InferSpore

README

        
# 🌱 InferSpore 🧩

A fully independent Large Language Model (LLM) inference engine, built leveraging cuBLAS and cub.