https://github.com/anakin87/llama2-haystack

Using Llama2 with Haystack, the NLP/LLM framework.
https://github.com/anakin87/llama2-haystack

haystack large-language-models llama llm nlp

Last synced: 28 days ago
JSON representation

Using Llama2 with Haystack, the NLP/LLM framework.

• Host: GitHub
• URL: https://github.com/anakin87/llama2-haystack
• Owner: anakin87
• Created: 2023-07-20T00:30:40.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-07-21T07:08:36.000Z (over 1 year ago)
• Last Synced: 2024-10-18T21:17:09.079Z (about 1 month ago)
• Topics: haystack, large-language-models, llama, llm, nlp
• Language: Jupyter Notebook
• Homepage:
• Size: 451 KB
• Stars: 16
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# llama2-haystack

**(Experimental) Using Llama2 with Haystack**

The [notebook](llama2-haystack.ipynb) contains my hacky experiments in which I try to load and use [Llama2](https://ai.meta.com/llama/) with [Haystack](https://github.com/deepset-ai/haystack), the NLP/LLM framework.

*It's nothing official or well refined, but perhaps it may be useful to other people experimenting.*

![llama2-haystack](./assets/logo.png)

## Notebook summary

1) Installed Transformers from the main branch (and other libraries) 📚

2) Loaded Llama-2-13b-chat-hf on Colab using 4-bit quantizazion, thanks to the great material shared by Younes Belkada 🙌

3) Disabled Tensor Parallelism, which caused some issues 🛠️

4) Installed a minimal version of Haystack

5) Found a hacky way to load the model in Haystack's PromptNode

6) Had a llama-zing chat session, from 🎧🎶 David Guetta to Don Matteo ⛪📿 (an Italian TV series)!