{"id":15055274,"url":"https://github.com/notthestallion/graphrag-local-model_huggingface","last_synced_at":"2026-01-05T21:06:36.161Z","repository":{"id":255655068,"uuid":"852124594","full_name":"NotTheStallion/graphrag-local-model_huggingface","owner":"NotTheStallion","description":"Microsoft's graphrag using ollama and hugging face to support all LLMs (Llama3, mistral, gemma2, fine-tuned Llama3 ...).","archived":false,"fork":false,"pushed_at":"2024-09-09T10:44:29.000Z","size":444,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-02T18:35:06.691Z","etag":null,"topics":["graph","graphrag","hugging-face","huggingface","ollama","ollama-api","rag"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NotTheStallion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-04T09:10:48.000Z","updated_at":"2024-09-28T13:18:21.000Z","dependencies_parsed_at":"2024-09-09T12:02:47.721Z","dependency_job_id":null,"html_url":"https://github.com/NotTheStallion/graphrag-local-model_huggingface","commit_stats":null,"previous_names":["notthestallion/graphrag_hugging-face","notthestallion/graphrag-local-model_huggingface"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotTheStallion%2Fgraphrag-local-model_huggingface","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotTheStallion%2Fgraphrag-local-model_huggingface/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotTheStallion%2Fgraphrag-local-model_huggingface/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotTheStallion%2Fgraphrag-local-model_huggingface/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NotTheStallion","download_url":"https://codeload.github.com/NotTheStallion/graphrag-local-model_huggingface/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239126447,"owners_count":19586097,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph","graphrag","hugging-face","huggingface","ollama","ollama-api","rag"],"created_at":"2024-09-24T21:39:52.428Z","updated_at":"2025-10-31T06:30:33.796Z","avatar_url":"https://github.com/NotTheStallion.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GraphRAG with Hugging Face\n\nThis repository enables you to use GraphRAG with any models from Hugging Face, as well as your own models.\n\n## Installation and Setup\n\nTo set up this repository and utilize GraphRAG with local models from Ollama, follow these steps:\n\n### 1. Create and Activate a New Conda Environment\n\nEnsure that Python version 3.10 is being used to prevent any compatibility issues.\n\n```bash\nconda create -n graphrag-ollama-local python=3.10\nconda activate graphrag-ollama-local\n```\n\n### 2. Install Ollama\n\nInstall Ollama by visiting their official website or by running the following commands:\n\n```bash\ncurl -fsSL https://ollama.com/install.sh | sh  # Ollama for Linux\npip install ollama\n```\n\n### 3. Download Required Models via Ollama\n\nSelect from available models such as Mistral, Gemma2, or Qwen2 for the LLM, and any embedding model provided by Ollama:\n\n```bash\nollama pull llama3  # LLM model\nollama pull nomic-embed-text  # Embedding model\n```\n\n### 4. Clone the Repository\n\nClone this repository to your local machine:\n\n```bash\ngit clone https://github.com/TheAiSingularity/graphrag-local-ollama.git\n```\n\n### 5. Install the GraphRAG Package\n\nThis step is crucial. Ensure the package is installed with the following command:\n\n```bash\npip install -e .\n```\n\n### 6. Create the Input Directory\n\nCreate a directory where the experiment data and results will be stored:\n\n```bash\nmkdir -p ./ragtest/input\n```\n\n### 7. Copy Sample Data to the Input Directory\n\nCopy the sample data from the `input/` folder to `./ragtest/input`. You can also add your own `.txt` files here:\n\n```bash\ncp input/* ./ragtest/input\n```\n\n### 8. Initialize the `./ragtest` Directory\n\nInitialize the `./ragtest` directory to generate the required files:\n\n```bash\npython -m graphrag.index --init --root ./ragtest\n```\n\n### 9. Move the `settings.yaml` File\n\nCopy the pre-configured `settings.yaml` file to the `./ragtest` directory:\n\n```bash\ncp settings.yaml ./ragtest\n```\n\nYou can modify this file to experiment with different models. For LLM models, use options like `llama3`, `mistral`, or `phi3`. For embedding models, choose options like `mxbai-embed-large` or `nomic-embed-text`. The full list of models provided by Ollama is available on their website, which can be deployed locally. \n\nTo add your own custom model, refer to the guide [`Testing a New Model`](#testing-a-new-model).\n\nDefault API base URLs:\n\n- LLM: `http://localhost:11434/v1`\n- Embeddings: `http://localhost:11434/api`\n\n### 10. Run Indexing to Create a Graph\n\nRun the indexing process to generate a graph:\n\n```bash\npython -m graphrag.index --root ./ragtest\n```\n\n### 11. Run a Query (Global Method Only)\n\nExecute a query using the global method:\n\n```bash\npython -m graphrag.query --root ./ragtest --method global \"What are some common applications and challenges of using MRI in medical imaging?\"\n```\n\n## Testing a New Model\n\nTo test a new model, follow these steps:\n\n1. Change the model name.\n\n2. Install the required dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n3. Convert the Hugging Face model to GGUF format:\n\n```bash\npython convert_hf_to_gguf.py -h\n```\n\n4. Use the following command to convert the custom model:\n\n```bash\npython3 llama.cpp/convert_hf_to_gguf.py custom_model --outfile custom_model.gguf --outtype q8_0\n```\n\n### Importing from GGUF\n\nOllama supports importing GGUF models in the `Modelfile`:\n\n1. Create a file named `Modelfile`, with a `FROM` instruction pointing to the local file path of the model you want to import:\n\n```bash\nFROM ./custom_model.gguf\n```\n\n2. Create the model in Ollama:\n\n```bash\nollama create custom_model -f Modelfile\n```\n\n3. Run the model:\n\n```bash\nollama run custom_model \n```\nYou can also just use `ollama list` to check if the model is correctly added.\n\n4. Update `settings.yaml` to include `custom_model` and copy it to the `./ragtest` directory:\n\n```bash\ncp settings.yaml ./ragtest\n```\n\n5. Run the indexing process:\n\n```bash\npython -m graphrag.index --root ./ragtest\n```\n\n\nNote : Some custom models will make your graphrag crash if they do not follow the openai prompt style (change in `settings.yaml`), embedding (change in `settings.yaml`), or output a bad JSON format. So make sure to make the appropriate changes to adapt the graph rag to your own model.\n\n\n## Citations\n\n- Original GraphRAG repository by Microsoft: [GraphRAG](https://github.com/microsoft/graphrag)\n- Ollama: [Ollama](https://ollama.com/)\n\nInspired by : [graph-rag-ollama](https://github.com/TheAiSingularity/graphrag-local-ollama)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnotthestallion%2Fgraphrag-local-model_huggingface","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnotthestallion%2Fgraphrag-local-model_huggingface","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnotthestallion%2Fgraphrag-local-model_huggingface/lists"}