{"id":29178341,"url":"https://github.com/alwayssany/huggingface-local-embedding","last_synced_at":"2026-05-05T14:32:13.513Z","repository":{"id":301816345,"uuid":"1010373796","full_name":"AlwaysSany/huggingface-local-embedding","owner":"AlwaysSany","description":"A Fast API server that provides local text and multi-modal embedding using LlamaIndex Hugging Face Embedding","archived":false,"fork":false,"pushed_at":"2025-06-29T01:31:08.000Z","size":1891,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-29T02:29:54.689Z","etag":null,"topics":["docker","embeddings","fastapi","google-colab","huggingface-embeddings","llama-index","local-embeddings","python"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlwaysSany.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-28T23:45:23.000Z","updated_at":"2025-06-29T01:32:07.000Z","dependencies_parsed_at":"2025-06-29T02:40:20.061Z","dependency_job_id":null,"html_url":"https://github.com/AlwaysSany/huggingface-local-embedding","commit_stats":null,"previous_names":["alwayssany/huggingface-local-embedding"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/AlwaysSany/huggingface-local-embedding","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlwaysSany%2Fhuggingface-local-embedding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlwaysSany%2Fhuggingface-local-embedding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlwaysSany%2Fhuggingface-local-embedding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlwaysSany%2Fhuggingface-local-embedding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlwaysSany","download_url":"https://codeload.github.com/AlwaysSany/huggingface-local-embedding/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlwaysSany%2Fhuggingface-local-embedding/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263019619,"owners_count":23400933,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","embeddings","fastapi","google-colab","huggingface-embeddings","llama-index","local-embeddings","python"],"created_at":"2025-07-01T18:46:06.578Z","updated_at":"2026-05-05T14:32:13.469Z","avatar_url":"https://github.com/AlwaysSany.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hugging Face Local Embedding\n\nA FastAPI server that provides local text and multi-modal embedding using LlamaIndex and Hugging Face Embedding models. Supports text, document, and image embedding via API endpoints. Easily deployable locally, with Docker, or on Google Colab.\n\n---\n\n## Project Structure\n\n```\nhuggingface-local-embedding/\n├── README.md                           # Project documentation\n├── requirements.txt                    # Python dependencies\n├── Dockerfile                          # Docker configuration\n├── docker-compose.yml                  # Docker Compose configuration\n├── huggingface_embedding_server.py     # Main FastAPI application\n├── huggingface_embedding_server.ipynb  # Jupyter notebook version\n├── LICENSE                             # MIT License\n└── .gitignore                          # Git ignore rules\n```\n\n---\n\n## Features\n- Local text and multi-modal (image, text) embedding\n- FastAPI server with REST endpoints\n- Uses LlamaIndex and Hugging Face models for embedding\n- Docker and Colab ready\n- Python 3.13.3 compatible\n\n---\n\n## Demo\n\n![FastAPI Server Demo](screenshots/hugging-face-fastapi-endpoint-docs.png)\n\n\n## Prerequisites\n\n- Python 3.13.3 or higher\n- Docker (optional, for containerized deployment)\n- ngrok account (optional, for public URL exposure)\n\n---\n\n## Setup Guide\n\n## Setting ngrok Secrets\n\n### In Google Colab\n- Use Colab secrets:\n  ```python\n  from google.colab import userdata\n  userdata.set_secret('NGROK_AUTH_TOKEN')\n  ```\n- The notebook will read the token and set it for ngrok automatically.\n\n![Colab Secrets](screenshots/google-colab-hugging-face-fastapi-server.png)\n\n### Locally (.env file)\n- Create a `.env` file in the project root using a copy of the `.env.example` file, and set the `NGROK_AUTH_TOKEN` to your ngrok token.\n\n  ```bash\n  cp .env.example .env\n  ```\n\n  ```env\n  NGROK_AUTH_TOKEN=your-ngrok-token-here\n  ```\n\n- The Docker Compose setup will load this automatically. For local runs, you can load it in your shell:\n  ```bash\n  export $(cat .env | xargs)\n  ```\n\n- Or set the token in your Python code before starting ngrok:\n  ```python\n  import os\n  from pyngrok import ngrok\n  ngrok.set_auth_token(os.getenv('NGROK_AUTH_TOKEN'))\n  ```\n\n---\n\n### 1. Standalone (Local) Setup\n\n#### Using pip\n\n- Create a virtual environment and install dependencies:\n```bash\npython3.13 -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\nuvicorn huggingface_embedding_server:app --host 0.0.0.0 --port 8000\n```\n\n#### Using [uv+pip](https://github.com/astral-sh/uv)\n\n- Install uv:\n  ```bash\n  curl -LsSf https://astral.sh/uv/install.sh | sh\n  ```\n\n- Create a virtual environment and install dependencies:\n```bash\nuv venv --python 3.13\nsource .venv/bin/activate\nuv pip install -r requirements.txt\nuvicorn huggingface_embedding_server:app --host 0.0.0.0 --port 8000\n```\n\n#### Using only [uv](https://github.com/astral-sh/uv)\n\n- Install uv:\n  ```bash\n  curl -LsSf https://astral.sh/uv/install.sh | sh\n  ```\n\n- Create a virtual environment and install dependencies:\n```bash\nuv sync\nuvicorn huggingface_embedding_server:app --host 0.0.0.0 --port 8000\n```\n\n### 2. Dockerized Setup\n\n#### Build and Run with Docker\n```bash\ndocker build -t huggingface-embedding-server .\ndocker run --env-file .env -p 8000:8000 huggingface-embedding-server\n```\n\n#### Using Docker Compose\n```bash\ndocker-compose up --build\n```\n\n### 3. Google Colab Setup\n\n- Open `huggingface_embedding_server.ipynb` in Colab.\n- Run the first cell to install dependencies:\n  ```python\n  !pip install fastapi uvicorn pyngrok nest_asyncio llama-index llama-index-embeddings-huggingface\n  ```\n- Set ngrok token in Colab secrets:\n  - Go to Colab menu: `Runtime` \u003e `Change runtime type` \u003e `Hardware accelerator` (optional)\n  - In Colab, run:\n    ```python\n    from google.colab import userdata\n    userdata.set_secret('NGROK_AUTH_TOKEN')\n    ```\n- Run all cells to start the server and expose via ngrok.\n\n---\n\n\n\n## API Endpoints\n\n- `GET /` — Health check\n- `POST /embed_text` — Embed a single text\n- `POST /embed_docs` — Embed a list of texts\n- `POST /embed_image` — Embed an image\n- `POST /embed_batch` — Upload a text file for batch embedding\n- `POST /embed_multimodal` — Embed text and image together\n\n---\n\n## Notes\n- For Hugging Face model access, you may need to set `HF_TOKEN` as a Colab secret or environment variable if using private models.\n- For best performance, use a machine with sufficient RAM and CPU/GPU for model inference.\n- This project is tested with Python 3.13.3 but should work with Python 3.11+.\n- I haved used huggingface sentence-transformers for embedding. You can use any other embedding model from huggingface,\nsee https://huggingface.co/models?library=sentence-transformers for more details.\n\n---\n\n## License\n\nMIT License. See [LICENSE](LICENSE).\n\n---\n\n## Contributing\nTo contribute, please fork the repository and submit a pull request. If you have any questions, please contact me at sany2k8@gmail.com or create an issue with the tag `enhancement` or `bug`.\n\n## Credits\n- [LlamaIndex](https://github.com/run-llama/llama-index)\n- [Hugging Face](https://huggingface.co/)\n- [FastAPI](https://fastapi.tiangolo.com/)\n- [Uvicorn](https://www.uvicorn.org/)\n- [Pyngrok](https://github.com/ngrok/pyngrok)\n\n---\n\n## TODO\n- [ ] Add a way to upload a file from UI and embed it\n- [ ] Add a way to embed a pdf, csv, json, docx, etc. file.\n- [ ] Add a way to embed a video file.\n- [ ] Add a way to embed a audio file.\n\n\n## Contact\n\n- [LinkedIn](https://www.linkedin.com/in/md-sany-ahmed-9ab00745/)\n- [Email](mailto:sany2k8@gmail.com)\n- [Website](https://sany2k8.github.io/)\n- [GitHub](https://github.com/sany2k8)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falwayssany%2Fhuggingface-local-embedding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falwayssany%2Fhuggingface-local-embedding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falwayssany%2Fhuggingface-local-embedding/lists"}