{"id":15671435,"url":"https://github.com/winstxnhdw/llm-api","last_synced_at":"2025-07-01T04:40:33.720Z","repository":{"id":210518143,"uuid":"726759339","full_name":"winstxnhdw/llm-api","owner":"winstxnhdw","description":"A fast CPU-based API for Llama 3.2 using CTranslate2, hosted on Hugging Face Spaces.","archived":false,"fork":false,"pushed_at":"2025-06-23T00:00:34.000Z","size":871,"stargazers_count":0,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-23T00:29:12.825Z","etag":null,"topics":["ctranslate2","docker","huggingface","huggingface-spaces","llama","transformers","uv"],"latest_commit_sha":null,"homepage":"https://huggingface.co/spaces/winstxnhdw/llm-api","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/winstxnhdw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-03T10:13:57.000Z","updated_at":"2025-06-23T00:00:38.000Z","dependencies_parsed_at":"2024-01-01T11:26:17.004Z","dependency_job_id":"5be679de-efcc-44e2-9ee6-90439d00ecd4","html_url":"https://github.com/winstxnhdw/llm-api","commit_stats":{"total_commits":147,"total_committers":4,"mean_commits":36.75,"dds":"0.25850340136054417","last_synced_commit":"8af5ae868ca0e4817aceee085f4a91403e015279"},"previous_names":["winstxnhdw/llm-api"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/winstxnhdw/llm-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fllm-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fllm-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fllm-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fllm-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/winstxnhdw","download_url":"https://codeload.github.com/winstxnhdw/llm-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fllm-api/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262899511,"owners_count":23381625,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ctranslate2","docker","huggingface","huggingface-spaces","llama","transformers","uv"],"created_at":"2024-10-03T15:02:18.302Z","updated_at":"2025-07-01T04:40:33.713Z","avatar_url":"https://github.com/winstxnhdw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# llm-api\n\n[![build.yml](https://github.com/winstxnhdw/llm-api/actions/workflows/main.yml/badge.svg)](https://github.com/winstxnhdw/llm-api/actions/workflows/main.yml)\n[![deploy.yml](https://github.com/winstxnhdw/llm-api/actions/workflows/deploy.yml/badge.svg)](https://github.com/winstxnhdw/llm-api/actions/workflows/deploy.yml)\n[![formatter.yml](https://github.com/winstxnhdw/llm-api/actions/workflows/formatter.yml/badge.svg)](https://github.com/winstxnhdw/llm-api/actions/workflows/formatter.yml)\n\n[![Open in Spaces](https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-md-dark.svg)](https://huggingface.co/spaces/winstxnhdw/llm-api)\n[![Open a Pull Request](https://huggingface.co/datasets/huggingface/badges/raw/main/open-a-pr-md-dark.svg)](https://github.com/winstxnhdw/llm-api/compare)\n\nA fast CPU-based API for Llama-3.2, hosted on Hugging Face Spaces. To achieve faster executions, we are using [CTranslate2](https://github.com/OpenNMT/CTranslate2) as our inference engine.\n\n## Usage\n\nSimply cURL the endpoint like in the following.\n\n```bash\ncurl -N 'https://winstxnhdw-llm-api.hf.space/api/v1/chat' \\\n     -H 'Content-Type: application/json' \\\n     -d \\\n     '{\n         \"messages\": [\n             {\n                 \"role\": \"user\",\n                 \"content\": \"What is the capital of France?\"\n             }\n         ]\n      }'\n```\n\n## Development\n\nThere are a few ways to run `llm-api` locally for development.\n\n### Local\n\nIf you spin up the server using `uv`, you may access the Swagger UI at [localhost:49494/schema/swagger](http://localhost:49494/schema/swagger).\n\n```bash\nuv run llm-api\n```\n\n### Docker\n\nYou can access the Swagger UI at [localhost:7860/schema/swagger](http://localhost:7860/schema/swagger) after spinning the server up with Docker.\n\n```bash\ndocker build -f Dockerfile.build -t llm-api .\ndocker run --rm -e SERVER_PORT=7860 -p 7860:7860 llm-api\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwinstxnhdw%2Fllm-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwinstxnhdw%2Fllm-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwinstxnhdw%2Fllm-api/lists"}