{"id":30176330,"url":"https://github.com/pulp/pulp_hugging_face","last_synced_at":"2026-01-20T17:37:30.230Z","repository":{"id":302797823,"uuid":"1011901895","full_name":"pulp/pulp_hugging_face","owner":"pulp","description":null,"archived":false,"fork":false,"pushed_at":"2025-09-07T15:34:42.000Z","size":114,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-07T17:35:01.662Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pulp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":"COPYRIGHT","agents":null,"dco":null,"cla":null}},"created_at":"2025-07-01T14:00:20.000Z","updated_at":"2025-08-31T15:34:42.000Z","dependencies_parsed_at":"2025-08-04T05:16:37.153Z","dependency_job_id":"d0b87eb5-e0de-439f-8bab-4c60e5bcaf01","html_url":"https://github.com/pulp/pulp_hugging_face","commit_stats":null,"previous_names":["pulp/pulp_hugging_face"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/pulp/pulp_hugging_face","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pulp%2Fpulp_hugging_face","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pulp%2Fpulp_hugging_face/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pulp%2Fpulp_hugging_face/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pulp%2Fpulp_hugging_face/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pulp","download_url":"https://codeload.github.com/pulp/pulp_hugging_face/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pulp%2Fpulp_hugging_face/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274153363,"owners_count":25231465,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-12T02:45:25.183Z","updated_at":"2026-01-20T17:37:30.219Z","avatar_url":"https://github.com/pulp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pulp Hugging Face Plugin\n\nA Pulp plugin for managing Hugging Face Hub content with pull-through caching support.\n\n## Features\n\n- **Pull-through caching**: Automatically fetch and cache content from Hugging Face Hub on first access\n- **Support for all Hugging Face content types**: Models, datasets, and spaces\n- **Authentication support**: Use Hugging Face tokens for private repositories\n- **API proxying**: Forward API requests to Hugging Face Hub for metadata operations\n- **File downloads**: Cache and serve model files, configuration files, and other artifacts\n\n## Installation\n\n```bash\npip install pulp-hugging-face\n```\n\n## Usage\n\n### Setting up a Remote\n\nFirst, create a Hugging Face remote that points to the Hugging Face Hub using the REST API:\n\n```bash\n# Using curl (REST API)\ncurl -X POST https://your-pulp-instance.com/api/pulp/public-domain-name/api/v3/remotes/hugging_face/hugging-face/ \\\n  -H \"Content-Type: application/json\" \\\n  -u admin:password \\\n  -d '{\n    \"name\": \"hf-remote\",\n    \"url\": \"https://huggingface.co\",\n    \"policy\": \"on_demand\"\n  }'\n```\n\nFor private repositories, include your Hugging Face token:\n\n```bash\n# Using curl with authentication token\ncurl -X POST https://your-pulp-instance.com/api/pulp/public-domain-name/api/v3/remotes/hugging_face/hugging-face/ \\\n  -H \"Content-Type: application/json\" \\\n  -u admin:password \\\n  -d '{\n    \"name\": \"hf-private\",\n    \"url\": \"https://huggingface.co\",\n    \"policy\": \"on_demand\",\n    \"hf_token\": \"YOUR_HF_TOKEN\"\n  }'\n```\n\n### Creating a Distribution with Pull-through Caching\n\nCreate a distribution that uses the remote for pull-through caching:\n\n```bash\n# First get the remote href\nREMOTE_HREF=$(curl -s https://your-pulp-instance.com/api/pulp/public-domain-name/api/v3/remotes/hugging_face/hugging-face/ -u admin:password | jq -r '.results[] | select(.name==\"hf-remote\") | .pulp_href')\n\n# Create distribution\ncurl -X POST https://your-pulp-instance.com/api/pulp/public-domain-name/api/v3/distributions/hugging_face/hugging-face/ \\\n  -H \"Content-Type: application/json\" \\\n  -u admin:password \\\n  -d \"{\n    \\\"name\\\": \\\"hf-proxy\\\",\n    \\\"base_path\\\": \\\"huggingface\\\",\n    \\\"remote\\\": \\\"$REMOTE_HREF\\\"\n  }\"\n\nhttps://cert.console.redhat.com/api/pulp-content/public-ytrahnov/huggingface/\n```\n\n\u003e **Note**: CLI support (`pulp hugging-face` commands) is planned but not yet implemented. Currently, you need to use the REST API directly or create a simple script for automation.\n\n### Accessing Content\n\nOnce configured, you can access Hugging Face content through your Pulp instance:\n\n```bash\n# Download a model file\ncurl http://your-pulp-instance/api/pulp-content/public-domain-name/huggingface/microsoft/DialoGPT-medium/resolve/main/config.json\n\n# Access API endpoints\ncurl http://your-pulp-instance/api/pulp-content/public-domain-name/huggingface/api/models/microsoft/DialoGPT-medium\n\n# List repository files\ncurl http://your-pulp-instance/api/pulp-content/public-domain-name/huggingface/models/microsoft/DialoGPT-medium/tree/main\n\n#If you want to use it with the huggingface-cli\nexport HF_ENDPOINT=\"http://your-pulp-instance/api/pulp-content/public-domain-name/huggingface\"\nhuggingface-cli download hf-internal-testing/tiny-random-bert\n```\n\n### How Pull-through Caching Works\n\n1. **First request**: When content is requested but not available locally, Pulp fetches it from Hugging Face Hub\n2. **Caching**: The content is stored locally and associated with the appropriate metadata\n3. **Subsequent requests**: Future requests for the same content are served from the local cache\n4. **API forwarding**: API requests are forwarded to Hugging Face Hub for real-time metadata\n\n### Supported URL Patterns\n\nThe plugin supports the standard Hugging Face Hub URL patterns:\n\n- **File downloads**: `/{repo_id}/resolve/{revision}/{filename}`\n- **API endpoints**: `/api/models/{repo_id}`, `/api/datasets/{repo_id}`, `/api/spaces/{repo_id}`\n- **Repository trees**: `/api/{repo_type}s/{repo_id}/tree/{revision}`\n- **Git LFS**: `/api/{repo_type}s/{repo_id}/git/lfs/*`\n\n### Configuration Options\n\n#### Remote Configuration\n\n- `hf_hub_url`: Base URL for Hugging Face Hub (default: https://huggingface.co)\n- `hf_token`: Authentication token for private repositories\n- `policy`: Set to `on_demand` to enable pull-through caching\n\n#### Content Types\n\nThe plugin handles various Hugging Face content types:\n\n- **Models**: PyTorch models, TensorFlow models, configuration files\n- **Datasets**: Training data, evaluation data, data descriptions\n- **Spaces**: Gradio apps, Streamlit apps, static sites\n\n## Development\n\n### Setting up Development Environment\n\n```bash\ngit clone https://github.com/pulp/pulp_hugging_face.git\ncd pulp_hugging_face\npip install -e .\n```\n\n\n### CLI Support (TODO)\n\nCLI support for this plugin is planned but not yet implemented. The plugin currently supports:\n\n- ✅ **REST API**: Full functionality via `/pulp/api/v3/`\n- ❌ **CLI Commands**: `pulp hugging-face` commands not yet available\n\nTo add CLI support, the following would need to be implemented:\n1. CLI command definitions in a `cli/` directory\n2. Client library generation\n3. Integration with `pulp-cli` package\n\nFor now, use the REST API directly or the provided example script for automation.\n\n## How to File an Issue\n\nFile through this project's GitHub issues and appropriate labels.\n\n\u003e **WARNING** Is this security related? If so, please follow the [Security Disclosures](https://docs.pulpproject.org/pulpcore/bugs-features.html#security-bugs) procedure.\n\n## Contributing\n\nContributions are welcome! Please read our contributing guidelines and submit pull requests to our GitHub repository.\n\n## License\n\nThis project is licensed under the GNU General Public License v2.0 or later.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpulp%2Fpulp_hugging_face","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpulp%2Fpulp_hugging_face","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpulp%2Fpulp_hugging_face/lists"}