{"id":30582973,"url":"https://github.com/context-labs/cliptagger-playground","last_synced_at":"2025-08-29T08:09:24.053Z","repository":{"id":310738728,"uuid":"1041043564","full_name":"context-labs/cliptagger-playground","owner":"context-labs","description":"A playground for the ClipTagger model.","archived":false,"fork":false,"pushed_at":"2025-08-19T23:13:01.000Z","size":2657,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-20T00:26:38.579Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://cliptagger-playground.vercel.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/context-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-19T22:35:42.000Z","updated_at":"2025-08-19T23:13:04.000Z","dependencies_parsed_at":"2025-08-20T00:26:41.184Z","dependency_job_id":"0c783c28-ea88-458c-ad86-697911616113","html_url":"https://github.com/context-labs/cliptagger-playground","commit_stats":null,"previous_names":["context-labs/cliptagger-playground"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/context-labs/cliptagger-playground","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/context-labs%2Fcliptagger-playground","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/context-labs%2Fcliptagger-playground/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/context-labs%2Fcliptagger-playground/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/context-labs%2Fcliptagger-playground/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/context-labs","download_url":"https://codeload.github.com/context-labs/cliptagger-playground/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/context-labs%2Fcliptagger-playground/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272652384,"owners_count":24970576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-29T02:00:10.610Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-29T08:09:23.359Z","updated_at":"2025-08-29T08:09:24.045Z","avatar_url":"https://github.com/context-labs.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## ClipTagger Demo\n\nClipTagger Demo is a Next.js app that annotates images and video keyframes using the `inference-net/cliptagger-12b` model via the Inference.net Chat Completions API. It lets you:\n\n- Upload a single image for annotation\n- Drop a short video to extract 5 representative frames client‑side and annotate each\n- View strict JSON output with timing/attempt metadata\n- Copy ready‑to‑use API code samples (cURL, TypeScript, Python)\n\nThe server route keeps your API key on the server and returns only the model output to the client.\n\n---\n\n## Quickstart\n\n### Prerequisites\n- Node.js 18+ (recommended 20+)\n- A package manager (pnpm, npm, or yarn)\n- Inference.net API key\n\n### 1) Install dependencies\n```bash\npnpm install\n# or\nnpm install\n```\n\nAvoid having multiple lockfiles in the project to prevent install/build warnings. Use a single tool consistently.\n\n### 2) Configure environment\nCreate a `.env` file in the project root:\n```bash\nINFERENCE_API_KEY=YOUR_API_KEY_HERE\n# Optional: Upstash Redis for rate limiting\n# If not provided, the API runs without rate limiting\nUPSTASH_REDIS_REST_URL=\"https://YOUR-UPSTASH-URL\"\nUPSTASH_REDIS_REST_TOKEN=\"YOUR_UPSTASH_TOKEN\"\n```\n\n### 3) Run the app\n```bash\npnpm dev\n# or\nnpm run dev\n```\n\nVisit http://localhost:3000 and try uploading an image or a video.\n\n### 4) Production build\n```bash\npnpm build \u0026\u0026 pnpm start\n# or\nnpm run build \u0026\u0026 npm start\n```\n\n---\n\n## How it works\n\n- UI: `app/page.tsx` provides the image flow; `components/VideoAnnotator.tsx` extracts 5 frames from a dropped video and invokes the same API per frame.\n- Server: `app/api/annotate/route.ts` calls Inference.net’s Chat Completions endpoint with a strict prompt and returns parsed JSON.\n- Prompts: `lib/prompts.ts` contains the system and user prompts that shape the response.\n- Code samples: `components/CodeModal.tsx` + `lib/code-snippets/*` show how to call the upstream API directly (cURL, TS, Python).\n\nKey model: `inference-net/cliptagger-12b`\n\n---\n\n## API\n\n### POST /api/annotate\nAccepts a base64 data URL for an image and returns structured JSON.\n\nRequest body:\n```json\n{\n  \"imageDataUrl\": \"data:image/png;base64,AAAA...\"\n}\n```\n\nSuccessful response (shape):\n```json\n{\n  \"success\": true,\n  \"result\": {\n    \"description\": \"...\",\n    \"objects\": [\"...\"],\n    \"actions\": [\"...\"],\n    \"environment\": \"...\",\n    \"content_type\": \"...\",\n    \"specific_style\": \"...\",\n    \"production_quality\": \"...\",\n    \"summary\": \"...\",\n    \"logos\": [\"...\"]\n  },\n  \"usage\": {},\n  \"upstreamStatus\": 200,\n  \"attempts\": 1,\n  \"timings\": { \"upstreamMs\": 0, \"totalMs\": 0 }\n}\n```\n\nError response (example):\n```json\n{\n  \"error\": \"Missing INFERENCE_API_KEY server environment variable\"\n}\n```\n\nTest locally with cURL:\n```bash\ncurl -X POST http://localhost:3000/api/annotate \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n    \"imageDataUrl\": \"data:image/png;base64,AAAA...\"\n  }'\n```\n\nSecurity note: the server route reads `INFERENCE_API_KEY` from the server environment and never exposes it to the browser.\n\nIf Upstash Redis variables are set (`UPSTASH_REDIS_REST_URL`, `UPSTASH_REDIS_REST_TOKEN`), the route enforces:\n- 100 requests per IP per hour\n- 500 requests per minute globally\n\nIf those variables are not set, no rate limiting is applied.\n\n---\n\n## Direct API usage (optional)\nIf you prefer to call Inference.net directly from your own backend, see:\n- `lib/code-snippets/curl-code.ts`\n- `lib/code-snippets/ts-code.ts`\n- `lib/code-snippets/python-code.ts`\n\nThese examples target `https://api.inference.net/v1/chat/completions` with the `inference-net/cliptagger-12b` model and a strict JSON response format.\n\n---\n\n## UI notes\n- Image upload accepts JPEG/PNG/WebP/GIF (client limit ~4.5MB).\n- Video workflow extracts 5 uniformly spaced frames client‑side and annotates each independently.\n- Result JSON is syntax‑highlighted and accompanied by timing/attempt metadata.\n\nYou can change the number of video frames by editing `NUM_FRAMES` in `components/VideoAnnotator.tsx`.\n\n---\n\n## Tech stack\n- Next.js 15, React 19\n- Tailwind CSS 4\n- Radix UI (Dialog, Tabs)\n- lucide-react icons\n- highlight.js for code/JSON display\n\n---\n\n## Troubleshooting\n- 500 from `/api/annotate`: ensure `INFERENCE_API_KEY` is set server‑side.\n- Install/build warning about multiple lockfiles: use a single package manager and delete extra lockfiles.\n- Slow or inconsistent upstream responses: the server implements basic retries with backoff; try again or check your network/API quota.\n\n---\n\n## Deploy\nOn platforms like Vercel:\n1. Set `INFERENCE_API_KEY` in your project’s environment variables.\n2. Deploy as usual; the app uses Node.js runtime for the API route and does not expose your key to the client.\n\n---\n\n## License\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontext-labs%2Fcliptagger-playground","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontext-labs%2Fcliptagger-playground","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontext-labs%2Fcliptagger-playground/lists"}