{"id":48127851,"url":"https://github.com/bawolf/breaking_vision_clip_cog","last_synced_at":"2026-04-04T16:28:07.015Z","repository":{"id":262531310,"uuid":"881107176","full_name":"bawolf/breaking_vision_clip_cog","owner":"bawolf","description":null,"archived":false,"fork":false,"pushed_at":"2024-12-12T05:49:02.000Z","size":62,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-12T06:29:12.368Z","etag":null,"topics":["ai","breakdance","computer-vision"],"latest_commit_sha":null,"homepage":"https://breakingvision.bawolf.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bawolf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-30T23:13:52.000Z","updated_at":"2024-12-12T05:49:06.000Z","dependencies_parsed_at":"2024-11-12T23:32:13.240Z","dependency_job_id":"69612617-e6af-4e37-a3ef-f1af0829cb6f","html_url":"https://github.com/bawolf/breaking_vision_clip_cog","commit_stats":null,"previous_names":["bawolf/breaking_vision_clip_cog"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bawolf/breaking_vision_clip_cog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bawolf%2Fbreaking_vision_clip_cog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bawolf%2Fbreaking_vision_clip_cog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bawolf%2Fbreaking_vision_clip_cog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bawolf%2Fbreaking_vision_clip_cog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bawolf","download_url":"https://codeload.github.com/bawolf/breaking_vision_clip_cog/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bawolf%2Fbreaking_vision_clip_cog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31405701,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","breakdance","computer-vision"],"created_at":"2026-04-04T16:28:06.430Z","updated_at":"2026-04-04T16:28:07.008Z","avatar_url":"https://github.com/bawolf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\nlicense: mit\nlanguage:\n- en\nbase_model:\n- openai/clip-vit-large-patch14\npipeline_tag: video-classification\ntags:\n- dance\n- vision\n- breaking\n---\n# CLIP-Based Break Dance Move Classifier\n\nA deep learning model for classifying break dance moves using CLIP (Contrastive Language-Image Pre-Training) embeddings. The model is fine-tuned on break dance videos to classify different power moves including windmills, halos, and swipes.\n\n## Features\n\n- Video-based classification using CLIP embeddings\n- Multi-frame temporal analysis\n- Configurable frame sampling and data augmentation\n- Real-time inference using Cog\n- Misclassification analysis tools\n- Hyperparameter tuning support\n\n## Setup\n\n```bash\n# Install dependencies\npip install -r requirements.txt\n\n# Install Cog (if not already installed)\ncurl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`\nchmod +x /usr/local/bin/cog\n```\n\n## Cog\n\ndownload the weights\n\n```bash\ngdown https://drive.google.com/uc?id=1Gn3UdoKffKJwz84GnGx-WMFTwZuvDsuf -O ./checkpoints/\n```\n\nbuild the image\n\n```bash\ncog build --separate-weights\n```\n\npush a new image\n\n```bash\ncog push\n```\n\n## Training\n\ndownload the training data\n\n```bash\ngdown https://drive.google.com/uc?id=11M6nSuSuvoU2wpcV_-6KFqCzEMGP75q6?usp=drive_link -O ./data/\n```\n\n```bash\n# Run training with default configuration\npython scripts/train.py\n\n# Run hyperparameter tuning\npython scripts/hyperparameter_tuning.py\n```\n\n## Inference\n\n```bash\n# Using Cog for inference\ncog predict -i video=@path/to/your/video.mp4\n\n# Using standard Python script\npython scripts/inference.py --video path/to/your/video.mp4\n```\n\n## Analysis\n\n```bash\n# Generate misclassification report\npython scripts/visualization/miscalculations_report.py\n\n# Visualize model performance\npython scripts/visualization/visualize.py\n```\n\n## Project Structure\n\n```\nclip/\n├── src/                    # Source code\n│   ├── data/              # Dataset and data processing\n│   ├── models/            # Model architecture\n│   └── utils/             # Utility functions\n├── scripts/               # Training and inference scripts\n│   └── visualization/     # Visualization tools\n├── config/                # Configuration files\n├── runs/                  # Training runs and checkpoints\n├── cog.yaml              # Cog configuration\n└── requirements.txt      # Python dependencies\n```\n\n## Training Data\n\nTo run training on your own, you can find the training data [here](https://drive.google.com/drive/folders/11M6nSuSuvoU2wpcV_-6KFqCzEMGP75q6?usp=drive_link) and put it in the a directory at the root of the project called `./data`.\n\n## Checkpoints\n\nTo run predictions with cog or locally on an existing checkpoint, you can find a checkpoint and configuration files [here](https://drive.google.com/drive/folders/1Gn3UdoKffKJwz84GnGx-WMFTwZuvDsuf?usp=sharing) and put them in the a directory at the root of the project called `./checkpoints`.\n\n## Model Architecture\n\n- Base: CLIP ViT-Large/14\n- Custom temporal pooling layer\n- Fine-tuned vision encoder (last 3 layers)\n- Output: 4-class classifier\n\n## License\n\nMIT License\n\nCopyright (c) 2024 Bryant Wolf\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Citation\n\nIf you use this model in your research, please cite:\n\n```bibtex\n@misc{clip-breakdance-classifier,\n  author = {Bryant Wolf},\n  title = {CLIP-Based Break Dance Move Classifier},\n  year = {2024},\n  publisher = {Hugging Face},\n  journal = {Hugging Face Model Hub},\n  howpublished = {\\url{https://github.com/bawolf/breaking_vision_clip_cog}}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbawolf%2Fbreaking_vision_clip_cog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbawolf%2Fbreaking_vision_clip_cog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbawolf%2Fbreaking_vision_clip_cog/lists"}