{"id":20147836,"url":"https://github.com/jman4162/pytorch-vision-transformers-vit","last_synced_at":"2026-04-10T12:31:16.337Z","repository":{"id":235552875,"uuid":"790905259","full_name":"jman4162/PyTorch-Vision-Transformers-ViT","owner":"jman4162","description":"Explore fine-tuning the Vision Transformer (ViT) model for object recognition in robotics using PyTorch. This tutorial covers setup, training, and evaluation processes, achieving impressive accuracy with practical resource constraints. Ideal for learners in AI and robotics.","archived":false,"fork":false,"pushed_at":"2026-02-05T17:51:01.000Z","size":88,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-05T17:53:35.588Z","etag":null,"topics":["ai-tutorial","computer-vision","deep-learning","engineering","image-classification","machine-learning","object-recognition","pytorch","pytorch-tutorial","robotics","vision-transformer","vision-transformer-image-classification","vit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jman4162.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-23T18:35:19.000Z","updated_at":"2026-02-05T17:51:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"b71ed5b6-058b-41cd-8c45-6d25f88e3d46","html_url":"https://github.com/jman4162/PyTorch-Vision-Transformers-ViT","commit_stats":null,"previous_names":["jman4162/pytorch-vision-transformers-vit"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jman4162/PyTorch-Vision-Transformers-ViT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jman4162%2FPyTorch-Vision-Transformers-ViT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jman4162%2FPyTorch-Vision-Transformers-ViT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jman4162%2FPyTorch-Vision-Transformers-ViT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jman4162%2FPyTorch-Vision-Transformers-ViT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jman4162","download_url":"https://codeload.github.com/jman4162/PyTorch-Vision-Transformers-ViT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jman4162%2FPyTorch-Vision-Transformers-ViT/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31642669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T07:40:12.752Z","status":"ssl_error","status_checked_at":"2026-04-10T07:40:11.664Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-tutorial","computer-vision","deep-learning","engineering","image-classification","machine-learning","object-recognition","pytorch","pytorch-tutorial","robotics","vision-transformer","vision-transformer-image-classification","vit"],"created_at":"2024-11-13T22:32:20.007Z","updated_at":"2026-04-10T12:31:16.328Z","avatar_url":"https://github.com/jman4162.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vit-trainer\n\n[![PyPI](https://img.shields.io/pypi/v/vit-trainer.svg)](https://pypi.org/project/vit-trainer/)\n[![CI](https://github.com/jman4162/PyTorch-Vision-Transformers-ViT/actions/workflows/ci.yml/badge.svg)](https://github.com/jman4162/PyTorch-Vision-Transformers-ViT/actions/workflows/ci.yml)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/jman4162/PyTorch-Vision-Transformers-ViT/blob/main/notebooks/tutorial.ipynb)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n\nA simple, educational package for fine-tuning Vision Transformer (ViT) models using PyTorch. Achieves **97.65% accuracy** on CIFAR-10 with modern training techniques.\n\n![ViT](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/vit_architecture.jpg)\n\n## Why vit-trainer?\n\n| vs. timm/transformers | vit-trainer |\n|-----------------------|-------------|\n| 1000+ model architectures | Focused on ViT fine-tuning |\n| Complex APIs | Simple, readable code |\n| Research-oriented | Educational + Production ready |\n\n**Features:**\n- Mixed precision training (AMP) for 2-3x speedup\n- AdamW optimizer with cosine annealing + warmup\n- Attention visualization for interpretability\n- ONNX export for deployment\n- CLI and Python API\n\n## Installation\n\n```bash\npip install vit-trainer\n```\n\n### Optional Dependencies\n\n```bash\n# Gradio web demo\npip install \"vit-trainer[demo]\"\n\n# ONNX export\npip install \"vit-trainer[export]\"\n\n# Everything\npip install \"vit-trainer[all]\"\n```\n\n### Install from Source\n\n```bash\ngit clone https://github.com/jman4162/PyTorch-Vision-Transformers-ViT.git\ncd PyTorch-Vision-Transformers-ViT\npip install -e \".[dev]\"\n```\n\n## Quick Start\n\n### Python API\n\n```python\nfrom vit_trainer import Trainer, load_model, get_cifar10_loaders\n\n# Load data and model\ntrain_loader, val_loader, test_loader = get_cifar10_loaders(batch_size=64)\nmodel = load_model(\"vit_b_16\", num_classes=10)\n\n# Train\ntrainer = Trainer(model, lr=1e-4, use_amp=True)\nhistory = trainer.fit(train_loader, val_loader, epochs=10)\n\n# Evaluate\nloss, accuracy = trainer.evaluate(test_loader)\nprint(f\"Test Accuracy: {accuracy:.2f}%\")\n```\n\n### Command Line Interface\n\n```bash\n# Train a model\nvit-train train --model vit_b_16 --dataset cifar10 --epochs 10\n\n# Evaluate a trained model\nvit-train eval --checkpoint best_model.pt --dataset cifar10 --plot-confusion\n\n# Predict on a single image\nvit-train predict --checkpoint best_model.pt --image cat.jpg --show-attention\n\n# Export to ONNX\nvit-train export --checkpoint best_model.pt --output model.onnx\n```\n\n### Configuration Files\n\n```bash\n# Use YAML config\nvit-train train --config configs/default.yaml\n```\n\n## Usage Examples\n\n### Training with Custom Settings\n\n```python\nfrom vit_trainer import Trainer, load_model, get_cifar10_loaders, TrainingConfig\n\n# Create config\nconfig = TrainingConfig(\n    model_variant=\"vit_b_16\",\n    batch_size=64,\n    epochs=10,\n    lr=1e-4,\n    weight_decay=0.05,\n    warmup_epochs=2,\n    patience=3,\n    use_amp=True,\n)\n\n# Train\ntrain_loader, val_loader, _ = get_cifar10_loaders(batch_size=config.batch_size)\nmodel = load_model(config.model_variant, num_classes=10)\ntrainer = Trainer(\n    model,\n    lr=config.lr,\n    weight_decay=config.weight_decay,\n    warmup_epochs=config.warmup_epochs,\n    use_amp=config.use_amp,\n)\ntrainer.fit(train_loader, val_loader, epochs=config.epochs, patience=config.patience)\n```\n\n### Attention Visualization\n\n```python\nfrom vit_trainer import visualize_samples_with_attention, CIFAR10_CLASSES\n\nvisualize_samples_with_attention(\n    model,\n    test_loader.dataset,\n    CIFAR10_CLASSES,\n    num_samples=4,\n)\n```\n\n### Evaluation Metrics\n\n```python\nfrom vit_trainer import get_predictions, compute_metrics, plot_confusion_matrix\n\ny_pred, y_true, probs = get_predictions(model, test_loader)\nmetrics = compute_metrics(y_true, y_pred, CIFAR10_CLASSES)\n\nprint(metrics[\"classification_report\"])\nplot_confusion_matrix(y_true, y_pred, CIFAR10_CLASSES)\n```\n\n### Loading Trained Models\n\n```python\nfrom vit_trainer import load_model\n\n# Load from checkpoint\nmodel = load_model(\n    \"vit_b_16\",\n    num_classes=10,\n    checkpoint_path=\"best_model.pt\",\n)\n```\n\n### ONNX Export\n\n```python\nfrom vit_trainer import load_model, ExportConfig\n\n# Load trained model\nmodel = load_model(\"vit_b_16\", num_classes=10, checkpoint_path=\"best_model.pt\")\n\n# Export to ONNX\nconfig = ExportConfig(output_path=\"model.onnx\", opset_version=14)\nconfig.export(model)\n\n# Or use CLI\n# vit-train export --checkpoint best_model.pt --output model.onnx\n```\n\n## API Reference\n\n```python\nfrom vit_trainer import (\n    # Configuration\n    TrainingConfig,           # Training hyperparameters\n    ExportConfig,             # ONNX export settings\n\n    # Models\n    load_model,               # Load ViT with pretrained weights\n    VIT_VARIANTS,             # Available model variants\n\n    # Data\n    get_cifar10_loaders,      # CIFAR-10 data loaders\n    get_cifar100_loaders,     # CIFAR-100 data loaders\n    CIFAR10_CLASSES,          # Class names\n\n    # Training\n    Trainer,                  # Training loop with AMP\n    EarlyStopping,            # Early stopping callback\n    ModelCheckpoint,          # Save best model\n\n    # Evaluation\n    evaluate_model,           # Loss and accuracy\n    compute_metrics,          # Precision, recall, F1\n    plot_confusion_matrix,    # Visualization\n\n    # Visualization\n    visualize_attention,      # Attention heatmaps\n)\n```\n\n## Project Structure\n\n```\nvit-trainer/\n├── vit_trainer/\n│   ├── __init__.py         # Public API\n│   ├── config.py           # TrainingConfig dataclass\n│   ├── cli.py              # Command-line interface\n│   ├── data/               # Data loaders and transforms\n│   ├── models/             # Model registry and factory\n│   ├── training/           # Trainer and callbacks\n│   ├── evaluation/         # Metrics and plotting\n│   └── visualization/      # Attention maps\n├── tests/                  # Unit tests (44 tests)\n├── configs/                # YAML configurations\n├── notebooks/              # Tutorial notebooks\n├── app.py                  # Gradio demo\n└── pyproject.toml          # Package configuration\n```\n\n## ViT Variants\n\n| Variant | Patch Size | Parameters | ImageNet Acc | Use Case |\n|---------|------------|------------|--------------|----------|\n| `vit_b_16` | 16x16 | 86M | 81.1% | Best accuracy/speed |\n| `vit_b_32` | 32x32 | 88M | 75.9% | Faster inference |\n| `vit_l_16` | 16x16 | 304M | 79.7% | Higher accuracy |\n\n## Training Results\n\n| Metric | Value |\n|--------|-------|\n| **Test Accuracy** | 97.65% |\n| **Model** | vit_b_16 |\n| **Training Time** | ~11 min/epoch (GPU) |\n\n## Gradio Demo\n\n```bash\n# Launch interactive web interface\npython app.py\n# Opens at http://localhost:7860\n```\n\n## Development\n\n```bash\n# Install dev dependencies\npip install -e \".[dev]\"\n\n# Run tests\npytest tests/\n\n# Format code\nblack vit_trainer/\nruff check vit_trainer/\n\n# Type check\nmypy vit_trainer/\n```\n\n## Troubleshooting\n\n### CUDA Out of Memory\n- Reduce batch size: `--batch-size 32` or `16`\n- AMP is enabled by default\n\n### Slow Training on CPU\n- Use Google Colab (free GPU)\n- Training on CPU is very slow (~60 min/epoch)\n\n### Import Errors\n- Make sure to install the package: `pip install vit-trainer`\n\n## Resources\n\n- [Original ViT Paper](https://arxiv.org/abs/2010.11929)\n- [PyTorch ViT Documentation](https://pytorch.org/vision/main/models/vision_transformer.html)\n- [Hugging Face ViT](https://huggingface.co/docs/transformers/en/model_doc/vit)\n- [CIFAR-10 SOTA](https://paperswithcode.com/sota/image-classification-on-cifar-10)\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjman4162%2Fpytorch-vision-transformers-vit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjman4162%2Fpytorch-vision-transformers-vit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjman4162%2Fpytorch-vision-transformers-vit/lists"}