{"id":40872581,"url":"https://github.com/acebot712/rl-token-compression","last_synced_at":"2026-01-22T00:42:02.619Z","repository":{"id":296588739,"uuid":"981406720","full_name":"acebot712/rl-token-compression","owner":"acebot712","description":"A reinforcement learning framework for token compression, including data processing, model training, evaluation, and visualization tools.","archived":false,"fork":false,"pushed_at":"2025-09-06T14:50:47.000Z","size":1886,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-06T16:28:56.338Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/acebot712.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-11T03:11:54.000Z","updated_at":"2025-09-06T14:50:51.000Z","dependencies_parsed_at":"2025-06-01T06:48:35.096Z","dependency_job_id":"2cc1bf4b-0663-4ffc-a3fa-4f4d9ab5840d","html_url":"https://github.com/acebot712/rl-token-compression","commit_stats":null,"previous_names":["acebot712/rl-token-compression"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/acebot712/rl-token-compression","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acebot712%2Frl-token-compression","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acebot712%2Frl-token-compression/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acebot712%2Frl-token-compression/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acebot712%2Frl-token-compression/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/acebot712","download_url":"https://codeload.github.com/acebot712/rl-token-compression/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acebot712%2Frl-token-compression/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28648460,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T21:29:11.980Z","status":"ssl_error","status_checked_at":"2026-01-21T21:24:31.872Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-22T00:42:01.998Z","updated_at":"2026-01-22T00:42:02.614Z","avatar_url":"https://github.com/acebot712.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RL-Based Token Compression\n\nReinforcement learning system that learns to compress text by selectively masking predictable tokens while preserving semantic meaning. Uses a lightweight policy network (1M parameters) to make compression decisions and a fine-tuned GPT-2 model for reconstruction.\n\n## Quick Start\n\n**Try the demo (\u003c 5 seconds):**\n```bash\n./setup.sh \u0026\u0026 source activate.sh\npython demo.py\n```\n\n**Run baseline evaluation:**\n```bash\nsource activate.sh\npython evaluation/evaluate.py --config configs/evaluation/baseline_only.json\n```\n\n**Full training pipeline (requires trained models):**\n```bash\npython data/prepare.py --config configs/data/sample.json\npython training/train.py --config configs/training/default.json  # or mps.json / cuda.json\npython evaluation/evaluate.py --config configs/evaluation/default.json\n```\n\n**Manual setup:**\n```bash\npython -m venv venv \u0026\u0026 source venv/bin/activate\npip install -r requirements.txt\n```\n\n## Key Results\n\n**Baseline Compression Performance (1000 sequences, 30% target):**\n- Random: 29.5% token retention\n- Frequency-based: 29.5% token retention\n- Length-based: 29.5% token retention\n- Position-based: 29.5% token retention\n\n*Baseline evaluation results available in `outputs/evaluation_baseline/`*\n\n**Important Notes:**\n- Results above show baseline compression ratios only\n- Reconstruction quality requires full pipeline: compress → GPT-2 reconstruct → measure\n- No trained RL agent available yet for end-to-end evaluation\n- See `EVALUATION_NOTES.md` for detailed explanation of metrics\n\n**Inference Speed:** Not yet benchmarked. Theoretical speedup from token compression depends on:\n- Compression ratio achieved\n- Overhead of compression decision (policy network forward pass)\n- Reconstructor model efficiency\n\nEnd-to-end speed measurements will be added once trained models are available.\n\n## Architecture\n\n**Lightweight Policy Network**: 1M parameter feedforward network that decides which tokens to keep/mask based on local context and embeddings.\n\n**GPT-2 Reconstructor**: Fine-tuned GPT-2 model that reconstructs original text from masked sequences.\n\n**Joint Training**: Simultaneous training of both networks to solve circular dependency between compression and reconstruction quality.\n\n**Information-Theoretic Rewards**: Rate-distortion framework balances compression efficiency with reconstruction quality.\n\n## Configuration\n\nAll configs are organized in `configs/` with clear hierarchical structure:\n\n**Base Configuration:**\n- `configs/base.json` - Base configuration with common defaults (inherited by other configs)\n\n**Data Preparation:**\n- `configs/data/full.json` - Full dataset preparation for production training\n- `configs/data/sample.json` - Sample dataset (1000 sequences) for testing/development\n\n**Training Configurations:**\n- `configs/training/default.json` - Conservative production training (device=auto, batch_size=64, works across hardware)\n- `configs/training/mps.json` - **MPS-optimized** (Apple Silicon, batch_size=16, memory-efficient, proven working)\n- `configs/training/cuda.json` - **CUDA-optimized** (NVIDIA GPUs, batch_size=256, high-performance)\n- `configs/training/debug.json` - Debug training (1 epoch, 10 steps, minimal resources)\n\n**Evaluation:**\n- `configs/evaluation/default.json` - Comprehensive evaluation with all baselines\n\n**Integration Test Configs:**\n- `configs/integration/data.json` - Ultra-minimal dataset (10 sequences) for fast testing\n- `configs/integration/training.json` - Ultra-fast training (1 epoch, 3 steps) for pipeline validation\n- `configs/integration/evaluation.json` - Minimal evaluation (5 sequences) for smoke testing\n\nOverride any parameter via CLI: `python training/train.py --config configs/training/default.json --batch_size 128`\n\n### Advanced Features\n\n**Distributed Training:**\n```bash\n# Single node, 2 GPUs\npython scripts/launch_distributed.py --config configs/training/default.json --gpus 2\n\n# Multi-node setup\ntorchrun --nproc_per_node=2 --nnodes=2 training/train.py --config configs/training/default.json\n```\n\n**Hyperparameter Optimization:**\n```bash\npython scripts/hyperopt_train.py --config configs/training/default.json --max-trials 50\n```\n\n## Directory Structure\n\n```\nrl-token-compression/\n├── data/           # Data preprocessing\n├── training/       # Joint training system\n├── models/         # Policy network and reconstructor  \n├── evaluation/     # Evaluation framework\n├── configs/        # Configuration files\n├── outputs/        # All training/eval results\n└── utils/          # Shared utilities\n```\n\n## Hardware Optimization\n\n**Apple Silicon (MPS):**\n- Use `configs/training/mps.json` (proven working: batch_size=16, micro_batch_size=2)\n- Memory management handled automatically\n- Gradient accumulation enables effective batch sizes without OOM\n\n**NVIDIA GPUs (CUDA):**\n- Use `configs/training/cuda.json` (high-performance: batch_size=256, micro_batch_size=16)\n- Leverages GPU memory capacity for faster training\n- Higher learning rates for efficient convergence\n\n**CPU/Unknown Hardware:**\n- Use `configs/training/default.json` (conservative: batch_size=64, device=auto)\n- Safe defaults that work across different hardware configurations\n\n## Setup \u0026 Installation\n\n**Automated Setup (Recommended):**\n```bash\n./setup.sh              # Detects platform, installs dependencies\nsource activate.sh       # Activate environment\n```\n\nThe setup script automatically:\n- Detects your compute platform (CPU/CUDA/MPS)\n- Creates virtual environment\n- Installs correct PyTorch variant\n- Validates installation\n- Runs smoke tests\n\n**Manual Setup:**\n```bash\npython -m venv venv\nsource venv/bin/activate  # Windows: venv\\Scripts\\activate\npip install -r requirements.txt\n# Install PyTorch for your platform from pytorch.org\n```\n\n## Troubleshooting\n\n**Setup Issues**: \n- Run `./setup.sh --help` for options\n- Try `./setup.sh --force` to reinstall\n- Use `./setup.sh --cpu` to force CPU-only mode\n\n**Memory Issues**: Reduce `batch_size` or increase `gradient_accumulation_steps` \n**Slow Training**: Check device detection, ensure MPS/CUDA is available\n**Import Errors**: Verify environment is activated: `source activate.sh`\n\n## Implementation Notes\n\n**Why This Architecture?**\n- Simple policy networks with proper training beat complex models with broken paradigms\n- Joint training solves circular dependency without target networks or complex scheduling\n- Information-theoretic rewards provide principled optimization objective\n\n**Memory Management**: Automatic device-specific optimization for MPS/CUDA/CPU with fallback handling.\n\n**Evaluation**: Multi-seed statistical testing with comprehensive baselines ensures robust results.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Facebot712%2Frl-token-compression","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Facebot712%2Frl-token-compression","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Facebot712%2Frl-token-compression/lists"}