https://github.com/w3spi5/codepack
CLI tool to extract folder structure and file contents with a directory excluded possibility
https://github.com/w3spi5/codepack
Last synced: 4 months ago
JSON representation
CLI tool to extract folder structure and file contents with a directory excluded possibility
- Host: GitHub
- URL: https://github.com/w3spi5/codepack
- Owner: w3spi5
- License: mit
- Created: 2025-01-03T17:42:32.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-12-29T17:07:46.000Z (5 months ago)
- Last Synced: 2025-12-29T21:24:23.681Z (5 months ago)
- Language: Shell
- Size: 707 KB
- Stars: 14
- Watchers: 1
- Forks: 0
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
- awesome-cli-apps-in-a-csv - codepack v4 - CLI tool to extract folder structure and file contents with advanced minification for AI processing. (<a name="ai"></a>AI / ChatGPT)
- awesome-cli-apps - codepack v4 - CLI tool to extract folder structure and file contents with advanced minification for AI processing. (<a name="ai"></a>AI / ChatGPT)
README
# codepack v4
[See latest releases](https://github.com/w3spi5/codepack/releases)
Ultra-efficient CLI tool to extract folder structure and file contents with advanced minification for AI processing.
It's an essential utility for developers, system administrators, and power users who need an efficient solution for file management, data extraction, and AI-ready code preparation.
## Features
- 📂 Generate a complete directory structure overview
- 📄 Extract the content of all files in a directory tree
- 🚫 Automatically exclude specific directories from scanning (`.git`, `node_modules`, `venv`, `__pycache__`, `.next`, `dist`, `build`)
- 🔧 Filter files by extension with `--exclude` option
- 🎯 Include only specific file types with `--include` option
- ⚡ **ULTRA-AGGRESSIVE:** Advanced minification with `--minify` option (50-70% size reduction)
- 🛠️ **SMART:** External minification tools integration for maximum compression
- 📦 Compress output files with `--compress` option
- 📋 **CLIPBOARD:** Copy output directly to clipboard with `--copy`
- 🔍 **DEBUG MODE:** Advanced debugging with `--debug` option for troubleshooting
- 🗂️ **INTELLIGENT:** Skip empty files automatically (no more useless sections)
- 🧠 **AI TOKENS:** Estimated token count in stats
- 🚫 **BINARY SAFE:** Automatically detects and skips binary files
- 🔄 Support for special files (`.gitignore`, `Dockerfile`, `README`, etc.)
- 🎨 Produces aesthetically pleasing tree-like output
- 🔄 Supports versioning of extracted data
- ✅ Automated tests and continuous integration with GitHub Actions
- ✨ **NEW:** **DEFAULT CURRENT DIRECTORY:** Simply run `./codepack` to process current directory
## Functionality Checklist
- ✅ **Directory Structure Generation**
- Tree-like visualization of folders and files
- Proper indentation and branch lines
- Marking of excluded directories and files
- Non-recursive exploration of excluded directories
- ✅ **Content Extraction**
- Multi-file content extraction in a single operation
- Proper file content formatting with headers
- Automatic character encoding fixes
- Filtering of non-printable characters
- Intelligent empty file detection and skipping
- ✅ **Filtering Options**
- Automatic exclusion of system directories (`.git`, `node_modules`, `venv`, `__pycache__`, `.next`, `dist`, `build`)
- Automatic exclusion of sensitive files (`.env`, `.DS_Store`)
- Extension-based inclusion with `--include` option
- Extension-based exclusion with `--exclude` option
- ✅ **Ultra-Aggressive Content Optimization**
- Smart minification with `--minify` option (50-70% reduction)
- External minification tools integration (terser, pyminify, csso, etc.)
- Language-aware ultra-aggressive comment removal
- Advanced whitespace optimization
- Special file types support (`.gitignore`, `Dockerfile`, `YAML`, etc.)
- Python ultra-minification (removes excess spaces around operators)
- AI-optimized content formatting
- ✅ **Tool Management**
- `--install-minifiers` command for automatic tool installation
- `--minify-info` command to check available tools
- Intelligent fallback to bash minification
- External tool detection and configuration
- ✅ **Compression & Output**
- Gzip compression with `--compress` option
- File size reduction statistics
- Compression ratio reporting
- Processing statistics (files processed vs skipped)
- ✅ **User Experience**
- Progress bar with percentage display
- File counter with thousands separator
- Formatted file size display (bytes, KB, MB)
- Clear completion messages
- Timestamped output files
- Detailed minification tool status reporting
- **Default current directory processing** - no path required
- ✅ **Debugging & Troubleshooting**
- Debug mode with `--debug` option
- Detailed file processing information
- Content length tracking
- Minification process monitoring
- Smart debug output (only when requested)
## Use Cases
- Project documentation
- Codebase analysis
- System audits
- Backup preparation
- Development project snapshots
- **AI code analysis and processing** (optimized for Claude, GPT, etc.)
- **Large codebase compression for sharing**
- **Ultra-compact code preparation for AI tools**
- **Quick current directory analysis** (just run `./codepack`)
## Installation
```bash
git clone https://github.com/w3spi5/codepack.git
cd codepack
chmod +x codepack.sh
```
### Install External Minification Tools (Recommended)
For maximum compression (50-70% size reduction), install external tools:
```bash
# Automatic installation
./codepack.sh --install-minifiers
# Manual installation
npm install -g terser csso-cli html-minifier-terser # JavaScript, CSS, HTML
pip3 install pyminify python-minifier # Python
brew install jq # JSON (macOS)
# sudo apt install jq # JSON (Ubuntu)
go install github.com/tdewolff/minify/v2/cmd/minify@latest # Multi-format (optional)
```
## Usage
```bash
./codepack.sh [path/to/directory] [options]
```
**NEW:** The directory path is now **optional**! If no directory is specified, codepack will process the current directory.
### Options
- `--exclude [ ...]` - Exclude files with specified extensions
- `--include [ ...]` - Include ONLY files with specified extensions
- `--minify` - **Ultra-aggressive minification** (50-70% size reduction with external tools)
- `--compress` - Compress output file with gzip
- `--copy` - Copy output to system clipboard (requires pbcopy, xclip, wl-copy, or clip.exe)
- `--debug` - **Enable debug mode** for detailed processing information
- `--install-minifiers` - Install recommended external minification tools
- `--minify-info` - Show status of available minification tools
**Note:** You cannot use both `--include` and `--exclude` at the same time
## Automated Testing
This project now includes automated tests to ensure reliability and stability.
- Basic and advanced tests are located in the `test/` folder.
- To run all tests locally:
```bash
cd test
./test_basic.sh
```
- Tests are also executed automatically on every push and pull request via GitHub Actions.
See the [test/](test/) folder for details.
## Continuous Integration
Every push and pull request triggers a GitHub Actions workflow that:
- Runs minimal and advanced tests (see `.github/workflows/publish.yml`)
- Verifies that the CLI works with `--minify-info` and other options
- Ensures code quality before publishing to npm or GitHub Packages
You can check the build status at the top of this README.
## How to publish
To publish this package to GitHub Packages or npm:
1. Add your npm or GitHub token as a repository secret named `NPM_TOKEN` (see GitHub > Settings > Secrets and variables > Actions).
2. The `.npmrc` file is already configured for GitHub Packages.
3. The publish step is automated in the GitHub Actions workflow.
**Never commit your token in the code or repository.**
### Examples
#### Current Directory Processing
```bash
# Process current directory (no path needed!)
./codepack.sh
# Current directory with ultra-minification (recommended for AI)
./codepack.sh --minify
# Current directory with maximum compression
./codepack.sh --minify --compress
# Current directory with specific file types only
./codepack.sh --include js css html py --minify
# Current directory excluding binary files
./codepack.sh --exclude pdf jpg png --minify
# Current directory with debug mode
./codepack.sh --debug
# Current directory with minification and debug
./codepack.sh --minify --debug
```
#### Specific Directory Processing
```bash
# Process specific directory
./codepack.sh /home/user/project
# Ultra-aggressive minification on specific directory
./codepack.sh /home/user/project --minify
# Maximum compression on specific directory
./codepack.sh /home/user/project --minify --compress
# Process only code files with ultra-minification
./codepack.sh /home/user/project --include js css html py --minify
# Exclude binary files and minify
./codepack.sh /home/user/project --exclude pdf jpg png --minify
# Debug mode for troubleshooting specific directory
./codepack.sh /home/user/project --debug
# Debug with minification to see processing details
./codepack.sh /home/user/project --minify --debug
# Debug with specific file types
./codepack.sh /home/user/project --include js py --minify --debug
```
#### Tool Management
```bash
# Check available minification tools
./codepack.sh --minify-info
# Install minification tools automatically
./codepack.sh --install-minifiers
```
#### Quick Workflow Examples
```bash
# Quick analysis of current project (most common use case)
./codepack.sh --minify
# Share current project with AI tools (ultra-compact)
./codepack.sh --minify --compress
# Debug current project processing
./codepack.sh --debug --minify
# Process only code files in current directory
./codepack.sh --include js py css html --minify
```
## Ultra-Aggressive Minification
The `--minify` option provides **intelligent ultra-aggressive optimization** with **external tool integration**:
### Supported External Tools
#### JavaScript/TypeScript
- **terser** (recommended) - Ultra-aggressive JS minification
- **esbuild** - Fast alternative minifier
- **Expected reduction:** 60-80%
#### Python
- **pyminify** - Specialized Python minifier
- **python-minifier** - Advanced Python optimization
- **Expected reduction:** 50-70%
#### CSS
- **csso-cli** - Advanced CSS optimizer
- **clean-css-cli** - Alternative CSS minifier
- **Expected reduction:** 70-85%
#### HTML
- **html-minifier-terser** - Comprehensive HTML minification
- **Expected reduction:** 40-60%
#### JSON
- **jq** - Native JSON compactor
- **Expected reduction:** 30-50%
#### Multi-format
- **minify** (Go tool) - Universal minifier for multiple formats
### Supported File Types
#### Programming Languages
- **JavaScript/TypeScript** (`.js`, `.jsx`, `.ts`, `.tsx`, `.mjs`)
- **Python** (`.py`, `.pyw`) - **Ultra-aggressive space removal**
- **CSS/Preprocessors** (`.css`, `.scss`, `.sass`, `.less`)
- **HTML** (`.html`, `.htm`, `.xhtml`)
- **Java/C/C++** (`.java`, `.c`, `.cpp`, `.h`, `.hpp`)
- **Shell Scripts** (`.sh`, `.bash`)
- **PHP** (`.php`)
- **Ruby** (`.rb`, `.ruby`)
- **Go** (`.go`)
- **Rust** (`.rs`)
- **XML** (`.xml`, `.xsl`, `.xsd`)
- **SQL** (`.sql`)
#### Configuration & Special Files
- **Ignore files** (`.gitignore`, `.dockerignore`, `.eslintignore`, etc.)
- **Dockerfile** (preserves functionality)
- **YAML** (`.yml`, `.yaml`) - preserves indentation
- **TOML** (`.toml`)
- **Config files** (`.ini`, `.conf`, `.config`)
- **Markdown** (`.md`) - light optimization for AI readability
- **Makefile**
- **README**, **LICENSE** files
### Minification Features
#### Ultra-Aggressive Optimizations
- ✅ **Comments removal** (all types: `//`, `/* */`, `#`, `