https://github.com/offensiveedge/urls-le
Zero Hassle URL Extraction & Validation - Extract, validate, and analyze URLs from logs, configs, code, and documentation. Features accessibility checking, link validation, pattern analysis, and comprehensive reporting.
https://github.com/offensiveedge/urls-le
accessibility analysis code configs developer-tools documentation extension extraction functional-programming link-validation logs pattern-analysis productivity typescript url-extraction url-validation urls visual-studio-code vscode vscode-extension
Last synced: about 19 hours ago
JSON representation
Zero Hassle URL Extraction & Validation - Extract, validate, and analyze URLs from logs, configs, code, and documentation. Features accessibility checking, link validation, pattern analysis, and comprehensive reporting.
- Host: GitHub
- URL: https://github.com/offensiveedge/urls-le
- Owner: OffensiveEdge
- License: mit
- Created: 2025-10-10T03:11:21.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2025-11-02T16:46:17.000Z (3 months ago)
- Last Synced: 2025-11-02T18:15:50.708Z (3 months ago)
- Topics: accessibility, analysis, code, configs, developer-tools, documentation, extension, extraction, functional-programming, link-validation, logs, pattern-analysis, productivity, typescript, url-extraction, url-validation, urls, visual-studio-code, vscode, vscode-extension
- Language: TypeScript
- Homepage:
- Size: 8.52 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Governance: docs/governance/CHANGELOG_GOVERNANCE.md
Awesome Lists containing this project
README
URLs-LE: Zero Hassle URL Extraction
Extract 10,000+ URLs per second • 100x faster than manual searching
HTML, CSS, JavaScript, JSON, YAML, XML, TOML, INI, Properties, Markdown, and more
---
---
## ⚡ See It In Action
**Before**: Manually searching through HTML/CSS for broken links (20 minutes)
```html
```
**After**: One command extracts all 103 URLs in 0.5 seconds
```
https://cdn.example.com/style.css (line 5)
https://images.example.com/logo.png (line 8)
https://api.example.com/users (line 45)
... (103 URLs total)
```
**Time Saved**: 20 minutes → 1 second ⚡
---
## ✅ Why URLs-LE?
- **10,000+ URLs per second** - 100x faster than manual searching
- **Zero Config** - Install → Press `Cmd+Alt+U` → Done
- **Battle-Tested** - 347 unit tests, 95% coverage, zero critical vulnerabilities
- **Security-Hardened** - 83 tests prevent URL injection, SSRF attacks, protocol exploitation
Perfect for API audits, link validation, and resource tracking.
---
## 🙏 Thank You
If URLs-LE saves you time, a quick rating helps other developers discover it:
⭐ [Open VSX](https://open-vsx.org/extension/OffensiveEdge/urls-le) • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.urls-le)
---
### Key Features
- **Automatic cleanup** - Sort, dedupe, and filter by protocol or domain
- **10+ file formats** - HTML, CSS, JavaScript, JSON, YAML, XML, Markdown, TOML, INI
- **Smart filtering** - Excludes `data:` URIs and `javascript:` pseudo-protocols
- **Fast at scale** - Process large documentation and config files efficiently
- **13 languages** - English, Chinese, German, Spanish, French, Indonesian, Italian, Japanese, Korean, Portuguese, Russian, Ukrainian, Vietnamese
## 🚀 More from the LE Family
- **[String-LE](https://open-vsx.org/extension/OffensiveEdge/string-le)** - Extract user-visible strings for i18n and validation • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.string-le)
- **[Numbers-LE](https://open-vsx.org/extension/OffensiveEdge/numbers-le)** - Extract and analyze numeric data with statistics • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.numbers-le)
- **[EnvSync-LE](https://open-vsx.org/extension/OffensiveEdge/envsync-le)** - Keep .env files in sync with visual diffs • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.envsync-le)
- **[Paths-LE](https://open-vsx.org/extension/OffensiveEdge/paths-le)** - Extract file paths from imports and dependencies • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.paths-le)
- **[Regex-LE](https://open-vsx.org/extension/OffensiveEdge/regex-le)** - Test and validate regex patterns with live feedback • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.regex-le)
- **[Secrets-LE](https://open-vsx.org/extension/OffensiveEdge/secrets-le)** - Detect and sanitize secrets before you commit • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.secrets-le)
- **[Scrape-LE](https://open-vsx.org/extension/OffensiveEdge/scrape-le)** - Validate scraper targets before debugging • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.scrape-le)
- **[Colors-LE](https://open-vsx.org/extension/OffensiveEdge/colors-le)** - Extract and analyze colors from stylesheets • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.colors-le)
- **[Dates-LE](https://open-vsx.org/extension/OffensiveEdge/dates-le)** - Extract temporal data from logs and APIs • [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.dates-le)
## 💡 Use Cases
- **Web Auditing** - Extract all links and resources from HTML/CSS for validation
- **API Documentation** - Pull API endpoints from docs and code for cataloging
- **Link Validation** - Find all external URLs for broken link checking
- **Resource Tracking** - Audit CDN and asset URLs across your project
## 🚀 Quick Start
1. Install from [Open VSX](https://open-vsx.org/extension/OffensiveEdge/urls-le) or [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.urls-le)
2. Open any supported file type (`Cmd/Ctrl + P` → search for "URLs-LE")
3. Run Quick Extract (`Cmd+Alt+U` / `Ctrl+Alt+U` / Status Bar)
## ⚙️ Configuration
URLs-LE has minimal configuration to keep things simple. Most settings are available in VS Code's settings UI under "URLs-LE".
Key settings include:
- Output format preferences (side-by-side, clipboard copy)
- Safety warnings and thresholds for large files
- Notification levels (silent, important, all)
- Status bar visibility
- Local telemetry logging for debugging
For the complete list of available settings, open VS Code Settings and search for "urls-le".
## 🌍 Language Support
**13 languages**: English, German, Spanish, French, Indonesian, Italian, Japanese, Korean, Portuguese (Brazil), Russian, Ukrainian, Vietnamese, Chinese (Simplified)
## 🧩 System Requirements
**VS Code** 1.70.0+ • **Platform** Windows, macOS, Linux
**Memory** 200MB recommended for large files
## 🔒 Privacy
100% local processing. No data leaves your machine. Optional logging: `urls-le.telemetryEnabled`
## ⚡ Performance
URLs-LE is built for speed and efficiently processes files from 100KB to 30MB+. See [detailed benchmarks](docs/PERFORMANCE.md).
| Format | File Size | Throughput | Duration | Memory | Tested On |
| -------- | --------- | ---------- | -------- | ------ | ------------- |
| **JSON** | 1K lines | 1,382,278 | ~1.58 | < 1MB | Apple Silicon |
| **CSS** | 3K lines | 1,048,387 | ~0.31 | < 1MB | Apple Silicon |
| **HTML** | 10K lines | 298,122 | ~4.26 | < 1MB | Apple Silicon |
**Note**: Performance results are based on files containing actual URLs. Files without URLs (like large JSON/CSV data files) are processed much faster but extract 0 URLs.
**Real-World Performance**: Tested with actual data up to 30MB (practical limit: 1MB warning, 10MB error threshold)
**Performance Monitoring**: Built-in real-time tracking with configurable thresholds
**Full Metrics**: [docs/PERFORMANCE.md](docs/PERFORMANCE.md) • Test Environment: macOS, Bun 1.2.22, Node 22.x
For detailed information, see [Performance Monitoring](docs/PERFORMANCE.md).
## 🔧 Troubleshooting
**Not detecting URLs?**
Ensure file is saved with supported extension (.html, .css, .js, .json, .yaml, .md)
**Large files slow?**
Files over 10MB may take longer. Consider splitting into smaller chunks
**Need help?**
Check [Issues](https://github.com/OffensiveEdge/urls-le/issues) or enable logging: `urls-le.telemetryEnabled: true`
## ❓ FAQ
**What URLs are extracted?**
HTTP/HTTPS, FTP, mailto, tel, file URLs (excludes `data:` and `javascript:` pseudo-protocols)
**Can I deduplicate?**
Yes, enable `urls-le.dedupeEnabled: true` to remove duplicates automatically
**Max file size?**
Up to 30MB. Practical limit: 10MB for optimal performance
**Perfect for web projects?**
Absolutely! Audit API endpoints, asset references, and external links for broken URLs
## 📊 Testing
**347 unit tests** • **95% function coverage, 86% line coverage**
Powered by Vitest • Run with `bun run test:coverage`
### Core Principle
**No broken or failed tests are allowed in commits.** All tests must pass before code can be committed or merged.
### Test Suite Highlights
- **83 security tests** for URL injection & SSRF prevention
- **71 edge case tests** for extraction logic & performance
- **37 tests** for JavaScript/TypeScript URL extraction
- **34 tests** for content limits, cancellation, and error handling
- **Comprehensive coverage** of all file formats and protocols
For detailed testing guidelines, see [Testing Guidelines](docs/TESTING.md).
---
Copyright © 2025
@OffensiveEdge. All rights reserved.