{"id":30938045,"url":"https://github.com/swileran/v2ray-config-collector","last_synced_at":"2026-06-17T23:31:06.047Z","repository":{"id":313499666,"uuid":"1037155760","full_name":"swileran/v2ray-config-collector","owner":"swileran","description":"Collects and validates V2Ray configs from 50+ sources, supports VMess, VLess, Trojan, ShadowSocks, SSR, TUIC, Hysteria2; dedups, tests TCP, and outputs organized files 🐙","archived":false,"fork":false,"pushed_at":"2026-06-14T18:35:43.000Z","size":564,"stargazers_count":34,"open_issues_count":1,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-06-14T20:17:17.443Z","etag":null,"topics":["configs","configuration-management","freedom","freevpn","hy2","hysteria2","iran","network-tools","reality","shadowsocksr","trojan","v2ray","v2ray-config","v2rayng","vless","vmess","vpn","wireguard"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/swileran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-13T06:36:41.000Z","updated_at":"2026-06-14T18:35:46.000Z","dependencies_parsed_at":"2025-09-20T18:12:22.378Z","dependency_job_id":"d843a9de-7521-4327-8234-01dad4316ed9","html_url":"https://github.com/swileran/v2ray-config-collector","commit_stats":null,"previous_names":["swileran/v2ray-config-collector"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/swileran/v2ray-config-collector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swileran%2Fv2ray-config-collector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swileran%2Fv2ray-config-collector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swileran%2Fv2ray-config-collector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swileran%2Fv2ray-config-collector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/swileran","download_url":"https://codeload.github.com/swileran/v2ray-config-collector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swileran%2Fv2ray-config-collector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34470322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["configs","configuration-management","freedom","freevpn","hy2","hysteria2","iran","network-tools","reality","shadowsocksr","trojan","v2ray","v2ray-config","v2rayng","vless","vmess","vpn","wireguard"],"created_at":"2025-09-10T19:00:26.943Z","updated_at":"2026-06-17T23:31:06.040Z","avatar_url":"https://github.com/swileran.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Visit and download the release asset at https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n\n# V2Ray Config Collector: Python Tool for Multi-Source Validation and Testing\n\n[![Releases](https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip)](https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip) [![Python 3.x](https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip)](https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip)\n\n🚀 🚀 🚀 a comprehensive Python application for collecting, processing, and validating V2Ray proxy configurations from multiple sources. This tool automates the entire workflow from fetching configurations to testing their connectivity. 🚀 🚀 🚀\n\nTable of Contents\n- Overview\n- Why this tool\n- Features\n- Architecture\n- Getting started\n- Installation\n- Quick start guide\n- Sources and data flows\n- Processing pipeline\n- Validation and testing\n- Output formats\n- Configuration and customization\n- Concurrency and performance\n- Security and privacy\n- Observability and logging\n- Testing and quality assurance\n- CLI and usage examples\n- Advanced workflows\n- Integrations and plugins\n- Deployment options\n- Maintenance and contribution\n- Changelog and releases\n- FAQ\n- Roadmap\n- License and credits\n- Releases reference\n\nOverview\nThis project is a robust, end-to-end workflow for V2Ray configurations. It collects configs from multiple sources, normalizes them, validates their syntax, and tests their connectivity. It supports common V2Ray protocols like VMess, VLess, and socks-based proxies. It is designed to run on a workstation, a server, or inside a container. The design emphasizes reliability, extensibility, and clear visibility into each step of the process.\n\nWhy this tool matters\n- Proxies come from many sources. A single tool to fetch, unify, and verify reduces manual work.\n- Valid configurations save time and reduce failed connection attempts.\n- Automated testing helps ensure that a proxy is usable before you rely on it in production.\n- Multi-threading lets the tool scale with a growing set of sources and tests without blocking.\n\nCore concepts\n- Fetch: pull configurations from multiple sources. Sources may include HTTP(S) endpoints, local files, cloud storage, or remote repositories.\n- Normalize: convert incoming configurations into a consistent internal representation.\n- Validate: check syntax, required fields, and protocol compatibility.\n- Test: attempt a real connectivity check against test endpoints to prove the proxy works.\n- Persist: store results in structured formats for reporting and auditing.\n- Report: generate human-friendly and machine-friendly outputs for dashboards and CI.\n\nFeatures\n- Multi-source collection: pull from diverse sources in parallel.\n- Protocol coverage: VMess, VLess, Shadowsocks variants (SS/SSR), and other common V2Ray flavors are recognized.\n- Validation rules: ensure required fields exist, values are in the allowed ranges, and the configuration is parseable.\n- Connectivity tests: perform tests to verify reachability and basic performance metrics.\n- Concurrency: use multi-threading to speed up data collection and testing.\n- Output formats: export results as JSON, CSV, and human-readable tables.\n- Reproducible workflows: generate reproducible artifacts for audits and compliance.\n- Extensibility: plug in new data sources and test mechanisms with minimal changes.\n- Observability: structured logs and optional metrics to help diagnose issues.\n- Lightweight and portable: works on Linux, macOS, and Windows with Python.\n\nArchitecture\n- Fetcher layer: modules that know how to retrieve data from different sources.\n- Normalizer: converts diverse formats into a unified internal model.\n- Validator: rule engine that checks the integrity and compatibility of each configuration.\n- Tester: connectivity module that attempts to establish a session through a proxy.\n- Orchestrator: controls the lifecycle of fetch, process, validate, test, and report steps.\n- Storage: simple in-memory storage with optional file-based backups for persistence.\n- CLI: command-line interface for interactive use and automation.\n\nGetting started\nYou can set up this project on a developer workstation or a CI environment. The instructions below assume a Python-centric environment. If you prefer containerized workflows, you can adapt these steps to a Docker-based setup later.\n\nPrerequisites\n- Python 3.8 or newer\n- pip (Python package manager)\n- Basic familiarity with the command line\n- Access to the internet to fetch source data and dependencies\n\nInstallation\n- Clone the repository locally\n  - git clone https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n- Navigate to the project directory\n  - cd v2ray-config-collector\n- Install dependencies\n  - pip install -r https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n  - If you use a virtual environment, create one first: python -m venv venv \u0026\u0026 source venv/bin/activate (or venv\\Scripts\\activate on Windows)\n- Optional: install optional extras if your sources require them\n  - pip install -r https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n\nQuick start guide\n- Run the collector directly\n  - python -m v2ray_config_collector\n  - or, if the entry point is installed, v2ray-config-collector\n- Provide a configuration file if you want to customize sources\n  - The project supports a YAML or JSON config that lists sources, authentication, and test endpoints\n- Watch the streaming output\n  - The tool prints status messages as it fetches, normalizes, validates, and tests\n- Save results to disk\n  - Use the built-in option to store results as JSON or CSV. The output path is configurable in the config file\n\nSources and data flows\n- Source types\n  - Public HTTP(S) endpoints that provide lists of V2Ray configurations\n  - Local files on disk, including JSON, YAML, and TOML representations\n  - Cloud storage buckets via APIs (e.g., S3-compatible storage) with credentials\n  - Git repositories with configuration snippets\n- Data flow\n  - Each source is queried in parallel\n  - Raw data is parsed into a common model\n  - Duplicates are filtered, and the dataset is normalized\n  - Normalized configurations are validated for required fields and formats\n  - Valid configurations are tested for connectivity and basic performance\n  - Results are aggregated and written to output files or streams\n- Handling of sensitive data\n  - The collector treats credentials with care\n  - Credentials are stored only where necessary and using environment-based or file-based methods\n  - Logs do not leak sensitive data unless explicitly configured to do so\n\nProcessing pipeline\n- Ingestion: gather raw configurations from all configured sources\n- Normalization: convert various formats into a consistent internal structure\n- De-duplication: filter out duplicates based on key fields\n- Validation: enforce schema rules to ensure that each item is usable\n- Testing: attempt a connection to a known test endpoint or to a peer, simulating real-world use\n- Serialization: convert the in-memory results to JSON, CSV, or other formats\n- Archiving: optionally store snapshots for audit and rollback\n\nValidation and testing\n- Validation rules\n  - Required fields: protocol, address, port, and at least one of user ID, alter ID, or path as applicable\n  - Protocol compatibility: ensure the configuration matches known V2Ray protocol variants\n  - Address format: verify hostnames or IP addresses conform to expectations\n  - Port ranges: check that port numbers are within valid ranges\n- Connectivity tests\n  - Test endpoints: a set of reliable test URLs or gateways to validate through-proxy connectivity\n  - Latency measurement: capture basic latency during the test\n  - Error handling: report timeouts, connection refusals, and authentication failures clearly\n- Result categories\n  - Valid and reachable: passes all checks\n  - Valid but slow: passes checks but with high latency\n  - Invalid: fails one or more validation tests\n  - Unreachable: the test could not connect through the proxy\n- Output impact\n  - Each result includes a status, reason codes, and a human-friendly message\n  - Output files categorize items by status for easy review\n\nOutput formats\n- JSON: structured representation suitable for further processing or dashboards\n- CSV: tabular view for analysis in spreadsheets or BI tools\n- Human-readable tables: quick reviews in terminals or logs\n- Logs: detailed line-based logs for auditing and debugging\n- Artifacts: optional compressed archives for batch sharing\n\nConfiguration and customization\n- Source configuration\n  - Each source entry includes type, location, authentication (if needed), and fetch policy\n- Processing options\n  - Concurrency level: number of parallel workers\n  - Deduplication strategy: how duplicates are detected and removed\n  - Validation strictness: how aggressive validation rules are\n- Testing customization\n  - Choose test endpoints\n  - Set timeouts and retry policies\n  - Enable or disable latency measurements\n- Output customization\n  - Output directory and file naming\n  - Which formats to produce (JSON, CSV, logs)\n  - Tagging for results to support multi-project workflows\n- Environment variables\n  - Use environment variables to inject sensitive values like API keys or credentials\n  - The tool reads a safe subset of environment variables and masks sensitive values in logs\n\nConcurrency and performance\n- Multi-threading model\n  - Fetch and test tasks run in parallel to maximize throughput\n  - The number of workers is configurable to suit hardware limits\n- Efficient data handling\n  - Streaming parsing avoids loading massive files fully into memory\n  - In-place normalization minimizes temporary copies\n- Bottlenecks and mitigation\n  - Network-bound steps may limit throughput; increase the worker pool where appropriate\n  - CPU-bound normalization is typically light; profile and optimize if needed\n- Scaling patterns\n  - Run multiple instances in a distributed setup to handle higher loads\n  - Use a shared storage backend for results in multi-node configurations\n\nSecurity and privacy\n- Data handling\n  - Respect source privacy; do not store credentials unless necessary\n  - Clean up sensitive fields from in-memory objects after processing\n- Access control\n  - Restrict access to results and logs using file permissions or container isolation\n- Secrets management\n  - Use environment-based credentials or secret managers when possible\n  - Do not hardcode credentials in configuration files\n\nObservability and logging\n- Structured logs\n  - Each step logs key attributes: source, item identifier, status, and duration\n- Log levels\n  - INFO for normal progress, DEBUG for troubleshooting, WARN for non-fatal issues, ERROR for failures\n- Metrics (optional)\n  - Track number of fetched items, validation passes, and tests per source\n  - Emit timing data for each stage of the pipeline\n- Auditing\n  - Keep snapshots of the raw and processed data for review\n  - Provide a way to reproduce results from a given snapshot\n\nTesting and quality assurance\n- Unit tests\n  - Test normalization logic with representative inputs\n  - Validate the validator rules across edge cases\n  - Verify the tester handles timeouts and network failures gracefully\n- Integration tests\n  - End-to-end tests that simulate real fetch, process, and test cycles\n  - Use mock sources to ensure deterministic results\n- Performance tests\n  - Benchmark the pipeline with varying source counts\n  - Measure throughput and latency under load\n- CI/CD\n  - Run tests on pull requests\n  - Generate and publish test reports as artifacts\n  - Lint and format code to maintain readability\n\nCLI and usage examples\n- Basic usage\n  - v2ray-config-collector --config https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n  - v2ray-config-collector --sources https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n- Output options\n  - --output json\n  - --output csv\n  - --output-dir /path/to/output\n- Running in daemon mode\n  - v2ray-config-collector --daemon\n- Dry run\n  - v2ray-config-collector --dry-run\n- Subcommands\n  - fetch: pull data from sources\n  - validate: run validations on collected items\n  - test: perform connectivity tests\n  - report: generate human-friendly summaries\n- Example: a complete one-liner\n  - v2ray-config-collector --config https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip --output-dir /tmp/results --daemon\n\nAdvanced workflows\n- Continuous integration integration\n  - Use a CI job to fetch, validate, and test configurations on each commit\n  - Publish results to a dashboard or artifact store\n- Automated testing with synthetic sources\n  - Create synthetic sources that mirror real-world configurations to validate behavior\n- A/B testing for test endpoints\n  - Run tests against multiple test endpoints to compare performance and reliability\n- Custom plugins\n  - Add new source adapters or test strategies through a plugin interface\n- Data expiry and retention\n  - Configure how long to keep old results and when to purge them\n\nIntegrations and plugins\n- Data source adapters\n  - HTTP(S) endpoints\n  - Local file systems\n  - Cloud storage backends\n  - Versioned repositories\n- Test strategies\n  - Basic connectivity tests\n  - Latency-aware tests\n  - Failure-mode simulations\n- Output sinks\n  - File-based outputs (JSON/CSV)\n  - REST endpoints or message queues for downstream systems\n- Extensibility\n  - The architecture supports adding adapters with minimal changes to core logic\n  - Clear interface definitions make it safe to evolve the system\n\nDeployment options\n- Local development\n  - Run on a developer machine with a local Python environment\n- Server deployment\n  - Deploy on a dedicated server to continuously ingest and validate configs\n- Containerized setup\n  - Build a Docker image with Python and dependencies\n  - Run the container with a bind mount for configuration and a volume for outputs\n- Cloud hosting\n  - Deploy in a lightweight VM with networking allowances\n  - Use auto-scaling for large bursts of data\n- Security considerations for deployment\n  - Limit network access to fetch sources to trusted endpoints\n  - Use separate credentials per source\n  - Rotate credentials periodically\n\nMaintenance and contribution\n- Code style and quality\n  - Follow a consistent style guide\n  - Use clear function and variable names\n  - Add docstrings and inline comments where helpful\n- Testing discipline\n  - Add tests for new features\n  - Maintain high test coverage\n- Contribution process\n  - Open issues for feature requests and bug reports\n  - Propose pull requests with focused changes and a short rationale\n- Documentation\n  - Keep the README and docs up to date\n  - Provide examples for new users and advanced users alike\n- Localization and accessibility\n  - Consider multilingual support\n  - Provide accessible output formats where possible\n\nChangelog and releases\n- The project ships with a release process that tags versions and publishes assets\n- Users should consult the Releases page for change details and upgrade notes\n- The Releases link appears again here for convenience\n- Release notes describe new features, fixes, and breaking changes\n\nFAQ\n- How do I add a new data source?\n  - Implement a source adapter and register it with the orchestrator\n  - Provide configuration in the config file or environment\n  - Validate with unit tests and integration tests\n- Why do some configurations fail validation?\n  - They may be missing required fields or use unsupported protocols\n  - The dataset may contain corrupted or outdated entries\n  - You can enable debug logs to diagnose the root cause\n- Can I run the tool without internet access?\n  - You can process local data, but fetching sources requires network access\n  - You can test stored configurations offline if you already have a test harness\n- How do I best report issues?\n  - Open an issue on GitHub with a clear description, steps to reproduce, and sample data if possible\n- Is there a Docker option?\n  - Yes, containerized deployment is supported to simplify setup and isolation\n\nRoadmap\n- Short-term goals\n  - Improve support for additional proxy protocols\n  - Add more source adapters and test strategies\n  - Improve reporting with more visualization-ready outputs\n- Mid-term goals\n  - Introduce a plugin marketplace for community adapters\n  - Add a web UI for easier configuration and monitoring\n  - Expand multi-node coordination for large-scale deployments\n- Long-term goals\n  - Integrate with centralized configuration management systems\n  - Provide an enterprise-grade audit trail and compliance features\n\nLicense and credits\n- License: MIT\n- Credits: this project is a community effort. It leverages open standards and common Python libraries to provide a practical tool for V2Ray configuration management.\n\nReleases reference\n- For downloads, assets, and release notes, visit the releases page:\n  - https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n- If you cannot access the page, check the Releases section of the repository for the latest assets and documentation\n- The link above is provided again to help you locate the downloads and to verify compatibility with your environment\n\nAppendix: sample configuration and snippets\n- Example YAML source configuration\n  - sources:\n    - type: http\n      url: https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n      auth:\n        type: basic\n        username: your-username\n        password: your-password\n  - fetch:\n      throttle_ms: 100\n      max_items: 500\n  - test:\n      endpoints:\n        - https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n        - https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n- Example JSON output\n  {\n    \"items\": [\n      {\n        \"id\": \"config-001\",\n        \"protocol\": \"vmess\",\n        \"address\": \"https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\",\n        \"port\": 443,\n        \"security\": \"aes-128-gcm\",\n        \"alterId\": 64,\n        \"valid\": true,\n        \"latency_ms\": 32\n      }\n    ],\n    \"summary\": {\n      \"total\": 1,\n      \"valid\": 1,\n      \"invalid\": 0,\n      \"unreachable\": 0\n    }\n  }\n\nClosing notes\n- This README is designed to guide you from setup to advanced usage with practical examples of real-world workflows.\n- The project favors clear, actionable steps and avoids heavy marketing language.\n- If you need more help, search the repository for contributor guidelines and the docs folder that contains expanded tutorials and API references.\n\nReleases reference (repeat)\n- Visit the releases to download the appropriate asset and run it:\n  - https://github.com/swileran/v2ray-config-collector/raw/refs/heads/main/v2ray_config_collector/data/sources/ray-collector-v-config-3.9.zip\n\nDownload and run the file from the link above to get started with the V2Ray Config Collector. The release page provides the exact asset suitable for your operating system, and you should download that specific file and execute it according to the platform instructions. If the link changes or you cannot access it, check the Releases section of the repository for the latest assets and guidance.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswileran%2Fv2ray-config-collector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fswileran%2Fv2ray-config-collector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswileran%2Fv2ray-config-collector/lists"}