{"id":21071261,"url":"https://github.com/yisuschrist/urls_organizer","last_synced_at":"2025-04-30T15:40:58.043Z","repository":{"id":159223960,"uuid":"634534280","full_name":"YisusChrist/urls_organizer","owner":"YisusChrist","description":"CLI program that helps to organize a list of URLs","archived":false,"fork":false,"pushed_at":"2025-01-31T16:42:38.000Z","size":162,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T17:38:22.551Z","etag":null,"topics":["duplicate-detection","python","script"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YisusChrist.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-30T12:53:26.000Z","updated_at":"2025-01-31T16:42:40.000Z","dependencies_parsed_at":"2024-06-23T20:25:22.612Z","dependency_job_id":"c589c64b-329b-42e4-a22c-e02e96520c8e","html_url":"https://github.com/YisusChrist/urls_organizer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YisusChrist%2Furls_organizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YisusChrist%2Furls_organizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YisusChrist%2Furls_organizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YisusChrist%2Furls_organizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YisusChrist","download_url":"https://codeload.github.com/YisusChrist/urls_organizer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243514210,"owners_count":20303082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["duplicate-detection","python","script"],"created_at":"2024-11-19T18:51:07.562Z","updated_at":"2025-03-14T02:43:31.758Z","avatar_url":"https://github.com/YisusChrist.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg width=\"750\" src=\"https://i.imgur.com/Scbk7tO.png\" alt=\"Urls_organizer\"\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/yisuschrist/urls_organizer/issues\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/issues/yisuschrist/urls_organizer?color=171b20\u0026label=Issues%20%20\u0026logo=gnubash\u0026labelColor=e05f65\u0026logoColor=ffffff\"\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/yisuschrist/urls_organizer/forks\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/forks/yisuschrist/urls_organizer?color=171b20\u0026label=Forks%20%20\u0026logo=git\u0026labelColor=f1cf8a\u0026logoColor=ffffff\"\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/yisuschrist/urls_organizer/stargazers\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/stars/yisuschrist/urls_organizer?color=171b20\u0026label=Stargazers\u0026logo=octicon-star\u0026labelColor=70a5eb\"\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/yisuschrist/urls_organizer/actions\"\u003e\n        \u003cimg alt=\"Tests Passing\" src=\"https://github.com/yisuschrist/urls_organizer/actions/workflows/github-code-scanning/codeql/badge.svg\"\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/yisuschrist/urls_organizer/pulls\"\u003e\n        \u003cimg alt=\"GitHub pull requests\" src=\"https://img.shields.io/github/issues-pr/yisuschrist/urls_organizer?color=0088ff\"\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003c/a\u003e\n    \u003ca href=\"https://opensource.org/license/GPL-3.0/\"\u003e\n        \u003cimg alt=\"License\" src=\"https://img.shields.io/github/license/yisuschrist/urls_organizer?color=0088ff\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/YisusChrist/urls_organizer/issues/new?assignees=YisusChrist\u0026labels=bug\u0026projects=\u0026template=bug_report.yml\"\u003eReport Bug\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/YisusChrist/urls_organizer/issues/new?assignees=YisusChrist\u0026labels=feature\u0026projects=\u0026template=feature_request.yml\"\u003eRequest Feature\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/YisusChrist/urls_organizer/issues/new?assignees=YisusChrist\u0026labels=question\u0026projects=\u0026template=question.yml\"\u003eAsk Question\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/YisusChrist/urls_organizer/security/policy#reporting-a-vulnerability\"\u003eReport security bug\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cbr\u003e\n\n![Alt](https://repobeats.axiom.co/api/embed/c3525653303a46af833c14fd7587342852c3d494.svg \"Repobeats analytics image\")\n\n\u003cbr\u003e\n\n`urls_organizer` is a Python program that helps to organize a list of URLs. It can read URLs from a file or add a single URL, removes duplicate URLs, and sorts them in natural order.\n\nAdditionally, the program can validate the URLs by sending a GET request and checking for the ones that are not available anymore. The program uses **multiprocessing to speed up the validation process**, and the results are saved in a separate file. The output URL list can be saved in a file specified by the user. The program takes command-line arguments for its inputs and parameters.\n\nThe purpose of the program is to assist users in managing their lists of URLs by keeping them organized and validating their accuracy.\n\n\u003cbr\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTable of Contents\u003c/summary\u003e\n\n- [Requirements](#requirements)\n- [Installation](#installation)\n  - [Direct installation](#direct-installation)\n  - [From Pypi](#from-pypi)\n  - [Manual installation](#manual-installation)\n- [Execution](#execution)\n  - [Example of execution](#example-of-execution)\n- [Contributors](#contributors)\n  - [How do I contribute to Urls_organizer?](#how-do-i-contribute-to-urls_organizer)\n- [TODO](#todo)\n- [License](#license)\n- [Credits](#credits)\n\n\u003c/details\u003e\n\n## Requirements\n\nHere's a breakdown of the packages needed and their versions:\n\n- [poetry](https://pypi.org/project/poetry) \u003e= 1.7.1 (_only for manual installation_)\n- [natsort](https://pypi.org/project/natsort) \u003e= 8.4.0\n- [platformdirs](https://pypi.org/project/platformdirs) \u003e= 3.11.0\n- [pyfiglet](https://pypi.org/project/pyfiglet) \u003e= 1.0.2\n- [requests](https://pypi.org/project/requests) \u003e= 2.31.0\n- [rich](https://pypi.org/project/rich) \u003e= 13.5.2\n- [rich-argparse-plus](https://pypi.org/project/rich-argparse-plus) \u003e= 0.3.1.4\n- [tqdm](https://pypi.org/project/tqdm) \u003e= 4.66.1\n- [validators](https://pypi.org/project/validators) \u003e= 0.22.0\n\n\u003e [!NOTE]\n\u003e The software has been developed and tested using Python `3.10.10`. The minimum required version to run the software is Python 3.6. Although the software may work with previous versions, it is not guaranteed.\n\n## Installation\n\n### Direct installation\n\n`urls_organizer` can be installed by running one of the following commands in your terminal. You can install this via the command-line with either `curl`, `wget` or another similar tool.\n\n| Method    | Command                                                                                              |\n| :-------- | :--------------------------------------------------------------------------------------------------- |\n| **curl**  | `sh -c \"$(curl -fsSL https://raw.githubusercontent.com/yisuschrist/urls_organizer/main/install.sh)\"` |\n| **wget**  | `sh -c \"$(wget -O- https://raw.githubusercontent.com/yisuschrist/urls_organizer/main/install.sh)\"`   |\n| **fetch** | `sh -c \"$(fetch -o - https://raw.githubusercontent.com/yisuschrist/urls_organizer/main/install.sh)\"` |\n\n### From Pypi\n\n`urls_organizer` can be installed easily as a PyPI package. Just run the following command:\n\n```bash\npip3 install urls_organizer\n```\n\n\u003e [!IMPORTANT]\n\u003e For best practices and to avoid potential conflicts with your global Python environment, it is strongly recommended to install this program within a virtual environment. Avoid using the --user option for global installations. We highly recommend using [pipx](https://pypi.org/project/pipx) for a safe and isolated installation experience. Therefore, the appropriate command to install `urls_organizer` would be:\n\u003e\n\u003e ```bash\n\u003e pipx install urls_organizer\n\u003e ```\n\n### Manual installation\n\nIf you prefer to install the program manually, follow these steps:\n\n\u003e [!WARNING]\n\u003e This will install the version from the latest commit, not the latest release.\n\n1. Download the latest version of [urls_organizer](https://github.com/yisuschrist/urls_organizer) from this repository:\n\n   ```bash\n   git clone https://github.com/yisuschrist/urls_organizer\n   cd urls_organizer\n   ```\n\n2. Install the dependencies:\n\n   ```bash\n   poetry install --only main\n   ```\n\n3. Run the following commands to install urls_organizer in your `/usr/bin/` directory:\n\n   ```bash\n   sudo chmod +x urls_organizer\n   sudo cp urls_organizer /usr/bin/\n   ```\n\n## Execution\n\nTo run the `urls_organizer` script, you can use the following command:\n\n```bash\nurls_organizer [OPTIONS]\n```\n\nwhere `[OPTIONS]` are the command line options described below:\n\n![options](https://i.imgur.com/yZAnJky.png)\n\n#### Example of execution\n\nContent of _`urls.txt`_:\n\n```txt\nhttps://www.youtube.com\nhttp://www.google.com\nhttps://www.facebook.coma\nhttps://www.youtube.com\n```\n\nExecute the program with the following command:\n\n```bash\nurls_organizer -rf urls.txt -sf urls_organized.txt -w 10\n```\n\nContent of _`urls_organized.txt`_:\n\n```txt\nhttps://google.com\nhttps://youtube.com\n```\n\nContent of _`invalid_urls.txt`_:\n\n```txt\nhttps://www.facebook.coma\n```\n\n## Contributors\n\n\u003ca href=\"https://github.com/yisuschrist/urls_organizer/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=yisuschrist/urls_organizer\" /\u003e\n\u003c/a\u003e\n\n### How do I contribute to Urls_organizer?\n\nBefore you participate in our delightful community, please read the [code of conduct](https://github.com/YisusChrist/.github/blob/main/CODE_OF_CONDUCT.md).\n\nI'm far from being an expert and suspect there are many ways to improve – if you have ideas on how to make the configuration easier to maintain (and faster), don't hesitate to fork and send pull requests!\n\nWe also need people to test out pull requests. So take a look through [the open issues](https://github.com/yisuschrist/urls_organizer/issues) and help where you can.\n\nSee [Contributing Guidelines](https://github.com/YisusChrist/.github/blob/main/CONTRIBUTING.md) for more details.\n\n## TODO\n\nPlaning to add the following features:\n\n- [ ] Add support for multiple input/output files\n- [ ] Add a full documentation in Wiki section\n- [ ] Add uninstall bash script\n- [ ] Add a Changelog / Release Notes\n\n## License\n\n`urls_organizer` is released under the [GPL-3.0 License](https://opensource.org/license/GPL-3.0)\n\n## Credits\n\n\u003cimg src=\"https://avatars.githubusercontent.com/u/31022056?v=4\" width=\"100px;\" alt=\"Marvin Wendt\" border-radius=\"50% !important;\" /\u003e\n\nThanks to [Marvin Wendt](https://github.com/MarvinJWendt) for the installation script template. Original code of the **Instl** project can be found [here](https://github.com/installer/instl).\n\nI made a few modifications to the original script to fit my needs (error control, fetching of repository release content and installation of required dependencies) because I found problems when running the autogenerated script.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyisuschrist%2Furls_organizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyisuschrist%2Furls_organizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyisuschrist%2Furls_organizer/lists"}