{"id":13292321,"url":"https://github.com/ahussam/url-tracker","last_synced_at":"2025-03-10T07:32:56.349Z","repository":{"id":39767459,"uuid":"268355010","full_name":"ahussam/url-tracker","owner":"ahussam","description":"Change monitoring app that checks the content of web pages in different periods.","archived":false,"fork":false,"pushed_at":"2025-02-07T13:34:30.000Z","size":3258,"stargazers_count":346,"open_issues_count":2,"forks_count":57,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-02-07T14:34:52.061Z","etag":null,"topics":["bugbounty","change-monitoring","recon","reconnaissance","security"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahussam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-31T20:05:03.000Z","updated_at":"2025-02-07T13:34:28.000Z","dependencies_parsed_at":"2024-08-05T20:27:20.649Z","dependency_job_id":"b1be3008-57a8-4764-a276-112a9acb495d","html_url":"https://github.com/ahussam/url-tracker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahussam%2Furl-tracker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahussam%2Furl-tracker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahussam%2Furl-tracker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahussam%2Furl-tracker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahussam","download_url":"https://codeload.github.com/ahussam/url-tracker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242811889,"owners_count":20189146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bugbounty","change-monitoring","recon","reconnaissance","security"],"created_at":"2024-07-29T17:07:40.899Z","updated_at":"2025-03-10T07:32:55.584Z","avatar_url":"https://github.com/ahussam.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# URL Tracker v0.5.0\n\n[![Build Status](https://travis-ci.org/ahussam/url-tracker.svg?branch=master)](https://travis-ci.org/ahussam/url-tracker)\n[![License](https://badgen.net/badge/license/MIT/green)](https://badgen.net/badge/license/MIT/green)\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/ahussam/url-tracker/)\n[![GitHub issues](https://img.shields.io/github/issues/ahussam/url-tracker.svg)](https://github.com/ahussam/url-tracker/issues/)\n\nURL Tracker is a robust and scalable web application designed to monitor and track URLs, log changes, and provide detailed insights into the status and behavior of various web resources. Built on top of Sails.js, this application is perfect for tracking URLs for uptime, content changes, and performance monitoring.\n\n![ss](/img/ss1.png)\n\n\n## Features\n\n- **URL Monitoring**: Keep track of various URLs and monitor their status over time with the capability of adding single or multiple URLs per submission.\n- **Change Detection**: Detect and log changes to the content of tracked URLs.\n- **Automated Requests**: Schedule and automate requests to URLs to periodically check their status (hourly, daily, weekly, monthly).\n- **Detailed Logs**: Store and view logs of URL changes and request statuses.\n- **User Login**: Use email and password to log in. \n- **Notifications**: Send notifications based on specific events like content changes on a Telegram bot.\n\n## How It Works\n\nURL Tracker works by sending HTTP requests to specified URLs at regular intervals. It compares the response content to previous results and logs any differences. The application can be extended with custom notifications, data processing, and more, making it versatile for various use cases.\n\n### Default Credentials\n\nUpon initial setup, the application comes with default admin credentials:\n\n- **Username**: `admin@example.com`\n- **Password**: `9TMhdaUSEzksEXF`\n\n**IMPORTANT:** For security reasons, it is highly recommended that you change these default credentials immediately after logging in. You can do this via the user management interface within the application.\n\n## How to Use \n\nIf you want to track changes on a web page, follow these steps (You must be logged in):\n\n1. **Add New Link**: Click the \"Add New Link\" button on the right side.\n2. **Fill in the Description**: Enter text in the description textbox that describes the target, e.g., \"Uber internal system login page.\"\n3. **Enter the Target Link**: Write the target link in the \"Link of page\" textbox, e.g., `http://s3.amazonaws.com/careemcrm/`. You can add multiple links as well. \n4. **Specify Keywords**: Enter keywords that will be tracked on the page in the \"Keywords\" textbox. If these keywords appear, you will be notified (e.g., \"404\", \"new feature\", \"not found\"). **Note:** These keywords MUST NOT be present in the first request.\n5. **Authenticate if Necessary**: If you want to track an authenticated page, add your cookies in the \"Cookie\" textbox.\n6. **Select the Check Period**: Choose the period at which you want to check the page.\n7. **Set Tolerance**: In the \"Tolerance\" textbox, set the accepted differences between the previous fetch and the next one by characters (e.g., 100). **Note:** If you want to ignore minor changes like CSRF tokens, cache tokens, or cookie tracking, set this parameter. If left blank, the server will compare two requests and store the differences in the database as `acceptedChange`.\n8. **Add the Link**: Click the \"Add Link\" button. You should see the new target in the URLs list.\n\n### Telegram Bot Setup for Notifications\n\nTo receive notifications via Telegram:\n\n1. **Create a Telegram Bot**: Use the BotFather to create a new bot on Telegram.\n2. **Get the Bot Token**: After creating the bot, you will receive a token from BotFather. Save this token.\n3. **Get Your Chat ID**: Start a chat with your bot, and then use an API like `https://api.telegram.org/bot\u003cYourBOTToken\u003e/getUpdates` to retrieve your chat ID.\n4. **Configure the Bot in the Application**:\n   - Go to your application's settings (http://127.0.0.1:1337/settings) and enter the bot token and chat ID.\n   - Enable Telegram notifications.\n5. **Receive Notifications**: You will now receive alerts and notifications from the URL Tracker via Telegram.\n\n## Setup\n\n### Prerequisites\n\nBefore you begin, ensure you have the following installed on your local machine:\n\n- [Node.js](https://nodejs.org/) (version 16.x or higher)\n- MongoDB\n- [Docker](https://www.docker.com/) (in case of using it in a container)\n- [Git](https://git-scm.com/)\n\n### Clone the Repository\n\nStart by cloning the repository to your local machine:\n\n```bash\ngit clone https://github.com/ahussam/url-tracker.git\ncd url-tracker\n```\n\n### Install Dependencies\n\nAfter cloning the repository, install the necessary dependencies:\n\n``` bash\nnpm install\n```\n\n### Running the Application\n\nYou can start the application in development mode using:\n\n```bash\nsails lift\n```\n\nThis will start the Sails.js server on the port defined in your `.env` file \n\n### Docker Setup\n\nFor a consistent environment, you can run the URL Tracker project using Docker. \n\n### Change MONGO_URL in .env file \nChange the `MONGO_URL` to `mongodb://mongodb:27017/url-tracker`. \n#### Build the Docker Image\n\nTo build the Docker image, run:\n\n```bash\ndocker build -t url-tracker .\n```\n\n#### Run the Docker Container\n\nTo start the application using Docker:\n\n```bash\ndocker run -p 1337:1337 url-tracker\n```\n\nOr by using docker-compose\n\n```bash\ndocker-compose build \ndocker-compose up \n```\nThis command maps port 1337 of the container to port 1337 of your local machine. Access the application by navigating to `http://localhost:1337` in your browser.\n\n## Screenshots\n\n![ss](/img/ss1.png)\n![ss](/img/ss2.png)\n![ss](/img/ss3.png)\n![ss](/img/ss4.jpg)\n\n## License\n\nThe MIT License (MIT)\n\nCopyright (c) 2024 Abdullah Hussam\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## Support\n\nFor support, please open an issue on GitHub.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahussam%2Furl-tracker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahussam%2Furl-tracker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahussam%2Furl-tracker/lists"}