{"id":29417839,"url":"https://github.com/thequick10/tracetoend","last_synced_at":"2026-04-09T17:35:12.863Z","repository":{"id":302776584,"uuid":"1013577986","full_name":"thequick10/TraceToEnd","owner":"thequick10","description":"A powerful and efficient URL resolution web app service that tracks campaign parameters, handles complex redirects, and validates marketing URLs.","archived":false,"fork":false,"pushed_at":"2025-08-13T05:36:10.000Z","size":298,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-13T07:14:12.928Z","etag":null,"topics":["api","bright-data","css","express","express-rate-limit","helmet","html","javascript","nodejs","proxies","puppeteer","puppeteer-core"],"latest_commit_sha":null,"homepage":"https://tracetoend.onrender.com/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thequick10.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-04T06:09:58.000Z","updated_at":"2025-08-13T05:36:14.000Z","dependencies_parsed_at":"2025-07-18T10:09:57.159Z","dependency_job_id":"a0b1f4d9-d6ba-46e4-855b-a83b94067fe7","html_url":"https://github.com/thequick10/TraceToEnd","commit_stats":null,"previous_names":["thequick10/tracetoend"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thequick10/TraceToEnd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thequick10%2FTraceToEnd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thequick10%2FTraceToEnd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thequick10%2FTraceToEnd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thequick10%2FTraceToEnd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thequick10","download_url":"https://codeload.github.com/thequick10/TraceToEnd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thequick10%2FTraceToEnd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006211,"owners_count":26084062,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","bright-data","css","express","express-rate-limit","helmet","html","javascript","nodejs","proxies","puppeteer","puppeteer-core"],"created_at":"2025-07-11T22:12:39.790Z","updated_at":"2025-10-11T04:54:46.901Z","avatar_url":"https://github.com/thequick10.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🎯 TraceToEnd\n\n\u003cdiv align=\"center\"\u003e\n\n![Node.js Version](https://img.shields.io/badge/node-%3E%3D%2014.0.0-green.svg)\n![Express Version](https://img.shields.io/badge/express-%5E4.18.2-lightgrey)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](http://makeapullrequest.com)\n\nA powerful and efficient URL resolution service that tracks campaign parameters, handles complex redirects, and validates marketing URLs.\n\n[Key Features](#-key-features) •\n[Installation](#%EF%B8%8F-installation) •\n[API Endpoints Usage](#-api-endpoints-usage) •\n[Deployment](#-deployment) •\n[Dependencies](#dependencies) •\n[Frontend Usage](#frontend-usage) •\n[Sample JSON Response](#sample-response) •\n[Import/Export Capabilities](#-importexport-capabilities)\n\n\u003c/div\u003e\n\n## 🚀 Key Features\n\n- 🌐 **Browser Emulation**: Uses Puppeteer for JavaScript-heavy redirects\n- 🌏 **Region Based URL Resolutions**: Region-based URL resolution with proxy support for multiple geographic zones\n- 🖇 **Multiple Endpoint Support**: Single and multiple region URL resolution endpoints\n- 📊 **Data Management**: Campaign management frontend with URL tracking, tagging, and status display\n- 📥 **File Import/Export Support**: Import and export campaigns via CSV and XLSX file formats\n- 📍 **Location Detection**: Auto-detection of user location and manual country selection\n- 🔐 **Security**: Rate limiting, security headers, and basic authentication for enhanced security\n- 📊 **API Usage Analytics**: Analytics page for usage tracking and monitoring\n- 🔗 **URL Resolution Stats** URL Resolutions stat page to track failed and success url in respect of regions\n- ✅ **Health Check Support**: System health and region listing API endpoints\n\n### 📁 Import/Export Capabilities\n- 📥 **Import Support**:\n  - CSV file import\n  - XLSX file import\n  - Drag \u0026 drop file support\n  - Smart column detection\n  - Batch processing\n- 📤 **Export Options**:\n  - Export to CSV\n  - Complete campaign history\n  - Formatted date and time\n\n### 🌍 Location Detection\n- 🔍 **Auto-Detection**: Automatic country detection\n- 🔄 **Multiple Services**: Fallback to multiple geolocation services\n- 🚥 **Status Indicators**: Visual feedback for detection process\n- 🔄 **Manual Refresh**: Option to refresh location detection\n\n### 🔍 Search \u0026 Filter\n- 🔎 **Real-time Search**: Instant search across all fields\n- 📅 **Date Range Filter**: \n  - Built-in date range picker\n  - Clear filter option\n  - Support for custom date formats\n- 📊 **Sorting Options**:\n  - Sort by newest/oldest\n  - Persistent sorting preferences\n  - Sort by File Import Order\n\n### 🔄 URL Refresh Features\n- 🔄 **Individual Refresh**: Refresh single URLs\n- 📊 **Batch Refresh**: Refresh all URLs with progress tracking\n- ⚠️ **Error Handling**: \n  - Automatic retry mechanism\n  - Error status indicators\n  - Restore previous URL on failure\n- 📈 **Progress Tracking**: Visual progress indicators\n\n## 🧑‍💻 User Agent Support\n- **Desktop**: Emulates a desktop browser user agent.\n- **Mobile**: Emulates a mobile browser user agent.\n- **Random (Rotating)**: Randomly selects a desktop or mobile user agent for each request. This is the default option and helps simulate diverse real-world traffic.\n\nYou can select the user agent type from the frontend dropdown. The selected type is sent to the backend and used for all URL resolutions and analytics.\n\n### 💫 Additional Features\n- 📋 **Clipboard Support**: One-click URL copying\n- ✏️ **Inline Editing**: Edit campaign URLs and tags directly\n- 🗑️ **Data Management**: Delete individual or all campaigns\n- 📱 **Responsive Design**: Works on desktop and mobile\n- 🔔 **Notifications**: Beautiful toast notifications for all actions\n- 🖥 **User Agent Selection**: Choose between Desktop, Mobile, or Random (rotating) user agents for each request. The Random option will select a new user agent for every request, simulating real-world browsing patterns.\n- ⏱️ **Timing Statistics Dashboard**: Dedicated dashboard to track and analyze the time taken to resolve URLs, with filtering, sorting, and CSV export capabilities.\n\n## 🛠️ Installation\n\n1. **Clone the repository**\n```bash\ngit clone https://github.com/thequick10/TraceToEnd.git\ncd TraceToEnd\n```\n\n2. **Install dependencies**\n```bash\nnpm install\n```\n\n3. **DotENV Config**\n```.env\nConfigure your dotenv file in your local server and add all variable values in dotenv\n\nFor Instance:\n\nBRIGHTDATA_API_KEY=\u003cYOUR_BRIGHTDATA_API_KEY\u003e\nBRIGHTDATA_US_PROXY=brd-customer-\u003cCUSTOMER_ID\u003e-zone-\u003cYOUR_ZONE_ID\u003e-country-\u003cCOUNTRY\u003e\n\nAdd all variables and their value just like above\n\nMake sure you use 2-letter country code like for united states use only - US\n```\n\n5. **Start the server**\n```bash\n# Development mode\nnpm run dev\n\n# Production mode\nnpm start\n```\n\n## 💻 API Endpoints Usage\n\nThe server provides multiple endpoints for URL resolution with different levels of depth and speed:\n\n- `GET /resolve?url=\u003cURL\u003e\u0026region=\u003cREGION_CODE\u003e`  \n  Resolve a single URL for a specified region.\n\n- `GET /resolve?url=\u003cURL\u003e\u0026region=\u003cREGION_CODE\u003e\u0026uaType=desktop|mobile`\n  Resolve a single URL for a specified region with a specific user agent type.\n\n- `GET /resolve-multiple?url=\u003cURL\u003e\u0026regions=us,ca,ae`  \n  Resolve a URL across multiple regions simultaneously.\n\n- `GET /resolve-multiple?url=\u003cURL\u003e\u0026regions=us,ca,ae\u0026uaTyp=desktop|mobile`\n  Resolve a URL across multiple regions simultaneously with a specific user agent type.\n\n- `GET /zone-usage?from=YYYY-MM-DD\u0026to=YYYY-MM-DD`  \n  Retrieve BrightData API usage statistics for the configured zone.\n\n- `GET /regions`  \n  List all supported proxy regions.\n\n- `GET /system-info`  \n  Get system health and resource usage information.\n\n- `GET /ip`  \n  Returns the client IP address.\n\n## Campaign link generator\n- **Dashboard:** Access the dashboard at `https://your-campaign-url.com/` (replace with your actual campaign URL).\n- **Add Campaigns:** Enter campaign URL, tags/notes, and select country, then click \"Add Campaign\".\n- **Refresh URLs:** Refresh individual or all campaign URLs to get updated final resolved URLs.\n- **Import/Export:** Import campaigns from CSV or XLSX files; export current campaigns to CSV.\n- **Filtering and Sorting:** Search campaigns, filter by date range, and sort by newest, oldest, or import order.\n- **Delete Campaigns:** Delete individual or all campaigns.\n- **Edit Campaigns:** Edit campaign details, including URL, tags, and notes.\n- **User Agent Selection:** Select user agent type (desktop or mobile) for each campaign.\n\n## API Request Analytics\n- Access the dashboard at `/analytics/stats.html`\n- View campaign statistics, including date, total requests and bandwidth etc.\n- Filter by date range \u0026 search requests by date.\n- Export data to CSV.\n\n## URL Resolution Stats\n- Access the dashboard at `/resolution-stats/resolutions.html`\n- View URL resolution statistics, including total requests, successful and failed requests, and bandwidth etc.\n- Filter by date range \u0026 search requests by date.\n- Export data to CSV.\n- View the stats in a table format.\n- View stats by regional performance \n\n## ⏱️ Timing Statistics Dashboard\n- Access the dashboard at `/timing-stat/timing-stat.html`.\n- View daily statistics for total time, average time per URL, and request counts.\n- Filter by date range, sort results, and export timing data as CSV.\n- The dashboard is fully responsive and mobile-friendly.\n\n### Sample Response\n```json\n{\n  \"originalUrl\": \"https://your-campaign-url.com\",\n  \"finalUrl\": \"https://final-destination.com?clickid=123\u0026utm_source=campaign\",\n  \"method\": \"browser-api\",\n  \"hasClickId\": true,\n  \"hasUtmSource\": true,\n  \"hasClickRef\": false\n  //more paramenters\n}\n```\n\n## 🚀 Deployment\n\n### Prerequisites\n- Node.js \u003e= 14.0.0\n- NPM or Yarn\n- 512MB RAM minimum\n\n### Environment Variables\n```env\nPORT=8080  # Server port (optional)\n```\n\n### Deployment Platforms\n- 🌐 Any Node.js compatible hosting\n\n## Dependencies\nKey npm packages used:\n\n- express\n- cors\n- dotenv\n- helmet\n- express-basic-auth\n- express-rate-limit\n- puppeteer-core\n- https\n- os\n- path\n- url\n\n## License and Author\n\nAuthor: Rupesh Shah  \nLicense: MIT (or specify your license)\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🙏 Acknowledgments\n\n- [Puppeteer](https://pptr.dev/) for headless browser automation\n- [Express.js](https://expressjs.com/) for the web framework\n- [node-fetch](https://github.com/node-fetch/node-fetch) for HTTP requests\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthequick10%2Ftracetoend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthequick10%2Ftracetoend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthequick10%2Ftracetoend/lists"}