{"id":28520018,"url":"https://github.com/brunorns/webscrapingdjango","last_synced_at":"2025-07-05T13:31:29.676Z","repository":{"id":297995512,"uuid":"998510196","full_name":"BrunoRNS/webScrapingDjango","owner":"BrunoRNS","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-28T15:55:19.000Z","size":2482,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-28T16:46:37.122Z","etag":null,"topics":["django","django-application","django-project","webscraper","webscraping","webscrapper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BrunoRNS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-08T19:01:24.000Z","updated_at":"2025-06-28T15:55:22.000Z","dependencies_parsed_at":"2025-06-28T16:34:07.322Z","dependency_job_id":"bc9885db-06ed-4e2b-8199-f8b50ee37110","html_url":"https://github.com/BrunoRNS/webScrapingDjango","commit_stats":null,"previous_names":["brunorns/webscrapingdjango"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/BrunoRNS/webScrapingDjango","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoRNS%2FwebScrapingDjango","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoRNS%2FwebScrapingDjango/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoRNS%2FwebScrapingDjango/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoRNS%2FwebScrapingDjango/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BrunoRNS","download_url":"https://codeload.github.com/BrunoRNS/webScrapingDjango/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoRNS%2FwebScrapingDjango/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263749810,"owners_count":23505454,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","django-application","django-project","webscraper","webscraping","webscrapper"],"created_at":"2025-06-09T06:38:03.100Z","updated_at":"2025-07-05T13:31:29.671Z","avatar_url":"https://github.com/BrunoRNS.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SPA Static Downloader (webScrapingDjango)\n\n**SPA Static Downloader** is a Django web application that lets you generate static, downloadable ZIP archives of JavaScript-heavy Single Page Applications (SPAs) like those built with React, Vue, Angular, or Svelte. It uses Playwright to render the site in a real browser (Firefox), downloads all assets (HTML, CSS, JS, images, fonts), and packages everything for easy hosting on platforms like Netlify or GitHub Pages.\n\n---\n\n## Use it Online\n\n### Usage\n\n**To access this website hosted with _pythonanywhere_ visit [SPA Static Downloader](.)**\n\n### Security\n\n\u003e Note: hosted with extremely security, configured using reCAPTCHA, csrf_token, ratelimit, environment variables and much more security tools.\n\n---\n\n## 🚀 Features\n\n- **SPA to Static**: Converts any public SPA URL into a static site.\n- **Full Asset Download**: Grabs HTML, CSS, JS, images, and fonts.\n- **Browser Rendering**: Uses Playwright with Firefox for accurate JS rendering.\n- **reCAPTCHA Protection**: Prevents abuse with Google reCAPTCHA.\n- **Rate Limiting**: Protects endpoints from excessive requests.\n- **Ready-to-Host ZIP**: Download a ZIP file ready for deployment.\n- **Modern UI**: Simple, responsive interface with dark/light mode.\n\n---\n\n## 🛠️ Quick Start\n\n1. **Clone the repository:**\n\n   ```bash\n   git clone https://github.com/BrunoRNS/webScrapingDjango.git\n   cd webScrapingDjango\n   ```\n\n2. **Run the setup script:**\n\n   ```bash\n   bash setup.sh\n   ```\n\n3. **Configure environment variables:**\n\n   - Edit `.env` (generated from `.env-example`) with your Django secret key and reCAPTCHA keys.\n\n4. **Start the server:**\n\n   ```bash\n   source venv/bin/activate\n   python manage.py runserver\n   ```\n\n5. **Open your browser:**  \n   Visit [http://localhost:8000](http://localhost:8000)\n\n---\n\n## 📝 Usage\n\n1. Enter the URL of the SPA site you want to download.\n2. Complete the reCAPTCHA.\n3. Click **Generate ZIP**.\n4. Download your static site archive!\n\n---\n\n## ⚙️ Tech Stack\n\n- **Backend:** Django 5, Playwright, BeautifulSoup, Requests\n- **Frontend:** HTML, CSS, JavaScript\n- **Security:** Google reCAPTCHA, django-ratelimit\n\n---\n\n## 📄 License\n\nThis project is licensed under the [GPLv3 License](https://www.gnu.org/licenses/gpl-3.0.html).\n\n---\n\n## 🙋‍♂️ Author\n\nMade by [BrunoRNS](https://github.com/BrunoRNS)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrunorns%2Fwebscrapingdjango","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrunorns%2Fwebscrapingdjango","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrunorns%2Fwebscrapingdjango/lists"}