{"id":49223127,"url":"https://github.com/hotaro26/wiki_web","last_synced_at":"2026-04-24T05:03:52.438Z","repository":{"id":351207113,"uuid":"1210020076","full_name":"Hotaro26/wiki_web","owner":"Hotaro26","description":"wiki content scrapper built with python ","archived":false,"fork":false,"pushed_at":"2026-04-14T02:39:02.000Z","size":15,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-14T04:21:21.487Z","etag":null,"topics":["python","scrapper","web"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Hotaro26.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-14T02:28:35.000Z","updated_at":"2026-04-14T02:39:58.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Hotaro26/wiki_web","commit_stats":null,"previous_names":["hotaro26/wiki_web"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Hotaro26/wiki_web","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hotaro26%2Fwiki_web","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hotaro26%2Fwiki_web/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hotaro26%2Fwiki_web/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hotaro26%2Fwiki_web/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Hotaro26","download_url":"https://codeload.github.com/Hotaro26/wiki_web/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hotaro26%2Fwiki_web/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32209897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T03:15:14.334Z","status":"ssl_error","status_checked_at":"2026-04-24T03:15:11.608Z","response_time":64,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","scrapper","web"],"created_at":"2026-04-24T05:03:43.892Z","updated_at":"2026-04-24T05:03:52.434Z","avatar_url":"https://github.com/Hotaro26.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Wiki Scraper \u0026 Web App\n\nA versatile Python project that offers multiple ways to fetch and save Wikipedia content, ranging from command-line scrapers to a full-featured web application.\n\n## 🚀 Features\n\n- **Standard Scraping:** Uses `BeautifulSoup4` and `requests` to parse Wikipedia pages directly.\n- **API Integration:** Uses the official Wikipedia API for cleaner and faster data retrieval.\n- **File Management:** Option to save fetched articles as `.txt` files for offline reading.\n- **Visual Enhancements:** A dedicated CLI version with colored terminal output for better readability.\n- **Web Interface:** A Flask-based web application to search and scrape Wikipedia from your browser.\n\n## 📁 Project Structure\n\n```text\nwiki_web/\n├── app.py                         # Flask web application\n├── main.py                        # BeautifulSoup4-based scraper\n├── wiki_api.py                    # Basic Wikipedia API scraper\n├── wiki_api_with_save.py          # API scraper with .txt save functionality\n├── wiki_api_with_save-colours.py  # API scraper with save functionality \u0026 colored UI\n├── requirements.txt               # Project dependencies\n├── templates/                     # HTML files for the web app\n└── venv/                          # Virtual environment\n```\n\n## 🛠️ Installation\n\n1. **Clone the repository:**\n   ```bash\n   git clone \u003cyour-repository-link\u003e\n   cd wiki_web\n   ```\n\n2. **Create and activate a virtual environment (optional but recommended):**\n   ```bash\n   python -m venv venv\n   # On Windows:\n   venv\\Scripts\\activate\n   # On macOS/Linux:\n   source venv/bin/activate\n   ```\n\n3. **Install dependencies:**\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n## 💻 Usage\n\n### 1. Web Application\nTo run the web interface on `localhost`:\n```bash\npython app.py\n```\nOpen your browser and navigate to `http://127.0.0.1:5000`.\n\n### 2. Command Line Tools\nYou can run any of the specific scrapers depending on your needs:\n\n- **Basic Scraper:** `python main.py`\n- **API Scraper:** `python wiki_api.py`\n- **Save to Text:** `python wiki_api_with_save.py`\n- **Colored UI Scraper:** `python wiki_api_with_save-colours.py`\n\n## 📦 Dependencies\nThis project uses several Python libraries:\n- `requests`: To handle HTTP requests.\n- `BeautifulSoup4`: For parsing HTML content.\n- `Wikipedia-API`: For interacting with the official Wiki API.\n- `Flask`: To power the web application.\n- `colorama`: (Assuming) For the colored terminal output.\n\n---\n\u003cimg width=\"942\" height=\"760\" alt=\"image\" src=\"https://github.com/user-attachments/assets/81030780-7959-4382-890c-bb899f704ba0\" /\u003e\n\u003cimg width=\"936\" height=\"666\" alt=\"image\" src=\"https://github.com/user-attachments/assets/fa925192-3081-4d3f-844e-a4e9d7d79f7c\" /\u003e\n\u003cimg width=\"892\" height=\"606\" alt=\"image\" src=\"https://github.com/user-attachments/assets/fe366f7f-c3d1-4439-b619-c6ad72ee7736\" /\u003e\n\u003cimg width=\"895\" height=\"459\" alt=\"image\" src=\"https://github.com/user-attachments/assets/c5c691a5-b889-425a-9df8-f69e6380ec6a\" /\u003e\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhotaro26%2Fwiki_web","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhotaro26%2Fwiki_web","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhotaro26%2Fwiki_web/lists"}