{"id":26412673,"url":"https://github.com/moustafamohamed01/web-summarizer-ai","last_synced_at":"2026-04-17T00:35:33.399Z","repository":{"id":282576896,"uuid":"949031108","full_name":"MoustafaMohamed01/web-summarizer-ai","owner":"MoustafaMohamed01","description":"A Python tool to scrape and summarize website content using AI. Built with Selenium, BeautifulSoup, LLaMA 3.2, and Google's Gemini AI, this project extracts the main text from any website and generates a concise summary in markdown format. Perfect for quickly understanding long articles, blogs, or news pages.","archived":false,"fork":false,"pushed_at":"2025-04-26T18:33:49.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-26T19:29:31.246Z","etag":null,"topics":["ai","beautifulsoup","gemini-ai","llama","python","selenium","text-preprocessing","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MoustafaMohamed01.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-15T14:17:29.000Z","updated_at":"2025-04-26T18:33:52.000Z","dependencies_parsed_at":"2025-03-15T15:38:42.744Z","dependency_job_id":null,"html_url":"https://github.com/MoustafaMohamed01/web-summarizer-ai","commit_stats":null,"previous_names":["moustafamohamed01/web-summarizer-ai"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MoustafaMohamed01/web-summarizer-ai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MoustafaMohamed01%2Fweb-summarizer-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MoustafaMohamed01%2Fweb-summarizer-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MoustafaMohamed01%2Fweb-summarizer-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MoustafaMohamed01%2Fweb-summarizer-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MoustafaMohamed01","download_url":"https://codeload.github.com/MoustafaMohamed01/web-summarizer-ai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MoustafaMohamed01%2Fweb-summarizer-ai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31910165,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"ssl_error","status_checked_at":"2026-04-16T18:21:47.142Z","response_time":69,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","beautifulsoup","gemini-ai","llama","python","selenium","text-preprocessing","web-scraping"],"created_at":"2025-03-17T22:08:27.833Z","updated_at":"2026-04-17T00:35:33.390Z","avatar_url":"https://github.com/MoustafaMohamed01.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Website Summarizer\n\nThis project provides an AI-powered website summarization tool that extracts and summarizes website content using two different AI models: **Google Gemini API** and **LLaMA 3.2**. It scrapes the text from a given URL, removes irrelevant elements, and generates a concise summary in markdown format.\n\n---\n\n## Features\n\n- **Web Scraping**: Extracts the title and main text content from a website.\n- **Content Cleaning**: Removes unnecessary HTML elements (e.g., scripts, styles, images) to keep only the relevant content.\n- **AI Summarization**:\n  - Uses the **Gemini API** to generate summaries in `web_summarizer_ai.py` and `web_summarizer_ai.ipynb`.\n  - Uses **LLaMA 3.2** (via Ollama API) to generate summaries in `web_summarizer_llama.py` and `web_summarizer_llama.ipynb`.\n- **IPython Support**: Displays the summary in markdown format for easy readability in Jupyter notebooks or IPython environments.\n\n---\n\n## Setup\n\n### Prerequisites\n\n- Python 3.7 or higher.\n- Either a **Gemini API key** (for Google AI) or **LLaMA 3.2 API** (for local inference via Ollama).\n\n### Installation\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/MoustafaMohamed01/web-summarizer-ai.git\n   cd web-summarizer-ai\n   ```\n\n2. Install the required dependencies:\n   ```bash\n   pip install requests beautifulsoup4 python-dotenv google-generativeai IPython\n   ```\n\n3. Set up your environment variables:\n   - Create a `.env` file in the project root directory:\n     \n     ```bash\n     cp .env.example .env\n     ```\n   - Open the `.env` file and add your Gemini API key:\n     \n     ```\n     GEMINI_API_KEY=your_api_key_here\n     ```\n\nFor LLaMA 3.2 support, ensure you have **Ollama** installed and running locally:\n```bash\ncurl -fsSL https://ollama.com/install.sh | sh\nollama pull llama3.2\n```\n\n---\n\n## Usage\n\n### 1. Summarizing Websites with Gemini API\n\nRun the Python script:\n```bash\npython web_summarizer_ai.py\n```\n\nOr use the interactive Jupyter Notebook:\n```bash\njupyter notebook web_summarizer_ai.ipynb\n```\n\n### 2. Summarizing Websites with LLaMA 3.2 (Ollama)\n\nRun the LLaMA-based summarizer script:\n```bash\npython web_summarizer_llama.py\n```\n\nOr open and run the Jupyter Notebook:\n```bash\njupyter notebook web_summarizer_llama.ipynb\n```\n\n---\n\n## File Structure\n\n- `web_summarizer_ai.py`: Python script that scrapes a website and generates a summary using the **Gemini API**.\n- `web_summarizer_ai.ipynb`: Jupyter Notebook version for interactive summarization using **Gemini API**.\n- `web_summarizer_llama.py`: Python script that scrapes a website and generates a summary using **LLaMA 3.2**.\n- `web_summarizer_llama.ipynb`: Jupyter Notebook version for interactive summarization using **LLaMA 3.2**.\n- `.env.example`: Template file for storing API keys.\n\n---\n\n## Dependencies\n\nThe project relies on the following Python packages:\n- `requests`: For making HTTP requests to fetch website content.\n- `beautifulsoup4`: For parsing HTML and extracting relevant content.\n- `python-dotenv`: For loading API keys from a `.env` file.\n- `google-generativeai`: For interacting with the **Gemini API**.\n- `IPython`: For displaying markdown summaries in Jupyter notebooks.\n- `Ollama`: For running **LLaMA 3.2** locally (required only for `web_summarizer_llama.py` and `web_summarizer_llama.ipynb`).\n\n---\n\n## Contributing\n\nContributions are welcome! If you’d like to contribute, please follow these steps:\n1. Fork the repository.\n2. Create a new branch for your feature or bugfix.\n3. Commit your changes.\n4. Submit a pull request.\n\n---\n\n## Acknowledgments\n\n- [Google AI Studio](https://makersuite.google.com/)\n- [LLM Engineering: Master AI, Large Language Models \u0026 Agents](https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models/?couponCode=KEEPLEARNING)\n- [Ollama](https://ollama.com/)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoustafamohamed01%2Fweb-summarizer-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoustafamohamed01%2Fweb-summarizer-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoustafamohamed01%2Fweb-summarizer-ai/lists"}