{"id":20165769,"url":"https://github.com/aliktk/ecommerce_scraping","last_synced_at":"2026-06-07T08:32:47.428Z","repository":{"id":250180828,"uuid":"833653640","full_name":"Aliktk/ecommerce_scraping","owner":"Aliktk","description":"Develop a simplified version of an AI-powered Search Engine","archived":false,"fork":false,"pushed_at":"2024-07-31T13:07:41.000Z","size":10429,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-13T14:52:48.117Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aliktk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-25T13:32:21.000Z","updated_at":"2024-07-31T13:07:45.000Z","dependencies_parsed_at":"2024-07-31T16:23:46.028Z","dependency_job_id":null,"html_url":"https://github.com/Aliktk/ecommerce_scraping","commit_stats":null,"previous_names":["aliktk/ecommerce_scraping"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aliktk%2Fecommerce_scraping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aliktk%2Fecommerce_scraping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aliktk%2Fecommerce_scraping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aliktk%2Fecommerce_scraping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aliktk","download_url":"https://codeload.github.com/Aliktk/ecommerce_scraping/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241601750,"owners_count":19988971,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T00:39:05.287Z","updated_at":"2025-03-03T03:24:46.285Z","avatar_url":"https://github.com/Aliktk.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🛒 Ecommerce Scraping Project\n\nWelcome to the **Ecommerce Scraping Project**! This project is a powerful Django-based web application designed to scrape product data from various e-commerce websites and present it in an intuitive and user-friendly interface.\n\n## 🌟 Features\n\n- **Multi-site Scraping:** Collect product data from Amazon, eBay, and Best Buy.\n- **Database Storage:** Efficiently store scraped data in a PostgreSQL database.\n- **Detailed Product Display:** Show comprehensive product information including image, name, price, reviews, sentiment score, and sentiment label.\n- **Smart Product Selection:** Automatically identify and highlight the best product based on sentiment score and price.\n\n## 🛠 Requirements\n\n- Python 3.10\n- Django 4.2\n- Django REST framework 3.14.0\n- psycopg2-binary 2.9.6\n- requests 2.31.0\n- beautifulsoup4 4.12.2\n- lxml 4.9.3\n- vaderSentiment==3.3.2\n\n## ⚙️ Installation\n\nFollow these steps to set up the project:\n\n1. **Clone the Repository:**\n\n    ```bash\n    git clone https://github.com/yourusername/ecommerce_scraping.git\n    cd ecommerce_scraping\n    ```\n\n2. **Create a Virtual Environment and Activate it:**\n\n    ```bash\n    python -m venv venv\n    source venv/bin/activate  # On Windows use `venv\\Scripts\\activate`\n    ```\n\n3. **Install the Dependencies:**\n\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n4. **Apply the Migrations:**\n\n    ```bash\n    python manage.py makemigrations\n    python manage.py migrate\n    ```\n\n5. **Run the Development Server:**\n\n    ```bash\n    python manage.py runserver\n    ```\n\n## 🐳 Docker Setup\n\nTo run the project using Docker, follow these steps:\n\n1. **Build the Docker Image:**\n\n    ```bash\n    docker build -t ecommerce_scraping .\n    ```\n\n2. **Run the Docker Container:**\n\n    ```bash\n    docker run -p 8000:8000 ecommerce_scraping\n    ```\n\n## 🚀 Usage\n\n1. Open your web browser and navigate to `http://127.0.0.1:8000`.\n2. Enter a product name in the search bar and click **\"Search\"**.\n3. If the product data is already in the database, it will be displayed immediately.\n4. If the product data is not in the database, the application will scrape the data and display it once the scraping is complete.\n\n## 🛠 Configuration\n\nEnsure to set environment variables for API keys, database credentials, etc., as needed. This can be done by creating a `.env` file in the root directory and adding the required variables.\n\n## 🧩 Contributing\n\nWe welcome contributions to enhance the project! Please follow these steps:\n\n1. **Fork the repository**.\n2. **Create a new branch** for your feature or bug fix.\n3. **Submit a Pull Request** with a clear description of your changes.\n\n## ❓ Troubleshooting\n\nIf you encounter any issues, please refer to the [FAQ](#) section or open an issue on GitHub.\n\n## 💼 Credits\n\nWe appreciate the use of external libraries and tools that made this project possible. Special thanks to the developers of BeautifulSoup, Scrapy, Django, and others.\n\n## 📜 License\n\nThis project is licensed under the Apache License 2.0.\n\n## 📧 Contact\n\nFor any questions, suggestions, or feedback, please feel free to contact:\n\n**Name:** Ali Nawaz  \n**Email:** [nawazktk99@gmail.com](mailto:nawazktk99@gmail.com)\n\n---\nHappy Scraping! 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faliktk%2Fecommerce_scraping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faliktk%2Fecommerce_scraping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faliktk%2Fecommerce_scraping/lists"}