Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/arv-anshul/ecommerce-scrapper-api
Provides API endpoints to scrape data from e-commerce websites. (For now flipkart.com only)
https://github.com/arv-anshul/ecommerce-scrapper-api
amazon curler e-commerce ecommerce fastapi flipkart http-requests modular project pydantic python3 web-scrapping
Last synced: 22 days ago
JSON representation
Provides API endpoints to scrape data from e-commerce websites. (For now flipkart.com only)
- Host: GitHub
- URL: https://github.com/arv-anshul/ecommerce-scrapper-api
- Owner: arv-anshul
- Created: 2023-10-01T03:06:25.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-02-14T05:35:12.000Z (11 months ago)
- Last Synced: 2024-02-14T06:28:58.490Z (11 months ago)
- Topics: amazon, curler, e-commerce, ecommerce, fastapi, flipkart, http-requests, modular, project, pydantic, python3, web-scrapping
- Language: Python
- Homepage: https://www.linkedin.com/posts/arv-anshul_fastapi-api-ecommerce-activity-7128575593570340864-WE2m
- Size: 687 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# E-Commerce Scrapper API
Welcome to the E-commerce API project! 🌐 This API is designed to scrape valuable data from E-commerce websites, with a focus on [Flipkart](https://flipkart.com) (for now).
#### 🧐 What I Learned While Creating This API?
- 🕵️ **FastAPI Framework:** Explored the ins and outs of the powerful `FastAPI` framework, leveraging its asynchronous capabilities for efficient and speedy data retrieval.
- 🤓 **HTTP Request Methods and Status Codes:** Delved deeper into HTTP request methods at an intermediate level, mastering the use of various HTTP status codes for a more nuanced and controlled API interaction.
- 💡 **Error Handling and Exception Tricks:** Implemented clever techniques to handle errors and exceptions, ensuring a robust and graceful API experience for users.
- 🗺️ **Pydantic Library and Modular Programming:** Explored the versatility of the `pydantic` library for data validation and modular programming. Embraced object-oriented programming (OOPs) concepts for a more structured and maintainable codebase.
- 🧪 **Writing Tests:** Implemented thorough [tests](ecommerce/tests/test_api/) for the FastAPI application, ensuring the reliability and correctness of every API endpoint.
### 🤝 What You Can Achieve With My APIs?
- **Data Analysis:** Conduct in-depth data analysis on the information fetched from E-commerce websites. Gain insights into price variations, product specification differences, and overall market trends.
- **Machine Learning Integration:** Leverage the scraped data to build machine learning models. Explore possibilities such as sentiment analysis, product price prediction (for items like Mobiles, Laptops, etc.), and other innovative applications.
- **Custom Use Cases:** The versatility of this API allows you to tailor its application to your specific needs. Whether you're interested in market research, trend analysis, or creating unique machine learning applications, the possibilities are vast.
- **Data-Driven Decision Making:** Empower your decision-making process with accurate and up-to-date data from E-commerce websites. Use the insights gained to make informed choices in various domains.
### 🧑💻 Tech Stacks
| Stack | Tech |
| --------------: | :---------------------------------------------------------------------------------------------------------- |
| Web scraping |  |
| Backend |  |
| Data Validation |  |
| Websites |  |
| Curl Parser |  |
### 🧩 Usage
1. Clone this repo.
2. Install the projects requirements with following command:
```sh
pip install -r requirements.txt
```
3. Run the fastapi app.
```sh
uvicorn --reload ecommerce.api.app:app
```
4. Now FastAPI instance is running on your system. So, you can access it on your localhost with the following url:
```url
http://localhost:8000/
```
For better representation of the APIs head to:
```url
http://localhost:8000/docs
```
### 📌 API Endpoints
#### 🖼️ Screenshot of OpenAPI Docs
