Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pintu544/assignmentpartb
Scrape books data from http://books.toscrape.com and store it in Database.
https://github.com/pintu544/assignmentpartb
Last synced: 6 days ago
JSON representation
Scrape books data from http://books.toscrape.com and store it in Database.
- Host: GitHub
- URL: https://github.com/pintu544/assignmentpartb
- Owner: pintu544
- Created: 2023-09-28T05:42:35.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-09-28T05:44:32.000Z (about 1 year ago)
- Last Synced: 2024-04-13T23:09:54.648Z (8 months ago)
- Language: JavaScript
- Size: 21.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
```markdown
# Book Scraper and Database Storer
## Overview
This project is a web scraper that collects book data from [http://books.toscrape.com](http://books.toscrape.com) and stores it in an SQLite database. It scrapes book attributes like name, price, availability, and ratings from all 50 pages of the website.
## Prerequisites
Before running the application, make sure you have the following installed:
- Node.js: [https://nodejs.org/](https://nodejs.org/)
- npm (Node Package Manager): Included with Node.js installation
## Getting Started
1. Clone this repository to your local machine:
```bash
git clone [](https://github.com/pintu544/AssignmentPartB)
```
2. Navigate to the project directory:
```bash
cd server
```
3. Install project dependencies:
```bash
npm install
```
4. Create the SQLite database by running the database setup script:
```bash
node /db/db.js
```
## Usage
To scrape book data and store it in the database, run the main script:
```bash
node index.js
```
The script will initiate the scraping process for all 50 pages of books from [http://books.toscrape.com](http://books.toscrape.com). The scraped data will be stored in the `books.db` SQLite database.
## Database Structure
The SQLite database (`books.db`) contains two tables:
1. `books`: Stores book data including title, price, availability, and rating.
2. `sqlite_sequence`: Automatically generated by SQLite to keep track of table row counts.
## Contributing
Contributions to this project are welcome. Feel free to open issues and pull requests.
## License
This project is licensed under the [MIT License](LICENSE).
## Acknowledgments
- This project uses [axios](https://axios-http.com/) for making HTTP requests.
- Web scraping is done using [cheerio](https://cheerio.js.org/) for parsing HTML.
- Data is stored in an SQLite database using the [sqlite3](https://github.com/sqlitejs/sqlite3) library.
## Contact
For questions or inquiries, contact [Pintu Kumar](mailto:[email protected]).
```
Replace `` with the URL of your project's Git repository. Customize the content under each section (Overview, Prerequisites, Getting Started, Usage, Database Structure, Contributing, License, Acknowledgments, Contact) to provide relevant information about your project.
The README.md file serves as documentation for your project, making it easier for others (and yourself) to understand, use, and contribute to your code.