https://github.com/delabi/typescript-scraper

A website scraper, API generator and backend for sugar price data around Africa build on Node.JS, ExpressJS, TypeORM and Axios
https://github.com/delabi/typescript-scraper

axios-restful database expressjs nodejs typeorm typescript

Last synced: 2 months ago
JSON representation

A website scraper, API generator and backend for sugar price data around Africa build on Node.JS, ExpressJS, TypeORM and Axios

• Host: GitHub
• URL: https://github.com/delabi/typescript-scraper
• Owner: delabi
• Created: 2023-01-19T17:48:33.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-01-28T03:35:57.000Z (over 2 years ago)
• Last Synced: 2025-02-06T12:17:38.782Z (4 months ago)
• Topics: axios-restful, database, expressjs, nodejs, typeorm, typescript
• Language: TypeScript
• Homepage: https://sugar-scraper.onrender.com/sugars
• Size: 138 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
This is the backend of the sugar index application.

It handles

1. Sugar information Scrapper from 3 jumia websites: Kenya, Uganda and Nigeria every 6 hours.

2. Stores the scrapped data in a PostgreSQL database via TypeORM.

3. Type ORM also Generates a REST API that can be used to consume the data.

Steps to run this project after cloning this repository:

1. Run `npm i` command in the project folder to install dependencies

2. Setup database settings inside `data-source.ts` file

3. Run `npm start` commands

The live application is running on https://sugar-scraper.onrender.com/sugars

You will be able to access the api locally using https://localhost:3000/sugars

From there you can access the REST API with information about sugar prices obtained from the Jumia Websites.

The current POSGTRES database is hosted on AWS RDS and the configurations are in the datasource.ts file.