https://github.com/j-p-d-e-v/forex_webscraper
Gathers news data in forex factory.
https://github.com/j-p-d-e-v/forex_webscraper
database forex forex-factory forex-news headlessbrowser rust seaorm websscraping
Last synced: about 2 months ago
JSON representation
Gathers news data in forex factory.
- Host: GitHub
- URL: https://github.com/j-p-d-e-v/forex_webscraper
- Owner: j-p-d-e-v
- Created: 2024-03-10T01:15:02.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-10T01:48:42.000Z (over 2 years ago)
- Last Synced: 2025-01-29T06:52:18.750Z (over 1 year ago)
- Topics: database, forex, forex-factory, forex-news, headlessbrowser, rust, seaorm, websscraping
- Language: Rust
- Homepage:
- Size: 7.81 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Forex News Webscraper
A webscraper for forex news written in RUST programming.
SUPPORTED FOREX NEWS SITE
- Forex Factory
## CONFIGURATION
### ENVIRONMENT VARIABLES
- DATABASE_URL - The connection string of the MySQL Database
```
mysql://:@:/
```
- START_YEAR - The start range of the year to crawl.
```
START_YEAR=
```
- END_YEAR - The end range of the year to crawl.
```
END_YEAR=
```
## DEVELOPMENT
### Generating Entity
```
sea-orm-cli generate entity -v -o src/models
Note: Do not modify the entity files. The cli command will overwrite it.
```
### Database Migration
```
sea-orm-cli migrate up
sea-orm-cli migrate down
```
### Container Database
```
docker-compsoe -f db-docker-compose.yml up -d
```
### TESTING
```
cargo test
```
### RUNNING
```
cargo run
```
## REFERENCES
- Database ORM - https://www.sea-ql.org/SeaORM/docs/index/
- SeaORM Crate - https://docs.rs/sea-orm/0.12.14/sea_orm/index.html
- Headless Chrome - https://docs.rs/headless_chrome/latest/headless_chrome/
## DEVELOPER
- JP Mateo (jpmateo022@gmail.com)