https://github.com/keithrozario/my_procurement_scraper
Scraper for the myprocurement Website
https://github.com/keithrozario/my_procurement_scraper
Last synced: about 2 months ago
JSON representation
Scraper for the myprocurement Website
- Host: GitHub
- URL: https://github.com/keithrozario/my_procurement_scraper
- Owner: keithrozario
- Created: 2017-12-18T14:55:53.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2017-12-18T16:20:23.000Z (almost 8 years ago)
- Last Synced: 2025-03-22T08:21:21.356Z (7 months ago)
- Language: Python
- Size: 777 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# my_procurement_scraper
Scrapes data from the Malaysian MyProcurement Website (http://myprocurement.treasury.gov.my/)
This replaces the old work I did here https://github.com/keithrozario/MyProcurementDataScrapper
If you're not interested in running the script, data from 19-Dec is in the repository (it's bad practice to include data in a git repo, but what the hell!)
# rundingan.py
Scrapes data from http://myprocurement.treasury.gov.my/custom/p_keputusan_rundingan.php?sort=&by=&page=
Outputs to rundingan.csv (362 rows)
# tender.py
Scrapes data from http://myprocurement.treasury.gov.my/custom/p_keputusan_tender_arkib_new.php?sort=&by=&page=
Outputs to tender.csv (16,615 rows)
+ Every tender that was jointly won by two companies, gets one row per company
+ Total tender value remains the same for both rows
+ An additional 'Jumlah Tender Berjaya' field contains number of companies that won the tender
# tender_perunding.py
Scrapes data from http://myprocurement.treasury.gov.my/custom/p_keputusan_tender_perunding_new.php?sort=&by=&page=
Output to tender-perunding.csv (127 rows)