https://github.com/nubisub/sscasn
Scrape and analyze SSCASN data 2024
https://github.com/nubisub/sscasn
rest-api sraping
Last synced: over 1 year ago
JSON representation
Scrape and analyze SSCASN data 2024
- Host: GitHub
- URL: https://github.com/nubisub/sscasn
- Owner: nubisub
- Created: 2024-11-08T14:03:23.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-11-17T15:28:43.000Z (over 1 year ago)
- Last Synced: 2025-01-31T15:29:06.190Z (over 1 year ago)
- Topics: rest-api, sraping
- Language: TypeScript
- Homepage: https://api-sscasn.vercel.app/api-docs
- Size: 56.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# SSCASN API Scraper
This repository contains a script to scrape data from SSCASN (Sistem Seleksi Calon ASN) API, which provides information about civil servant vacancies (formasi penerimaan CPNS 2024) in Indonesia. The script uses Jupyter Notebook and implements concurrency to efficiently gather data, which is then stored directly in a local MongoDB database.
## Features
- Concurrency: Utilizes concurrent requests to speed up the data scraping process.
- Jupyter Notebook: Interactive and easy-to-follow notebook format.
- Local MongoDB Storage: Stores the scraped data directly in a local MongoDB database for easy access and querying.
- High Performance: Successfully scraped 111,922 data entries in just 21 minutes.
Requirements
## Performance
- Scraped Data: 111,922 entries
- Time Taken: 21 minutes
- Execution Date and Time: 11/5/2024, 11:00:00 AM
- Internet Speed: 41 Mbps
- Latency:
- Unloaded: 8 ms
- Loaded: 13 ms