Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/lopezjurip/buscacursos-uc-scraper
BuscaCursos UC Scraper
https://github.com/lopezjurip/buscacursos-uc-scraper
Last synced: 25 days ago
JSON representation
BuscaCursos UC Scraper
- Host: GitHub
- URL: https://github.com/lopezjurip/buscacursos-uc-scraper
- Owner: lopezjurip
- License: mit
- Created: 2015-12-22T23:48:20.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2016-02-03T19:18:30.000Z (almost 9 years ago)
- Last Synced: 2024-10-07T09:18:31.315Z (about 1 month ago)
- Language: JavaScript
- Homepage: http://buscacursos.uc.cl/
- Size: 66.4 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# BuscaCursos UC Scraper
[![dependencies][dependencies-image]][dependencies-url] [![dev-dependencies][dev-dependencies-image]][dev-dependencies-url]
This project uses [mrpatiwi/buscacursos-uc](https://github.com/mrpatiwi/buscacursos-uc) and requires Node 5.3.x or newer.
> Use by your own responsibility, because this creates heavy load on [http://buscacursos.uc.cl](http://buscacursos.uc.cl)
## How does it works
When you query [http://buscacursos.uc.cl](http://buscacursos.uc.cl) you can have at most 50 results without be able to get the next results (no pagination).
This works by querying the initials (`MAT`, `LET`, etc) and counting the results, then if the results are more than 50, this perform another ten (recursive) queries like: `MAT0`, `MAT1`, ..., `MAT9`, otherwise just return the results.
In shorter words, this performs a kind of *Depth First Search (DFS)*.
## Install
This shall not be published to `npm`.
To install as a dependency, edit the `package.json` and add:
```json
"dependencies": {
"buscacursos-uc-scraper": "git+https://[email protected]/mrpatiwi/buscacursos-uc-scraper"
}
```
## Usage
The function `deepSearch` receives an array of course initials (strings of length 3).
* Example: `MAT`, `IIC`, `DNO`, ...
```javascript
scraper.deepSearch(['IIC', 'MAT'], { year: 2016, period: 1 }).then(courses => {
console.log('Count:', courses.length);
fs.writeFile('./some.json', JSON.stringify(courses, null, 4), function(err) {
if (err) return console.error(err);
console.log('success!');
});
}).catch(err => {
console.error(err);
});
```
All the (current) available identifiers are importable with:
```javascript
const scraper = require('buscacursos-uc-scraper');
const initials = scraper.initials;
```
Usage:
```javascript
'use strict';
const fs = require('fs');
const scraper = require('buscacursos-uc-scraper');
// Recommended:
// Use known initials to speed the process.
const initials = scraper.initials;
scraper.deepSearch(initials, { year: 2016, period: 1 }).then(courses => {
console.log('Count:', courses.length);
fs.writeFile('./all.json', JSON.stringify(courses, null, 4), function(err) {
if (err) return console.error(err);
console.log('success!');
});
}).catch(err => {
console.error(err);
});
// To perform a full search use (rediscover initials):
//
// This takes a lot of time to complete.
// Also, creates heavy load on the server, use with caution.
// Big chances of failing:
//
// scraper.all({ year: 2016, period: 1 }).then(courses => { ...
```
[dependencies-image]: https://david-dm.org/mrpatiwi/buscacursos-uc-scraper.svg
[dependencies-url]: https://david-dm.org/mrpatiwi/buscacursos-uc-scraper
[dev-dependencies-image]: https://david-dm.org/mrpatiwi/buscacursos-uc-scraper/dev-status.svg
[dev-dependencies-url]: https://david-dm.org/mrpatiwi/buscacursos-uc-scraper#info=devDependencies