Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/vitaee/jobscrapernode

Web Scraper Task via Beekin
https://github.com/vitaee/jobscrapernode

Last synced: 14 days ago
JSON representation

Web Scraper Task via Beekin

Host: GitHub
URL: https://github.com/vitaee/jobscrapernode
Owner: Vitaee
Created: 2022-03-10T16:40:38.000Z (almost 3 years ago)
Default Branch: main
Last Pushed: 2022-03-29T06:56:12.000Z (almost 3 years ago)
Last Synced: 2024-11-06T19:50:55.490Z (2 months ago)
Language: TypeScript
Size: 233 KB
Stars: 1
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# Job List Scraper Node & TypeScript

This project includes Beekin's "Web Scraper Task" project developed by me according to requirements.

## Run project

- npm install
- npm run dev

## Resource components / urls

Major resource components supported by the JobScraper API are:

- jobs

## Jobs components and identifiers

Resource components can be used in conjunction with identifiers to retrieve the metadata for that identifier.

| resource | description |
| :-------------- | :------------------------------------------------------------------------------------------------------------------------ |
| `/jobs/` | this endpoint handle with all jobs which api have, accept get request and return all data with pagination support. |
| `/jobs/search/` | this endpoint handle with searching through to all data. Accept get request return searched data with pagination support. |
| `/jobs/sort/` | this endpoint handle with sorting. Accept get request and datas sorted by `postedDate` |
| `/jobs/scrape/` | this endpoint handle with scraping data from target website and it saves scraped data to json file. |

# Example query using URI parameters **all jobs**

- Get jobs with simple get request and if you use page param you can check other pages. 10 data per page listed.

| resource | description |
| :---------------------------- | :---------------------------------------- |
| `/jobs/?{page}={page_number}` | example url `localhost:3000/jobs/?page=1` |

### Example output

```
{
"message": "success",
"data": {
"page": 1,
"per_page": 10,
"pre_page": null,
"next_page": 2,
"total": 60,
"total_pages": 6,
"data": [
{
"title": "Software Engineer - Simulation",
"location": "Chesterfield, MO",
"description": "Looking to hire Software Engineer two to ten+ year...",
"postedDate": "February 13, 2022",
"compName": "CompuScienceIT"
},
.
.
.
]
}
```

# Example query using URI parameters **search**

Example output

```
{
"message": "success",
"filtered": {
"page": 1,
"per_page": 5,
"pre_page": null,
"next_page": 2,
"total": 18,
"total_pages": 4,
"data": [
{
"title": "Software Engineer",
"location": "London, UK",
"description": "We are looking to hire Software Engineers...",
"postedDate": "February 13, 2022",
"compName": "CompuScienceIT"
},
.
.
.
}
]
```

# Example query using URI parameters **sort**

Example output

```
{
"message": "success",
"sorted": [
{
"title": "IT Cloud DevOps Engineer",
"location": "Altstetten, Zurich, 8048, Switzerland",
"description": "Requisition Number your role..",
"postedDate": "March 02, 2022",
"compName": "UBS"
},
.
.
.
}
]
}

```

- If the result length is more than 5 pagination suppor will work so you can also add `page` query param to the request url.

# Example query using URI parameters **scrape**

Example output

```
{
"message": "We save data to json file succesfully."
}
```

# Technologies used

### Should be add