https://github.com/sh20raj/webscrapperjs
WebScrapperJS - Get Content/HTML of any website without being blocked by CORS even using JavaScript by WhollyAPI
https://github.com/sh20raj/webscrapperjs
api javascript javascript-library sh20raj webscraping websitescraping whollyapi
Last synced: 10 months ago
JSON representation
WebScrapperJS - Get Content/HTML of any website without being blocked by CORS even using JavaScript by WhollyAPI
- Host: GitHub
- URL: https://github.com/sh20raj/webscrapperjs
- Owner: SH20RAJ
- License: mit
- Created: 2022-01-25T10:26:05.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-04-09T12:10:07.000Z (about 2 years ago)
- Last Synced: 2025-08-09T19:03:08.640Z (10 months ago)
- Topics: api, javascript, javascript-library, sh20raj, webscraping, websitescraping, whollyapi
- Language: JavaScript
- Homepage: https://sh20raj.github.io/WebScrapperJS/
- Size: 112 KB
- Stars: 10
- Watchers: 1
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# WebScrapperJS [](https://www.jsdelivr.com/package/gh/SH20RAJ/WebScrapperJS) [](https://visitorbadge.io/status?path=https%3A%2F%2Fgithub.com%2FSH20RAJ%2FWebScrapperJS%2F)
WebScrapperJS - Get Content/HTML of any website without being blocked by CORS even using JavaScript by WhollyAPI
---
Website :- https://sh20raj.github.io/WebScrapperJS/
---
GitHub | Repl.it | Dev.to Article
---
## Grab the CDN or Download the JavaScript File
```html
```
---
- **`WebScrapper.get()`** will return you the content of the provided url in a String.
- **`WebScrapper.gethtml()`** will return you the content of the provided url as Parsed DOM. ( Will get the html and Parse it as a DOM object . Will return you a #Document)
- **`WebScrapper.getjson()`** will return you the content of the provided url as Parsed JSON.
---
### To Get HTML/Text/Content of Any Website in a String.
```javascript
var html = WebScrapper.get('https://webscrapperjs.sh20raj.repl.co/');//This will be return the HTML/Text inside the webpage in a String.
console.log(html);
```
This will be return the HTML/Text inside the webpage in a String.
---
### To Get HTML Content of Any Website in DOM Parsed Form `WebScrapper.gethtml()`
```javascript
var url = 'https://google.com/';
var html = WebScrapper.gethtml(url);//html of the url will be Parsed and stored in this variable
console.log(html);
console.log(html.title);//As you Use document.title you can Use Like this to get the title.
```
---
### Intialise own WebScrapper with URL `new scrapper()`
```javascript
let MyWebScrapper = new scrapper('https://example.com/');
//You can now directly call gethtml() instead of passing a url into it.
console.log(MyWebScrapper.gethtml()); //Grab https://example.com/ and print on console
```
Still you can Use new created scrapper `MyWebScrapper` for grabbing new URLs. Like
```javascript
let MyWebScrapper = new scrapper('https://example.com/');
//You can now directly call gethtml() instead of passing a url into it.
console.log(MyWebScrapper.gethtml()); //Grab https://example.com/ and print on console
console.log(MyWebScrapper.gethtml('https://example.com/')); //Grab https://youtube.com/ and print on console
```
---
### You can also fetch JSON Using WebScrapperJS
```javascript
var json = WebScrapper.getjson('https://jsonplaceholder.typicode.com/todos/1');//Return result direct in json format
console.log(json);
```
Try This
---
## Getting Result more Faster
**Use the Below codes/methods only if the origin or feching URL is not blocked by CORS Like this**
if your origin is not blocking you then you must use the below fetch() code instead of gethtml() directly.
because it returns the results faster without using API.It will directly fetch origin using AJAX.
### Use `WebScrapper.fetch()` to get the html/text in a string
We will use this url `https://webscrapperjs.sh20raj.repl.co/` because it is not blocked.
```javascript
var html = WebScrapper.fetch('https://webscrapperjs.sh20raj.repl.co/');//This will be return the HTML/Text inside the webpage a string.
console.log(html);
```
This will be return the HTML/Text inside the webpage in a String.
---
### Use `WebScrapper.fetchhtml()` to get the Parsed HTML/DOM document as `WebScrapper.gethtml()`.
```javascript
var html = WebScrapper.fetchhtml('https://webscrapperjs.sh20raj.repl.co/');//This will be return the Parsed HTML inside the webpage.
console.log(html);
console.log(html.title);
```
Try this
---
### Use `WebScrapper.fetchjson()` to get the Parsed JSON
```javascript
var json = WebScrapper.fetchjson('https://webscrapperjs.sh20raj.repl.co/sample.json');//This will be return the JSON inside the webpage.
console.log(json);
console.log(json.id);
```
Try this
---
### Try this on Codepen
Sample Code | Codepen :- https://codepen.io/SH20RAJ/pen/VwrwjXJ?editors=1001
```html
let MyWebScrapper = new scrapper('https://google.com/');
//You can now directly call gethtml() instead of passing a url into it.
console.log(MyWebScrapper.gethtml()); //Grab https://example.com/ and print on console
var html = MyWebScrapper.gethtml('https://example.com/');
console.log(html); //Grab https://youtube.com/ and print on console
document.getElementById('scrappedcontent').innerHTML = html;
```
See Results Here
---
# Other Features
## `WebScrapper.getparam()` get URL Parameters
Assuming your Current URL is `https://example.com/?id=7`.
```javascript
let id = WebScrapper.getparam('id');
console.log(id);//Will Return "7" .
```
### Use Custom string instead of current URL
```javascript
let id = WebScrapper.getparam('id','https://example.com/?id=20');
console.log(id);//Will Return "20" .
```
## `WebScrapper.getRandomInt()` get random integer in range
This function take 2 parameter `WebScrapper.getRandomInt(min,max)` the generated number will be in between min and max.
```javascript
let id = WebScrapper.getRandomInt(10,100);
console.log(id);//Will Return a number between 10 and 100 .
```
## Star History
