Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/circa10a/web-link-tester
Flask App to scrape hrefs from websites and validate links
https://github.com/circa10a/web-link-tester
beautifulsoup docker-image flask flask-application flask-restful python3 uwsgi-nginx
Last synced: 3 months ago
JSON representation
Flask App to scrape hrefs from websites and validate links
- Host: GitHub
- URL: https://github.com/circa10a/web-link-tester
- Owner: circa10a
- Created: 2017-11-04T15:27:13.000Z (about 7 years ago)
- Default Branch: main
- Last Pushed: 2021-05-22T01:34:34.000Z (over 3 years ago)
- Last Synced: 2024-04-15T02:02:53.021Z (9 months ago)
- Topics: beautifulsoup, docker-image, flask, flask-application, flask-restful, python3, uwsgi-nginx
- Language: HTML
- Homepage:
- Size: 212 KB
- Stars: 11
- Watchers: 1
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# web-link-tester
[](https://microbadger.com/images/circa10a/web-link-tester "Get your own image badge on microbadger.com")
[](https://microbadger.com/images/circa10a/web-link-tester "Get your own version badge on microbadger.com")
Asynchronous Flask App to scrape and validate links via GUI or REST API
## Usage
Why use the web-link-tester?
1) Test your production site via GUI or programmatically to ensure no bad page routes or forgotten references.
2) It's free.
## To deploy the Web App/API
---
### Docker
```bash
docker run -d --name link-tester -p 8080:8080 circa10a/web-link-tester
```
Access via http://localhost:8080
### Python
```bash
python main.py
```
Access via http://localhost:8080
> Set environment variable `PORT` locally to change listening port from `8080`
### API Usage
```bash
curl -X POST -H "content-type: application/json" --data '{"url": "https://www.github.com"}' http://localhost:8080/api
```
### Example JSON Output
```bash
$ curl -s -X POST -H "content-type: application/json" --data '{"url": "https://caleblemoine.dev"}' http://localhost:8080/api
{
"links": [
{
"code": 999,
"url": "https://www.linkedin.com/in/caleblemoine/"
},
{
"code": 200,
"url": "https://github.com/circa10a"
},
{
"code": 200,
"url": "https://github.com/circa10a/express-jwt"
},
{
"code": 200,
"url": "https://github.com/pyouroboros/ouroboros"
},
{
"code": 200,
"url": "https://github.com/circa10a/filter-object-array"
},
{
"code": 200,
"url": "https://github.com/circa10a/easy-soap-request"
},
{
"code": 200,
"url": "https://medium.com/better-programming/how-to-perform-soap-requests-with-node-js-4a9627070eb6"
},
{
"code": 200,
"url": "https://github.com/circa10a/web-link-tester"
},
{
"code": 200,
"url": "https://github.com/circa10a/Device-Monitor-Dashboard"
},
{
"code": 200,
"url": "https://caleblemoine.dev/monitor/"
},
{
"code": 200,
"url": "https://hub.docker.com/u/circa10a"
},
{
"code": 200,
"url": "https://caleblemoine.dev/gitfolio/"
},
{
"code": 999,
"url": "https://www.linkedin.com/in/caleblemoine/"
},
]
}
```
### Stack
- Utilizes gunicorn for multiple workers/threading.
- Python 3
- BeautifulSoup4
- Jquery
## Screenshots
