https://github.com/imfht/webhelper

https://github.com/imfht/webhelper

Last synced: 22 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/imfht/webhelper
• Owner: imfht
• Created: 2022-04-24T09:29:31.000Z (over 2 years ago)
• Default Branch: master
• Last Pushed: 2022-06-15T06:52:31.000Z (over 2 years ago)
• Last Synced: 2024-08-05T17:44:16.661Z (4 months ago)
• Language: Python
• Size: 24.4 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-hacking-lists - imfht/webhelper - (Python)

README

        
# WebHelper

Web 爬虫小工具

## 安装

```bash

pip install web-helper

```

## 使用方法

```python

from web_helper import *

title = get_html(f'https://baidu.com', obey_robot=False).to_soup().find('title').text

assert title == '百度一下，你就知道'

```

## 使用代理

```python

from web_helper import *

set_global_random_proxy('ws_token')  # 设置全局代理,TOKEN 前往: https://proxy.webshare.io/subscription/ 获取

title = get_html(f'https://baidu.com', obey_robot=False).to_soup().find('title').text

assert title == '百度一下，你就知道'

```

## 设置日志级别

```python

from web_helper import *

import sys

logger.remove()

logger.add(sys.stdout, level="TRACE")

title = get_html(f'https://baidu.com', obey_robot=False).to_soup().find('title').text

assert title == '百度一下，你就知道'

```