https://github.com/marcnuth/websites-keywords-generator
This repo will provide with methods to generate keywords of given urls.
https://github.com/marcnuth/websites-keywords-generator
Last synced: over 1 year ago
JSON representation
This repo will provide with methods to generate keywords of given urls.
- Host: GitHub
- URL: https://github.com/marcnuth/websites-keywords-generator
- Owner: Marcnuth
- License: mit
- Created: 2016-06-10T10:59:56.000Z (about 10 years ago)
- Default Branch: master
- Last Pushed: 2016-06-11T15:37:13.000Z (about 10 years ago)
- Last Synced: 2025-01-20T06:13:54.344Z (over 1 year ago)
- Language: Python
- Size: 8.79 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Websites-Keywords-Generator
## What this for
This repo will provide with methods to generate keywords of given urls.
## Requirements
- scrapy: http://scrapy.org/
- jieba: https://github.com/fxsjy/jieba
## Usage
### Shell command
```
scrapy crawl keywords
```
### Output
The output result is in ./websites_keywords_extractor/data/keyword.db (The default file, or you can config it in settings.py)
The file is generated by sqlite3.
Two tables are involved:
#### Table: keywords:
- Columns:
- word: the extracted key word
- count: show times count
#### Table: urls
- Coulumns:
- name: the url
- timestamp: the time of parsing this url