https://github.com/codingo/succinct
A tool to create a text summary of a provided list of websites
https://github.com/codingo/succinct
Last synced: 8 months ago
JSON representation
A tool to create a text summary of a provided list of websites
- Host: GitHub
- URL: https://github.com/codingo/succinct
- Owner: codingo
- License: gpl-3.0
- Created: 2023-03-21T21:31:40.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-03-21T22:53:14.000Z (almost 3 years ago)
- Last Synced: 2025-04-20T09:59:21.362Z (8 months ago)
- Language: Go
- Size: 71.3 KB
- Stars: 11
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Succinct
This Go script takes a list of URLs and outputs the most common words for each URL along with a summary paragraph describing the website. The script uses the `github.com/JesusIslam/tldr` library for summarization.
## Usage
1. Install the required libraries:
```
go get github.com/JesusIslam/tldr
go get github.com/PuerkitoBio/goquery
```
2. Build the script:
```
go build main.go
```
3. Run the script with the required flags:
```
./main -t [-e ] [-n ] [-threads ] [-s ]
```
- `-t` or `--targets`: Targets file (newline per webpage to load)
- `-e` or `--exclude`: Exclude file (newline per word to exclude) - optional
- `-n`: The number of most common words to output - optional, default is 10
- `--threads`: The number of threads to use - optional, default is 10
- `-s`: The number of sentences in the summary - optional, default is 3
## Example
Create a `targets.txt` file with a list of URLs to process:
```
https://example.com
https://example.org
```
Create an `exclude.txt` file with a list of words to exclude:
```
the
and
```
Run the script:
```
./main -t targets.txt -e exclude.txt -n 10 -threads 10 -s 3
```