https://github.com/aweirddev/wikivoyage
Consume Wikivoyage pages into pure (elegant) Markdown. Probably.
https://github.com/aweirddev/wikivoyage
scraper travel wikipedia wikipedia-api wikivoyage
Last synced: about 1 year ago
JSON representation
Consume Wikivoyage pages into pure (elegant) Markdown. Probably.
- Host: GitHub
- URL: https://github.com/aweirddev/wikivoyage
- Owner: AWeirdDev
- License: mit
- Created: 2024-05-11T06:22:31.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-11T12:58:27.000Z (about 2 years ago)
- Last Synced: 2025-01-24T12:32:28.815Z (over 1 year ago)
- Topics: scraper, travel, wikipedia, wikipedia-api, wikivoyage
- Language: Python
- Homepage:
- Size: 7.81 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# wikivoyage
Consume [Wikivoyage](https://en.wikivoyage.org) into pure (elegant) Markdown. Probably.
```
$ pip install wikivoyage
```
## Usage
Just do what you want.
```python
import wikivoyage
# Learn about the States
res = wikivoyage.get("https://en.wikivoyage.org/wiki/United_States_of_America")
print(res)
```
```python
WikivoyageSections(
url='https://en.wikivoyag…',
sections=[
Section(
title='United States of America',
content='The **United States of America** spans a continent…'
),
Section(
title='Regions',
content="Wikivoyage organizes the 50 states and the nation'…"
),
Section(
title='Cities',
content='The following is a list of nine of the most notabl…'
),
...
]
)
```
Super simple.
## Why?
I'm initially an "i-aint-reading-allat" person, thus I made this project so I can better categorize all the sections from a Wikivoyage page, feed it to an LLM and summarize it for me.
Check out [where-next](https://github.com/AWeirdDev/where-next), a Next.JS project created by me to learn more.
## API Reference
The juicy API reference. Press CTRL + F to search for what you need or type directly in the search bar if you're on mobile.
### def get()
```python
get(url: str, **requests_kwargs) -> WikivoyageSections
```
Fetches a Wikivoyage page and parses it into Markdown.
**Args**:
- url (str): The URL.
- \*\*requests_kwargs: Keyword-only arguments to pass into `requests.get`.
**Returns**:
WikivoyageSections: Wikivoyage sections.
### dataclass WikivoyageSections
Represents all the Wikivoyage sections. *(manually-created)*
```python
class WikivoyageSections:
url: str # The request URL
sections: List[Section] # Sections
```
### dataclass Section
Repersents a Wikivoyage section. *(manually-created)*
```python
class Section:
title: str # The title. Alias: `section`
content: str # The content. Alias: `markdown`
```
## Contributing
Contributions are welcome! And yes, I'm not really familiar with GitHub so please help me.
## License
MIT License.