Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/znitche/ao3-web-reader
ao3 scraper / reader
https://github.com/znitche/ao3-web-reader
ao3 archiveofourown beautifulsoup4 flask python3 webscraper
Last synced: 2 months ago
JSON representation
ao3 scraper / reader
- Host: GitHub
- URL: https://github.com/znitche/ao3-web-reader
- Owner: zNitche
- License: mit
- Created: 2022-12-13T17:05:42.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-28T22:20:11.000Z (2 months ago)
- Last Synced: 2024-11-28T23:22:21.741Z (2 months ago)
- Topics: ao3, archiveofourown, beautifulsoup4, flask, python3, webscraper
- Language: Python
- Homepage:
- Size: 248 KB
- Stars: 6
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## AO3 Web Reader
AO3 web reader / scraper app created using Flask, SQLite, Redis and Bootstrap, supporting:
- Accounts authentication.
- Getting works from AO3.
- Grouping works using tags.
- Adding works to favorites.
- Auto-update of added works.
- Downloading works.
- Force updating chapters, works and tags.
- Exporting works for ebook readers (html format).Hello there, It's been almost 2 years since I started working on this project, a lot of time right?
Anyway since then I became much better programmer, and despite that this project works pretty good
I decided that it is time for major refactoring which will ensure easier maintenance further down the road.And here we are, below you can find a list of things that have been done.
1. Complete dependencies overhaul and proper fragmentation.
2. Alembic based database migrations.
3. Database reimplementation (replaced Flask-SQLAlchemy with sqlalchemy) + custom pagination
(since sqlalchemy doesn't support page base pagination).
4. New session based authentication system (removed Flask-Login).
5. Background tasks refactoring (dev mode app auto reloading finally works).
6. Proper logging implementation.
7. Styling tweaks + UI bugfixes.
8. Internal app data flow refactoring, to improve readability.
9. Improved Docker services architecture.### Production Setup
1. Clone this repo.
2. Create `.env` config file and change config values (`DB_PATH` and `LOGS_PATH`).
```
cp .env.template .env
```
3. Run docker container.
```
sudo docker compose up -d
```### Dev Setup
1. Clone this repo.
2. Generate `.env` config file.
```
cp .env.template .env
```
3. Change `REDIS_SERVER_ADDRESS` in `.env` to `127.0.0.1`
4. Install development dependencies
```
pip3 install -r requirements/requirements-dev.txt
```
5. Run DEV docker-compose.
```
sudo docker compose -f docker-compose-dev.yml up
```### Database Migrations
```
python3 migrate.py
```### Accounts Management
1. Bash into container.
```
sudo docker container exec -it ao3_web_reader bash
```
2. Run accounts manager cli `python3 users_manager_cli.py`.### Tests
App contains some example tests for available routes. To run them:
```
pytest -v tests/
```### Extras
Back when I was learning Django I created this repo [reimplementation](https://github.com/zNitche/ao3-web-reader-django).Keep in mind that it hasn't been updated since it was finished and might not work anymore.