Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/raffomania/archive.observer

🔭 AskHistorians Archive Viewer
https://github.com/raffomania/archive.observer

archiving pushshift reddit rust static

Last synced: 2 days ago
JSON representation

🔭 AskHistorians Archive Viewer

Host: GitHub
URL: https://github.com/raffomania/archive.observer
Owner: raffomania
License: mit
Created: 2023-06-30T14:48:06.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2023-09-12T10:07:37.000Z (about 1 year ago)
Last Synced: 2023-12-17T16:40:03.801Z (11 months ago)
Topics: archiving, pushshift, reddit, rust, static
Language: Rust
Homepage: https://archive.observer
Size: 189 KB
Stars: 28
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE-MIT

Awesome Lists containing this project

README

# AskHistorians Archive
[![CI](https://github.com/raffomania/aharc/workflows/CI/badge.svg)](https://github.com/raffomania/aharc/actions?query=workflow%3ACI)
[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/raffomania/aharc/blob/main/LICENSE-MIT)

[https://archive.observer](https://ask-historians-archive.netlify.app)

This is an *unofficial* archive of the subreddit [/r/AskHistorians](https://old.reddit.com/r/AskHistorians/). It's
ad-free, works on
mobile, loads fast, and doesn't need JS.

## Features

- Full-text search
- Collapse comments
- No AutoMod comments
- Only shows posts with actual answers

Posts are sorted in reverse chronological order and filtered to show only those with accepted answers. Since it's hosted
on the Netlify
free tier, it only contains posts made after {} for now.

## Project Status

The project is pretty much done for now. I'm actively reviewing pull requests, and I might revisit it in the future if
enough people request changes :) Here are
some ideas for improvements:

- Show nested comments
- Other ways to sort posts, like upvotes, number of answers, chronological order
- Option to hide read posts
- Add posts from other subreddits like AskScience

## Your Own Subreddit Archive

Requires `just`, `rustup` and `npm` to bootstrap all tools and configuration.

```bash
cargo install just
just init # setup repo, install required tools
```

This project takes ZSTD-compressed JSON dumps as found in the pushshift archives. Files need to be pre-processed before the tool can use them:

```bash
bin/preprocess-dump.sh .zst input/submissions.json
```

To generate the files for a website:
```bash
just run
```

To show a preview of your site, you can install [miniserve](https://github.com/svenstaro/miniserve/) and run
```bash
just serve
```
to show a preview of the generated files.

To test:
```bash
just test
```

Before committing work:
```bash
just pre-commit
```

To see all available commands:
```bash
just list
```

## Similar Projects

The [redarc viewer](https://github.com/yakabuff/redarc) allows browsing pushshift dumps, but has a different UI than what I had in mind. It's also not a static site.

## License

This project is licensed under either of:
* MIT license ([LICENSE-MIT] or http://opensource.org/licenses/MIT)

## Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as
defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[LICENSE-MIT]: ./LICENSE-MIT