https://github.com/rhnfzl/reddit-stash
Take the control of your reddit saved post and the comments with context locally or on dropbox
https://github.com/rhnfzl/reddit-stash
docker dropbox-api nas-storage python python-reddit reddit reddit-saved-posts reddit-scraper
Last synced: 5 months ago
JSON representation
Take the control of your reddit saved post and the comments with context locally or on dropbox
- Host: GitHub
- URL: https://github.com/rhnfzl/reddit-stash
- Owner: rhnfzl
- License: mit
- Created: 2024-08-25T12:35:34.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2026-02-08T23:17:31.000Z (5 months ago)
- Last Synced: 2026-02-09T03:07:08.531Z (5 months ago)
- Topics: docker, dropbox-api, nas-storage, python, python-reddit, reddit, reddit-saved-posts, reddit-scraper
- Language: Python
- Homepage:
- Size: 938 KB
- Stars: 83
- Watchers: 2
- Forks: 34
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Reddit Stash: Automatically Save Reddit Posts and Comments to Local or Dropbox
[](https://www.python.org/)
[](https://github.com/features/actions)
[](https://github.com/rhnfzl/reddit-stash/pkgs/container/reddit-stash)
[](https://www.dropbox.com/)
[](https://www.reddit.com/dev/api/)
[](LICENSE)
> **Reddit API Policy Change (November 2025):** Reddit now requires pre-approval to create new API apps. **If you already have API credentials, they still work normally.** New users must apply at [Reddit's API Request Form](https://support.reddithelp.com/hc/en-us/requests/new?ticket_form_id=14868593862164) (expect 2-4 weeks for approval). Alternatively, you can use the [GDPR Export Mode](#gdpr-only-mode-no-api-credentials-needed) to create an index of your saved content without any API credentials. See [Getting API Credentials](#getting-api-credentials) for full details.
**Reddit Stash** is a Python script designed to help you effortlessly back up your Reddit **saved/ posted/ upvoted** posts and comments to Dropbox or your local machine. Utilizing GitHub Actions, this script runs every 2 hours during peak hours and twice during off-peak hours, automating the process of archiving your Reddit data in Dropbox after a simple setup.
## 📋 What You Get
When Reddit Stash runs successfully, your saved content is organized by subreddit in a clean folder structure and stored as markdown files:
```
reddit/
├── r_AskReddit/
│ ├── POST_abcd123.md # Your posted content
│ ├── COMMENT_efgh456.md # Your comments
│ ├── SAVED_POST_xyz789.md # Posts you saved
│ └── SAVED_COMMENT_def012.md # Comments you saved
├── r_ProgrammerHumor/
│ ├── UPVOTE_POST_ghi345.md # Posts you upvoted
│ ├── UPVOTE_COMMENT_mno901.md # Comments you upvoted
│ └── GDPR_POST_jkl678.md # From GDPR export (if enabled)
├── gdpr_data/ # GDPR CSV files (if processing enabled)
│ ├── saved_posts.csv
│ └── saved_comments.csv
└── file_log.json # Tracks processed items
```
Each post and comment is formatted with:
- Original title and content
- Author information
- Post/comment URL
- Timestamp
- Subreddit details
- Any images or links from the original post
When Reddit Stash completes processing, you'll receive detailed storage information:
```
Processing completed. 150 items processed, 25 items skipped.
Markdown file storage: 12.45 MB
Media file storage: 89.32 MB
Total combined storage: 101.77 MB
```
This gives you clear visibility into:
- **Text content storage**: How much space your saved posts and comments use
- **Media storage**: How much space downloaded images and videos use
- **Total storage**: Combined space used for your complete Reddit archive
## Table of Contents
- [What You Get](#-what-you-get)
- [How It Works](#how-it-works)
- [Quick Start](#-quick-start)
- [Setup Method Comparison](#setup-method-comparison)
- [Key Features](#key-features)
- [Why Use Reddit Stash](#-why-use-reddit-stash)
- [Setup](#setup)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [GitHub Action Installation](#github-action-installation-recommended)
- [Local Installation](#local-installation)
- [Docker Installation](#docker-installation)
- [Setup Verification Checklist](#setup-verification-checklist)
- [Configuration](#configuration)
- [Configuration Quick Reference](#configuration-quick-reference)
- [Configuration Sections](#configuration-sections)
- [[Settings] - Core Application Behavior](#settings---core-application-behavior)
- [[Configuration] - Reddit API Credentials](#configuration---reddit-api-credentials)
- [[Media] - Advanced Media Download System](#media---advanced-media-download-system)
- [[Imgur] - Imgur API Configuration](#imgur---imgur-api-configuration)
- [[Recovery] - Content Recovery System](#recovery---content-recovery-system)
- [[Retry] - Retry Queue Configuration](#retry---retry-queue-configuration)
- [Important Configuration Notes](#important-configuration-notes)
- [Getting API Credentials](#getting-api-credentials)
- [GDPR-Only Mode (No API Credentials Needed)](#gdpr-only-mode-no-api-credentials-needed)
- [Setting Up Reddit Environment Variables](#setting-up-reddit-environment-variables)
- [Setting Up Dropbox App](#setting-up-dropbox-app)
- [Settings Index (Alphabetical)](#settings-index-alphabetical)
- [Docker Environment Variables](#docker-environment-variables)
- [Alternative Scheduling: External Cron Setup](#alternative-scheduling-external-cron-setup)
- [Important Notes](#important-notes)
- [About Unsaving](#important-note-about-unsaving)
- [GDPR Data Processing](#gdpr-data-processing)
- [File Organization and Utilities](#file-organization-and-utilities)
- [Frequently Asked Questions](#frequently-asked-questions)
- [Troubleshooting](#-troubleshooting)
- [Security Considerations](#-security-considerations)
- [Contributing](#contributing)
- [Acknowledgement](#acknowledgement)
- [Project Status](#project-status)
- [Future Enhancements](#future-enhancements)
- [License](#license)
## How It Works
```mermaid
graph LR
A[Reddit API] -->|Fetch Content| B[Reddit Stash Script]
B -->|Save as Markdown| C[Local Storage]
B -->|Check Settings| D{Save Type}
D -->|SAVED| E[Saved Posts/Comments]
D -->|ACTIVITY| F[User Posts/Comments]
D -->|UPVOTED| G[Upvoted Content]
D -->|ALL| H[All Content Types]
C -->|Optional| I[Dropbox Upload]
J[GDPR Export] -->|Optional| B
```
### Workflow Summary
1. **Data Collection**:
- The script connects to Reddit's API to fetch your saved, posted, or upvoted content
- Optionally, it can process your GDPR export data for a complete history
2. **Processing & Organization**:
- Content is processed based on your settings (SAVED, ACTIVITY, UPVOTED, or ALL)
- Files are organized by subreddit in a clean folder structure
- A log file tracks all processed items to avoid duplicates
3. **Storage Options**:
- Local storage: Content is saved as markdown files on your machine
- Cloud storage: Optional integration with Dropbox for backup
4. **Deployment Methods**:
- **GitHub Actions**: Fully automated with scheduled runs and Dropbox integration
- **Local Installation**: Run manually or schedule with cron jobs on your machine
- **Docker**: Run in a containerized environment with optional volume mounts
The script is designed to be flexible, allowing you to choose how you collect, process, and store your Reddit content.
## ⚡ Quick Start
For those who want to get up and running quickly, here's a streamlined process:
### Option 1: GitHub Actions (Easiest Setup)
1. Fork this repository.
2. Set up the required secrets in your GitHub repository:
- From Reddit: `REDDIT_CLIENT_ID`, `REDDIT_CLIENT_SECRET`, `REDDIT_USERNAME`, `REDDIT_PASSWORD`
- From Dropbox: `DROPBOX_APP_KEY`, `DROPBOX_APP_SECRET`, `DROPBOX_REFRESH_TOKEN`
3. Manually trigger the workflow from the Actions tab.
### Option 2: Local Installation
1. Clone the repository:
```bash
git clone https://github.com/YOUR_USERNAME/reddit-stash.git
cd reddit-stash
```
Replace `YOUR_USERNAME` with your GitHub username (or use `rhnfzl` if using the original repository).
2. Install dependencies:
```bash
pip install -r requirements.txt
```
3. Set up your environment variables and run:
```bash
python reddit_stash.py
```
### Option 3: Docker Installation
1. Build the Docker image:
```bash
docker build -t reddit-stash .
```
2. Run with your environment variables:
```bash
docker run -it \
-e REDDIT_CLIENT_ID=your_client_id \
-e REDDIT_CLIENT_SECRET=your_client_secret \
-e REDDIT_USERNAME=your_username \
-e REDDIT_PASSWORD=your_password \
-e DROPBOX_APP_KEY=your_dropbox_key \
-e DROPBOX_APP_SECRET=your_dropbox_secret \
-e DROPBOX_REFRESH_TOKEN=your_dropbox_token \
-e IMGUR_CLIENT_ID=your_imgur_client_id \
-e IMGUR_CLIENT_SECRET=your_imgur_client_secret \
-v $(pwd)/reddit:/app/reddit \
reddit-stash
```
*Note: IMGUR_CLIENT_ID and IMGUR_CLIENT_SECRET are optional - Imgur API registration is permanently closed, only users with existing applications can use these*
For detailed setup instructions, continue reading the [Setup](#setup) section.
### Setup Method Comparison
| Feature | GitHub Actions | Local Installation | Docker |
|---------|---------------|-------------------|--------|
| **Ease of Setup** | ⭐⭐⭐ (Easiest) | ⭐⭐ | ⭐⭐ |
| **Automation** | ✅ Runs on schedule | ✅ Manual control or cron | ✅ Built-in scheduling support |
| **Requirements** | GitHub account | Python 3.10-3.12 | Docker |
| **Data Storage** | Dropbox required | Local or Dropbox | Local or Dropbox |
| **Maintenance** | Minimal | More hands-on | Low to Medium |
| **Privacy** | Credentials in GitHub secrets | Credentials on local machine | Credentials in container |
| **Best For** | Set & forget users | Power users with customization needs | Containerized environments & flexible scheduling |
## Key Features
- 🤖 **Automated Reddit Backup:** Automatically retrieves saved posts and comments from Reddit, even your posts and comments if you set it up.
- 🔄 **Flexible Storage Options:** Allows for flexible saving options (all activity or only saved items) via `settings.ini`.
- 📦 **Dropbox Integration:** Downloads and uploads the files to Dropbox for storage.
- 📝 **Markdown Support:** Saves the content as markdown files.
- 🔍 **File Deduplication:** Uses intelligent file existence checking to avoid re-downloading content.
- ⏱️ **Rate Limit Management:** Implements dynamic sleep timers to respect Reddit's API rate limits.
- 🔒 **GDPR Data Processing:** Optional processing of Reddit's GDPR export data.
- 🖼️ **Enhanced Media Downloads:** Download images, videos, and other media with dramatically improved success rates (~80% vs previous ~10%), featuring intelligent fallback systems and modern web compatibility.
- 🔄 **Content Recovery System:** 4-provider cascade for failed downloads (Wayback Machine, PullPush.io, Reddit Previews, Reveddit) with SQLite caching and automatic retry across runs.
## 🎯 Why Use Reddit Stash
Reddit Stash was designed with specific use cases in mind:
### 1. Overcome Reddit's Limitations
Reddit only shows your most recent 1000 saved posts. With Reddit Stash, you can save everything and go beyond this limitation.
### 2. Create a Personal Knowledge Base
Many users save technical posts, tutorials, or valuable discussions on Reddit. Reddit Stash helps you build a searchable archive of this knowledge.
### 3. Preserve Content Before It's Deleted
Reddit posts and comments can be deleted by users or moderation. Reddit Stash preserves this content in your personal archive.
### 4. Access Your Content Offline
All of your saved posts are available locally in markdown format, making them easily accessible even without an internet connection.
### 5. Integration with Note-Taking Systems
Since content is saved in markdown, you can easily import it into note-taking systems like Obsidian, Notion, or any markdown-compatible tool.
### 6. Media Preservation
Beyond text, Reddit Stash can download and preserve images, videos, and other media from posts, ensuring you have complete archives even if external hosting services go offline.
## Setup
### Prerequisites
- ✅ Python 3.10-3.12 (Python 3.12 recommended for best performance)
- 🔑 Reddit API credentials
- 📊 A Dropbox account with an API token
### Installation
Before proceeding with any installation method, ensure that you have set the Reddit environment variables. Follow [Reddit API guide](#setting-up-reddit-environment-variables) to create a Reddit app and obtain the necessary credentials.
#### GitHub Action Installation (Recommended)
**Note:** The following process requires the [Dropbox App setup](#setting-up-dropbox-app). The GitHub Actions workflow runs the script every 2 hours during peak hours (6:00-21:00 UTC) and twice during off-peak hours (23:00 and 3:00 UTC), uploading the files to Dropbox. The workflow is defined in `.github/workflows/reddit_scraper.yml`.
1. **Fork this repository**.
2. **Set Up Secrets:**
- Go to your forked repository's **Settings** > **Secrets and variables** > **Actions** > Click on **New repository secret**.
- Add the following secrets individually:
- `REDDIT_CLIENT_ID`
- `REDDIT_CLIENT_SECRET`
- `REDDIT_USERNAME`
- `REDDIT_PASSWORD`
For Dropbox Setup
- `DROPBOX_APP_KEY`
- `DROPBOX_APP_SECRET`
- `DROPBOX_REFRESH_TOKEN`
For Enhanced Media Downloads (Optional - Imgur API registration is permanently closed)
- `IMGUR_CLIENT_ID` (only if you already have an existing Imgur application)
- `IMGUR_CLIENT_SECRET` (only if you already have an existing Imgur application)
- Enter the respective secret values without any quotes.
After adding all secrets: .
3. **Manually Trigger the Workflow**:
- Go to the **Actions** tab > Select the **Reddit Stash Scraper** from the list on the left > Click **Run workflow** > Select the branch `main` > Click the green **Run workflow** button. The workflow will then be triggered, and you can monitor its progress in the Actions tab. Upon successful completion, you should see the Reddit folder in your Dropbox.
4. The workflow runs automatically on a schedule:
- Every 2 hours during *peak hours* (6:00-21:00 UTC)
- Twice during *off-peak hours* (23:00 and 3:00 UTC)
- You can adjust these times in the workflow file to match your timezone if needed.
5. **Additional Workflows**: The repository includes automated workflows for maintenance and testing:
- `python-compatibility.yml`: Tests compatibility across Python versions 3.10-3.12
#### Local Installation
1. **Clone this repository**:
```
git clone https://github.com/YOUR_USERNAME/reddit-stash.git
cd reddit-stash
```
Replace `YOUR_USERNAME` with your GitHub username (or use `rhnfzl` if using the original repository).
2. Install the required Python packages:
```
pip install -r requirements.txt
```
3. Setup the [Dropbox App setup](#setting-up-dropbox-app). Skip it if you don't want to setup the dropbox and only want to save the file locally in your system.
4. Edit the settings.ini file, here is [how to](#settingsini-file)
5. Set Environment Variables (Optional but preferred):
For macOS and Linux:
```
export REDDIT_CLIENT_ID='your_client_id'
export REDDIT_CLIENT_SECRET='your_client_secret'
export REDDIT_USERNAME='your_username'
export REDDIT_PASSWORD='your_password'
# Optional, if you need dropbox locally
export DROPBOX_APP_KEY='dropbox-app-key'
export DROPBOX_APP_SECRET='dropbox-secret-key'
export DROPBOX_REFRESH_TOKEN='dropbox-secret-key'
# Optional, for enhanced Imgur downloading (if you have existing API access)
export IMGUR_CLIENT_ID='your_imgur_client_id'
export IMGUR_CLIENT_SECRET='your_imgur_client_secret'
```
For Windows:
```
set REDDIT_CLIENT_ID='your_client_id'
set REDDIT_CLIENT_SECRET='your_client_secret'
set REDDIT_USERNAME='your_username'
set REDDIT_PASSWORD='your_password'
# Optional, if you need dropbox locally
set DROPBOX_APP_KEY='dropbox-app-key'
set DROPBOX_APP_SECRET='dropbox-secret-key'
set DROPBOX_REFRESH_TOKEN='dropbox-secret-key'
# Optional, for enhanced Imgur downloading (if you have existing API access)
set IMGUR_CLIENT_ID='your_imgur_client_id'
set IMGUR_CLIENT_SECRET='your_imgur_client_secret'
```
You can verify the setup with:
```
echo $REDDIT_CLIENT_ID
echo $REDDIT_CLIENT_SECRET
echo $REDDIT_USERNAME
echo $REDDIT_PASSWORD
echo $DROPBOX_APP_KEY
echo $DROPBOX_APP_SECRET
echo $DROPBOX_REFRESH_TOKEN
echo $IMGUR_CLIENT_ID
echo $IMGUR_CLIENT_SECRET
```
6. Usage:
* First-time setup:
```
python reddit_stash.py
```
To upload to Dropbox (optional):
```
python dropbox_utils.py --upload
```
* Subsequent runs, as per your convenience:
1. Download from Dropbox (optional):
```
python dropbox_utils.py --download
```
2. Process Reddit saved items:
```
python reddit_stash.py
```
3. Upload to Dropbox (optional):
```
python dropbox_utils.py --upload
```
#### Docker Installation
🐳 **Pre-built Images Available!** No build required - pull and run directly from GitHub Container Registry.
**📁 Important: File Storage Location**
When running via Docker, files are downloaded to `/app/reddit/` **inside the container**. Without a volume mount, these files are **lost when the container stops**.
To persist your downloaded Reddit content, you **must** mount a volume:
- **Named Volume** (recommended): `-v reddit-data:/app/reddit` - Docker manages storage, data persists across container restarts
- **Bind Mount** (direct access): `-v $(pwd)/reddit:/app/reddit` - Files stored directly on your host machine at `./reddit/`
- **NAS Path**: `-v /volume1/reddit-stash:/app/reddit` - Store on your NAS at a specific location
##### Option 1: Use Pre-built Images (Recommended for NAS/HomeLab)
Pull pre-built multi-platform images (AMD64/ARM64) from GitHub Container Registry:
```bash
# Pull the latest stable image
docker pull ghcr.io/rhnfzl/reddit-stash:latest
# Run with your credentials
docker run -d \
--name reddit-stash \
-v reddit-data:/app/reddit \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
ghcr.io/rhnfzl/reddit-stash:latest
```
**Available Image Tags:**
| Tag | Description | Use Case |
|-----|-------------|----------|
| `latest` | Latest stable from main branch | Production deployments |
| `develop` | Development version | Testing new features |
| `py3.10-latest`, `py3.11-latest`, `py3.12-latest` | Python-specific versions | Specific Python requirements |
| `v1.0.0` | Semantic version tags | Version pinning |
| `sha-abc123` | Commit-specific builds | Reproducible deployments |
**Platform Support:**
- ✅ **AMD64** (x86_64) - Standard x86 systems
- ✅ **ARM64** - Raspberry Pi, ARM-based NAS devices
**NAS/HomeLab Compatibility:**
- Synology DSM (Container Manager)
- QNAP (Container Station)
- TrueNAS SCALE
- unRAID (Community Applications)
- OpenMediaVault (Docker plugin)
- Proxmox (LXC/Docker)
- Portainer, Yacht, CasaOS, Dockge
**Periodic execution with pre-built image:**
```bash
docker run -d \
--name reddit-stash \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e SCHEDULE_MODE='periodic' \
-e SCHEDULE_INTERVAL='7200' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
---
#### Docker Compose (Recommended for NAS/HomeLab)
Docker Compose is the recommended deployment method for NAS and HomeLab environments. It provides easier management, configuration persistence, and works seamlessly with Portainer, Yacht, CasaOS, and other GUI tools.
**1. Create docker-compose.yml**
**Basic Setup (Local-only with Periodic Execution):**
```yaml
version: '3.8'
services:
reddit-stash:
image: ghcr.io/rhnfzl/reddit-stash:latest
container_name: reddit-stash
restart: unless-stopped
environment:
- REDDIT_CLIENT_ID=${REDDIT_CLIENT_ID}
- REDDIT_CLIENT_SECRET=${REDDIT_CLIENT_SECRET}
- REDDIT_USERNAME=${REDDIT_USERNAME}
- REDDIT_PASSWORD=${REDDIT_PASSWORD}
- SCHEDULE_MODE=periodic
- SCHEDULE_INTERVAL=7200
volumes:
- reddit-data:/app/reddit
volumes:
reddit-data:
```
**2. Create .env file**
Create a `.env` file in the same directory as your `docker-compose.yml`:
```env
REDDIT_CLIENT_ID=your_client_id_here
REDDIT_CLIENT_SECRET=your_client_secret_here
REDDIT_USERNAME=your_reddit_username
REDDIT_PASSWORD=your_reddit_password
```
**3. Run with Docker Compose**
```bash
# Start the service
docker-compose up -d
# View logs
docker-compose logs -f
# Stop the service
docker-compose down
# Update to latest image
docker-compose pull && docker-compose up -d
```
**With Dropbox Sync:**
```yaml
version: '3.8'
services:
reddit-stash:
image: ghcr.io/rhnfzl/reddit-stash:latest
container_name: reddit-stash
restart: unless-stopped
environment:
- REDDIT_CLIENT_ID=${REDDIT_CLIENT_ID}
- REDDIT_CLIENT_SECRET=${REDDIT_CLIENT_SECRET}
- REDDIT_USERNAME=${REDDIT_USERNAME}
- REDDIT_PASSWORD=${REDDIT_PASSWORD}
- DROPBOX_APP_KEY=${DROPBOX_APP_KEY}
- DROPBOX_APP_SECRET=${DROPBOX_APP_SECRET}
- DROPBOX_REFRESH_TOKEN=${DROPBOX_REFRESH_TOKEN}
- SCHEDULE_MODE=periodic
- SCHEDULE_INTERVAL=7200
volumes:
- reddit-data:/app/reddit
volumes:
reddit-data:
```
Add to your `.env`:
```env
DROPBOX_APP_KEY=your_dropbox_app_key
DROPBOX_APP_SECRET=your_dropbox_app_secret
DROPBOX_REFRESH_TOKEN=your_dropbox_refresh_token
```
**With Imgur API (for Better Rate Limits):**
```yaml
version: '3.8'
services:
reddit-stash:
image: ghcr.io/rhnfzl/reddit-stash:latest
container_name: reddit-stash
restart: unless-stopped
environment:
- REDDIT_CLIENT_ID=${REDDIT_CLIENT_ID}
- REDDIT_CLIENT_SECRET=${REDDIT_CLIENT_SECRET}
- REDDIT_USERNAME=${REDDIT_USERNAME}
- REDDIT_PASSWORD=${REDDIT_PASSWORD}
- IMGUR_CLIENT_ID=${IMGUR_CLIENT_ID}
- SCHEDULE_MODE=periodic
- SCHEDULE_INTERVAL=7200
volumes:
- reddit-data:/app/reddit
volumes:
reddit-data:
```
Add to your `.env`:
```env
IMGUR_CLIENT_ID=your_imgur_client_id
```
**Full-Featured Example:**
```yaml
version: '3.8'
services:
reddit-stash:
image: ghcr.io/rhnfzl/reddit-stash:latest
container_name: reddit-stash
restart: unless-stopped
environment:
# Reddit credentials
- REDDIT_CLIENT_ID=${REDDIT_CLIENT_ID}
- REDDIT_CLIENT_SECRET=${REDDIT_CLIENT_SECRET}
- REDDIT_USERNAME=${REDDIT_USERNAME}
- REDDIT_PASSWORD=${REDDIT_PASSWORD}
# Dropbox sync (optional)
- DROPBOX_APP_KEY=${DROPBOX_APP_KEY}
- DROPBOX_APP_SECRET=${DROPBOX_APP_SECRET}
- DROPBOX_REFRESH_TOKEN=${DROPBOX_REFRESH_TOKEN}
# Imgur API (optional - for better rate limits)
- IMGUR_CLIENT_ID=${IMGUR_CLIENT_ID}
# Scheduling
- SCHEDULE_MODE=periodic
- SCHEDULE_INTERVAL=7200
volumes:
- reddit-data:/app/reddit
# Optional: mount custom settings.ini
# - ./settings.ini:/app/settings.ini:ro
deploy:
resources:
limits:
cpus: '1.0'
memory: 512M
volumes:
reddit-data:
```
**Using with Portainer:**
1. Go to **Stacks** → **Add stack**
2. Name your stack (e.g., `reddit-stash`)
3. Paste one of the compose examples above
4. Scroll down to **Environment variables** and add your credentials:
- `REDDIT_CLIENT_ID` = your_client_id
- `REDDIT_CLIENT_SECRET` = your_client_secret
- `REDDIT_USERNAME` = your_username
- `REDDIT_PASSWORD` = your_password
- (Add others as needed)
5. Click **Deploy the stack**
**Bind Mounts for Specific Paths:**
If you prefer to use a specific directory instead of a named volume, use bind mounts:
```yaml
volumes:
# Named volume (recommended)
- reddit-data:/app/reddit
# OR bind mount to specific path
# Synology:
# - /volume1/docker/reddit-stash:/app/reddit
# unRAID:
# - /mnt/user/appdata/reddit-stash:/app/reddit
# Generic:
# - ./reddit:/app/reddit
```
---
#### Additional Docker CLI Examples
**With Dropbox Sync (Single Run):**
```bash
docker run --rm \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e DROPBOX_APP_KEY='your_dropbox_key' \
-e DROPBOX_APP_SECRET='your_dropbox_secret' \
-e DROPBOX_REFRESH_TOKEN='your_dropbox_token' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
**With Dropbox Sync (Periodic):**
```bash
docker run -d \
--name reddit-stash \
--restart unless-stopped \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e DROPBOX_APP_KEY='your_dropbox_key' \
-e DROPBOX_APP_SECRET='your_dropbox_secret' \
-e DROPBOX_REFRESH_TOKEN='your_dropbox_token' \
-e SCHEDULE_MODE='periodic' \
-e SCHEDULE_INTERVAL='7200' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
**With Imgur API for Better Rate Limits:**
```bash
docker run -d \
--name reddit-stash \
--restart unless-stopped \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e IMGUR_CLIENT_ID='your_imgur_client_id' \
-e SCHEDULE_MODE='periodic' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
**With 2FA Authentication:**
If you use Reddit's Two-Factor Authentication, append your 6-digit code to your password:
```bash
docker run -d \
--name reddit-stash \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password:123456' \
-e SCHEDULE_MODE='periodic' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
**With Custom settings.ini:**
```bash
docker run -d \
--name reddit-stash \
--restart unless-stopped \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e SCHEDULE_MODE='periodic' \
-v reddit-data:/app/reddit \
-v /path/to/your/settings.ini:/app/settings.ini:ro \
ghcr.io/rhnfzl/reddit-stash:latest
```
**Custom Interval (e.g., every hour):**
```bash
docker run -d \
--name reddit-stash \
--restart unless-stopped \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e SCHEDULE_MODE='periodic' \
-e SCHEDULE_INTERVAL='3600' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
---
#### Special Operations
**Dropbox Upload Only:**
Upload local content to Dropbox without running the main Reddit scraper:
```bash
docker run --rm \
-e DROPBOX_APP_KEY='your_dropbox_key' \
-e DROPBOX_APP_SECRET='your_dropbox_secret' \
-e DROPBOX_REFRESH_TOKEN='your_dropbox_token' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest \
dropbox_utils.py --upload
```
**Dropbox Download Only:**
Download content from Dropbox to local storage:
```bash
docker run --rm \
-e DROPBOX_APP_KEY='your_dropbox_key' \
-e DROPBOX_APP_SECRET='your_dropbox_secret' \
-e DROPBOX_REFRESH_TOKEN='your_dropbox_token' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest \
dropbox_utils.py --download
```
---
#### Volume Management
**Named Volumes (Recommended for NAS):**
Named volumes are portable across different NAS systems and don't require specific path configurations:
```bash
-v reddit-data:/app/reddit
```
View volume location:
```bash
docker volume inspect reddit-data
```
**Bind Mounts (Specific Paths):**
Use bind mounts when you need data in a specific directory:
```bash
# Synology DSM
-v /volume1/docker/reddit-stash:/app/reddit
# QNAP
-v /share/Container/reddit-stash:/app/reddit
# TrueNAS SCALE
-v /mnt/tank/apps/reddit-stash:/app/reddit
# unRAID
-v /mnt/user/appdata/reddit-stash:/app/reddit
# Generic Linux/macOS
-v /path/to/reddit:/app/reddit
-v $(pwd)/reddit:/app/reddit
```
**Volume Permissions:**
The container runs as UID 1000. If you encounter permission errors with bind mounts:
```bash
sudo chown -R 1000:1000 /path/to/reddit
```
---
#### Advanced Configuration
**Resource Limits for NAS Devices:**
Limit CPU and memory usage to prevent overloading your NAS:
**Docker CLI:**
```bash
docker run -d \
--name reddit-stash \
--restart unless-stopped \
--memory="512m" \
--memory-swap="1g" \
--cpus="1.0" \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e SCHEDULE_MODE='periodic' \
-v reddit-data:/app/reddit \
ghcr.io/rhnfzl/reddit-stash:latest
```
**Docker Compose:**
```yaml
services:
reddit-stash:
# ... other config ...
deploy:
resources:
limits:
cpus: '1.0'
memory: 512M
reservations:
cpus: '0.5'
memory: 256M
```
**Monitoring & Logs:**
```bash
# View real-time logs
docker logs -f reddit-stash
# View last 100 lines
docker logs --tail 100 reddit-stash
# View logs since specific time
docker logs --since 1h reddit-stash
# Monitor resource usage
docker stats reddit-stash
```
**Updating the Container:**
```bash
# Stop and remove old container
docker stop reddit-stash
docker rm reddit-stash
# Pull latest image
docker pull ghcr.io/rhnfzl/reddit-stash:latest
# Recreate with same command or docker-compose up -d
```
Or with Docker Compose:
```bash
docker-compose pull
docker-compose up -d
```
---
#### Platform-Specific Deployment Guides
For detailed step-by-step guides specific to your NAS platform, see **[docs/DOCKER_DEPLOYMENT.md](docs/DOCKER_DEPLOYMENT.md)**:
- **Synology DSM** - Container Manager setup
- **QNAP** - Container Station configuration
- **TrueNAS SCALE** - Apps deployment
- **unRAID** - Docker tab and Community Applications
- **OpenMediaVault** - Docker plugin setup
- **Proxmox** - LXC container with Docker
- **Portainer** - Stack deployment
---
##### Option 2: Build Locally
If you prefer to build the Docker image yourself:
**1. Clone and Build:**
```bash
# Clone the repository
git clone https://github.com/rhnfzl/reddit-stash.git
cd reddit-stash
# Build with default Python 3.12
docker build -t reddit-stash:local .
# Or build with specific Python version
docker build --build-arg PYTHON_VERSION=3.11 -t reddit-stash:py3.11 .
docker build --build-arg PYTHON_VERSION=3.10 -t reddit-stash:py3.10 .
```
**2. Run the Locally Built Image:**
After building, use your local image tag (`reddit-stash:local`) instead of the GitHub Container Registry image (`ghcr.io/rhnfzl/reddit-stash:latest`) in any of the examples from Option 1 above.
**Example:**
```bash
docker run -d \
--name reddit-stash \
--restart unless-stopped \
-e REDDIT_CLIENT_ID='your_client_id' \
-e REDDIT_CLIENT_SECRET='your_client_secret' \
-e REDDIT_USERNAME='your_username' \
-e REDDIT_PASSWORD='your_password' \
-e SCHEDULE_MODE='periodic' \
-v reddit-data:/app/reddit \
reddit-stash:local
```
**For all other usage scenarios** (Dropbox sync, Imgur API, docker-compose, special operations, etc.), refer to the examples in Option 1 above, simply replacing the image name.
---
#### Docker Notes:
- **Python Support**: Build supports Python 3.10, 3.11, and 3.12 (3.12 is default)
- **Security**: The container runs as a non-root user for security
- **Data Persistence**: Data is persisted through a volume mount (`-v $(pwd)/reddit:/app/reddit`) to your local machine
- **Runtime Configuration**: Environment variables must be provided at runtime
- **Flexibility**: The container supports running different scripts (main script, dropbox operations)
- **Interactive Mode**: Use `-it` flags for interactive operation with output visible in your terminal
- **Shell Special Characters**: Always use single quotes around environment variable values to prevent shell interpretation of special characters (!, &, $, etc.)
- **Execution Modes**:
- **Single execution** (`SCHEDULE_MODE=once`, default): Runs once and exits
- **Periodic execution** (`SCHEDULE_MODE=periodic`): Runs continuously on schedule
- **Scheduling Options**:
- **Default interval**: 2 hours (7200 seconds)
- **Custom interval**: Set `SCHEDULE_INTERVAL` to any value ≥ 60 seconds
- **Graceful shutdown**: Responds to SIGTERM/SIGINT for clean container stops
- **Two Main Storage Modes**:
- **Local-only**: Just Reddit credentials, saves to mounted volume
- **Dropbox sync**: Full credentials for automatic cloud backup
- **Detached Mode**: You can also run in detached mode with `-d` if you prefer:
```bash
docker run -d \
-e REDDIT_CLIENT_ID='your_client_id' \
[other environment variables] \
-v $(pwd)/reddit:/app/reddit \
reddit-stash
```
- **Logging**: Logs are available through Docker's logging system when running in detached mode:
```bash
docker logs
```
### 🍴 Fork Setup Guide
If you've forked this repository to make custom modifications, follow these setup steps:
#### Using Docker with Your Fork
Docker works the same way with your fork - just build locally:
```bash
# Clone your fork
git clone https://github.com/YOUR_USERNAME/reddit-stash.git
cd reddit-stash
# Build your customized version
docker build -t reddit-stash .
# Run your custom build
docker run -it \
-e REDDIT_CLIENT_ID=your_client_id \
-e REDDIT_CLIENT_SECRET=your_client_secret \
-e REDDIT_USERNAME=your_username \
-e REDDIT_PASSWORD=your_password \
-e DROPBOX_APP_KEY=your_dropbox_key \
-e DROPBOX_APP_SECRET=your_dropbox_secret \
-e DROPBOX_REFRESH_TOKEN=your_dropbox_token \
-v $(pwd)/reddit:/app/reddit \
reddit-stash
```
Replace `YOUR_USERNAME` with your GitHub username.
### Setup Verification Checklist
After completing your chosen installation method, verify that everything is working correctly:
#### For GitHub Actions Setup:
- [ ] Repository forked successfully
- [ ] All required secrets added to repository settings
- [ ] Workflow manually triggered at least once
- [ ] Workflow completes without errors (check Actions tab)
- [ ] Reddit folder appears in your Dropbox account
- [ ] Content files are present and readable
#### For Local Installation:
- [ ] Python 3.10-3.12 installed and working (3.12 recommended)
- [ ] Repository cloned successfully
- [ ] Dependencies installed via `pip install -r requirements.txt`
- [ ] Environment variables set correctly
- [ ] Script runs without errors
- [ ] Content saved to specified directory
- [ ] (Optional) Content uploaded to Dropbox if configured
#### For Docker Installation:
- [ ] Docker installed and daemon running
- [ ] Image built successfully
- [ ] Container runs without errors
- [ ] Content appears in mounted volume
- [ ] (Optional) Content uploaded to Dropbox if configured
## Configuration
The `settings.ini` file in the root directory of the project allows you to configure how Reddit Stash operates. This comprehensive configuration system controls everything from basic saving behavior to advanced media processing and content recovery.
### Configuration Quick Reference
Jump to any section or browse the complete settings index:
| Section | Settings Count | Purpose | Jump To |
|---------|----------------|---------|---------|
| **[Settings]** | 8 | Core behavior (save paths, types, file checking) | [↓ View](#settings---core-application-behavior) |
| **[Configuration]** | 4 | Reddit API credentials (use env vars!) | [↓ View](#configuration---reddit-api-credentials) |
| **[Media]** | 12 | Media download controls (images, videos, albums) | [↓ View](#media---advanced-media-download-system) |
| **[Imgur]** | 3 | Imgur API configuration (optional) | [↓ View](#imgur---imgur-api-configuration) |
| **[Recovery]** | 9 | Content recovery system (4-provider cascade) | [↓ View](#recovery---content-recovery-system) |
| **[Retry]** | 7 | Retry queue management (exponential backoff) | [↓ View](#retry---retry-queue-configuration) |
| **Total** | **43 settings** | **Complete system configuration** | [↓ Settings Index](#settings-index-alphabetical) |
**Quick Tips:**
- 🔒 **Security First**: Use environment variables for credentials, not settings.ini
- ⚡ **Performance**: `check_type=LOG`, `max_concurrent_downloads=3-5`
- 💾 **Storage**: Configure `max_image_size`, `max_video_size`, `max_daily_storage_mb`
- 🔄 **Recovery**: Enable all 4 providers for best deleted content recovery
- 📖 **Full Docs**: Each setting includes type, defaults, examples, and trade-offs
---
### Configuration Sections
#### `[Settings]` - Core Application Behavior
```ini
[Settings]
save_directory = reddit/ # Local directory for saved content
dropbox_directory = /reddit # Dropbox folder path
save_type = ALL # Content types to save
check_type = LOG # File existence checking method
unsave_after_download = false # Auto-unsave after downloading
process_gdpr = false # Process GDPR export data
process_api = true # Use Reddit API for content
ignore_tls_errors = false # Bypass SSL certificate validation (use with caution)
```
#### Core Settings Explained:
* **`save_directory`** - Where to save downloaded content locally
- **Type**: String (directory path)
- **Default**: `reddit/`
- **Valid Values**:
- Relative paths: `reddit/`, `./backup/`, `../archives/reddit/`
- Absolute paths: `/home/user/reddit-archive/`, `C:\Users\Name\Documents\reddit\`
- **Behavior**:
- Creates directory if it doesn't exist
- Relative paths are relative to script location
- Subdirectories created automatically for each subreddit
- **Examples**:
```ini
save_directory = reddit/ # Default, creates ./reddit/
save_directory = /var/backups/reddit/ # Absolute Unix path
save_directory = D:\Reddit\ # Absolute Windows path
```
- **Considerations**:
- Ensure sufficient disk space (typical usage: 100MB-10GB+ depending on media)
- Write permissions required for the directory
- Avoid network drives for better performance
* **`dropbox_directory`** - Dropbox cloud storage path
- **Type**: String (Dropbox path)
- **Default**: `/reddit`
- **Valid Values**: Any valid Dropbox path starting with `/`
- **Format Rules**:
- Must start with `/` (Dropbox root)
- Forward slashes only (even on Windows)
- Case-sensitive on Dropbox side
- **Examples**:
```ini
dropbox_directory = /reddit # Root level folder
dropbox_directory = /Backups/reddit # Nested folder
dropbox_directory = /Archives/2024/reddit # Deep nesting
```
- **Notes**:
- Folder created automatically if it doesn't exist
- Only used when Dropbox credentials are configured
- Syncs file_log.json and all content
* **`save_type`** - What content to download from Reddit
- **Type**: String (enum)
- **Default**: `ALL`
- **Valid Values**: `ALL`, `SAVED`, `ACTIVITY`, `UPVOTED` (case-insensitive)
- **Detailed Options**:
| Value | What It Downloads | File Prefixes | Use Case |
|-------|------------------|---------------|----------|
| **`ALL`** | Everything: your posts, comments, saved items, upvoted items | `POST_`, `COMMENT_`, `SAVED_POST_`, `SAVED_COMMENT_`, `UPVOTE_POST_`, `UPVOTE_COMMENT_` | Complete Reddit archive |
| **`SAVED`** | Only items you saved | `SAVED_POST_`, `SAVED_COMMENT_` | Personal knowledge base |
| **`ACTIVITY`** | Only your posts & comments | `POST_`, `COMMENT_` | Your Reddit contributions |
| **`UPVOTED`** | Only upvoted items | `UPVOTE_POST_`, `UPVOTE_COMMENT_` | Track interests |
- **Performance Comparison** (approximate, 1000 items each):
```
ALL: ~15-30 minutes (processes 4 categories)
SAVED: ~5-10 minutes (1 category)
ACTIVITY: ~5-10 minutes (1 category)
UPVOTED: ~5-10 minutes (1 category)
```
- **API Rate Limit Impact**: Reddit limits to ~100 requests/minute, so `ALL` uses 4x more requests
- **Examples**:
```ini
save_type = ALL # Everything
save_type = SAVED # Just saved items
save_type = all # Case doesn't matter
```
* **`check_type`** - How to detect already-downloaded files
- **Type**: String (enum)
- **Default**: `LOG`
- **Valid Values**: `LOG`, `DIR` (case-insensitive)
- **Detailed Comparison**:
| Feature | `LOG` | `DIR` |
|---------|-------|-------|
| **Speed** | ⚡ Very fast (in-memory check) | 🐢 Slower (filesystem scan) |
| **Accuracy** | Requires intact `file_log.json` | Always accurate |
| **Recovery** | Fails if log corrupted/deleted | Self-healing |
| **Scalability** | Excellent (10,000+ files) | Degrades with size |
| **Best For** | GitHub Actions, automated runs | Local use, manual management |
- **`LOG` Mode Details**:
- Uses `file_log.json` in save_directory
- Tracks: filename, timestamp, subreddit, content type
- JSON structure: `{"file_id-subreddit-type-category": {...}}`
- Loads entire log into memory (typically <10MB for 10,000 items)
- **Risk**: If log file deleted, treats all files as new (duplicates)
- **Fix**: If log corrupted, switch to `DIR` temporarily to rebuild
- **`DIR` Mode Details**:
- Scans filesystem for matching files
- Checks filename patterns: `POST_*.md`, `SAVED_POST_*.md`, etc.
- Builds in-memory index during startup
- Performance: ~1 second per 1,000 files
- **Benefits**: No dependency on log file, always correct
- **Drawback**: Slow for large archives (10,000+ files = ~10s startup)
- **Examples**:
```ini
check_type = LOG # Fast, recommended for automation
check_type = DIR # Thorough, recommended for local
```
- **When to Switch**:
- `LOG` → `DIR`: Log file corrupted, seeing duplicate downloads
- `DIR` → `LOG`: Archive stable, want faster processing
* **`unsave_after_download`** - Automatically unsave items after downloading
- **Type**: Boolean
- **Default**: `false`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0` (case-insensitive)
- **⚠️ CRITICAL WARNING**: THIS IS PERMANENT AND IRREVERSIBLE!
- **How It Works**:
1. Downloads post/comment to local file
2. Verifies download successful
3. Unsaves item from Reddit (removes from your saved list)
4. Waits 0.5 seconds (rate limit protection)
5. Continues to next item
- **Use Case - Breaking the 1000-Item Limit**:
- Reddit shows max 1000 saved items in your list
- Older items beyond 1000 are hidden (but still saved on Reddit)
- By unsaving downloaded items, older items "bubble up" into view
- Run script multiple times to access progressively older saves
- **Workflow Example**:
```
Run 1: Download 1000 most recent, unsave them → Access items 1001-2000
Run 2: Download next 1000, unsave them → Access items 2001-3000
Run 3: Download next 1000, unsave them → Access items 3001-4000
Continue until no more items found
```
- **Safety Recommendations**:
1. **First Run**: Keep `false`, verify downloads work correctly
2. **Backup**: Ensure `file_log.json` is backed up (Dropbox/Git)
3. **Test**: Try with `save_type = UPVOTED` first (less critical)
4. **Enable**: Set to `true` only when confident
5. **Monitor**: Check logs for "unsave failed" messages
- **Error Handling**:
- If unsave fails (API error), script continues (doesn't stop)
- Failed unsaves logged but don't retry automatically
- Item remains saved on Reddit, can be re-downloaded (will skip if in log)
- **Examples**:
```ini
unsave_after_download = false # Safe default
unsave_after_download = true # Enable with caution
unsave_after_download = yes # Also valid
```
* **`process_gdpr`** - Process Reddit GDPR export files
- **Type**: Boolean
- **Default**: `false`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **Purpose**: Access complete Reddit history including items beyond 1000-item API limits
- **Requirements**:
- GDPR export CSV files in `{save_directory}/gdpr_data/`
- Expected files: `saved_posts.csv`, `saved_comments.csv`
- **How It Works**:
1. Reads post/comment IDs from CSV files
2. Fetches full content via Reddit API (one API call per item)
3. Saves with `GDPR_POST_` or `GDPR_COMMENT_` prefix
4. Respects rate limits (same 100 req/min as normal processing)
- **GDPR Export Process**:
1. Visit https://www.reddit.com/settings/data-request
2. Request data (takes 2-30 days to process)
3. Download ZIP file when ready
4. Extract `saved_posts.csv` and `saved_comments.csv`
5. Place in `{save_directory}/gdpr_data/`
- **Performance Impact**:
- Processes AFTER regular API content
- Each item requires separate API call
- 1000 GDPR items ≈ 10-15 additional minutes
- **Deduplication**: Items already in log are skipped (no duplicates)
- **Examples**:
```ini
process_gdpr = false # Default, skip GDPR processing
process_gdpr = true # Process GDPR files if present
```
- **Common Issues**:
- "GDPR directory not found" → Create `{save_directory}/gdpr_data/`
- "No CSV files found" → Verify files named exactly `saved_posts.csv`
- Deleted content → GDPR has IDs but content may be deleted (404 errors normal)
* **`process_api`** - Fetch content from Reddit API
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **Purpose**: Control whether to fetch current content from Reddit API
- **Use Cases**:
| `process_api` | `process_gdpr` | Behavior |
|---------------|----------------|----------|
| `true` | `false` | Normal mode: Fetch current saved/posted/upvoted content |
| `true` | `true` | Complete mode: Current content + GDPR history |
| `false` | `true` | GDPR-only: Only process export files (no API calls for current) |
| `false` | `false` | ⚠️ Does nothing (no content fetched) |
- **When to Set `false`**:
- You only want to process GDPR export files
- Testing GDPR processing without affecting API rate limits
- Already ran API processing, want to add GDPR data only
- **Examples**:
```ini
process_api = true # Normal operation
process_api = false # Skip API, only process GDPR
```
* **`ignore_tls_errors`** - Bypass SSL certificate verification
- **Type**: Boolean
- **Default**: `false`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **⚠️ SECURITY WARNING**: Setting to `true` reduces security significantly!
- **What It Does**:
- `false` (default): Validates SSL certificates, rejects expired/invalid/self-signed certs
- `true`: Accepts ANY certificate, including expired, self-signed, or invalid
- **Security Implications**:
- Vulnerable to man-in-the-middle attacks
- No guarantee you're downloading from legitimate source
- Should NEVER be used with sensitive content
- **Legitimate Use Cases** (rare):
- Archiving content from sites with expired certificates
- Corporate networks with self-signed proxy certificates
- Historical content preservation where security less critical
- **What It Affects**:
- Only affects media downloads (images, videos)
- Does NOT affect Reddit API (always uses valid TLS)
- Third-party hosting: Imgur, Gfycat, etc.
- **Warnings Generated**:
- Script prints "⚠️ TLS verification disabled" on startup
- Configuration validator shows security warning
- **Examples**:
```ini
ignore_tls_errors = false # Secure default (recommended)
ignore_tls_errors = true # Insecure, use only if absolutely necessary
```
- **Better Alternatives**:
- Use content recovery system (often has archived copies)
- Manually download problematic images
- Report invalid certificates to site owners
#### `[Configuration]` - Reddit API Credentials
```ini
[Configuration]
client_id = None # Reddit app client ID
client_secret = None # Reddit app client secret
username = None # Reddit username
password = None # Reddit password (or password:2FA_code)
```
#### ⚠️ **CRITICAL SECURITY RECOMMENDATION**
**🔒 USE ENVIRONMENT VARIABLES, NOT settings.ini FOR CREDENTIALS!**
Unless you **absolutely know what you're doing** and have a specific reason to store credentials in settings.ini (like an isolated, encrypted system), **always use environment variables instead**.
**Why Environment Variables Are Safer:**
| Risk | settings.ini | Environment Variables |
|------|-------------|----------------------|
| **Accidental Git Commit** | ❌ VERY HIGH - Credentials exposed publicly | ✅ Safe - Not in version control |
| **File Sharing** | ❌ HIGH - Shared accidentally with settings | ✅ Safe - Not in shared files |
| **Backup Exposure** | ❌ MEDIUM - In all backups/archives | ✅ Safe - Not in file backups |
| **Log Exposure** | ❌ MEDIUM - May appear in logs | ✅ Safer - Separate from logs |
| **Access Control** | ❌ File permissions only | ✅ Better - OS-level protection |
**Real-World Danger Example:**
```
You: Edit settings.ini with credentials
You: Test locally, everything works
You: git add . && git commit -m "Update settings"
You: git push
Result: 🚨 YOUR CREDENTIALS ARE NOW PUBLIC ON GITHUB 🚨
Anyone can: Access your Reddit account, download your data, post as you
```
##### API Configuration Settings:
* **`client_id`** - Reddit application client ID
- **Type**: String or `None`
- **Default**: `None`
- **⚠️ SECURITY**: Use environment variable `REDDIT_CLIENT_ID` instead
- **When to Use settings.ini**:
- Isolated test environment (not connected to internet)
- Encrypted filesystem only you access
- Single-use throwaway credentials
- **Never Use settings.ini If**:
- Repository is public or will be shared
- File will be backed up to cloud
- Multiple people access the system
- You're not 100% sure about security implications
- **Environment Variable Method** (RECOMMENDED):
```bash
# Linux/macOS
export REDDIT_CLIENT_ID='your_client_id_here'
# Windows
set REDDIT_CLIENT_ID=your_client_id_here
```
- **settings.ini Method** (NOT RECOMMENDED):
```ini
client_id = your_client_id_here # ⚠️ DANGEROUS
```
* **`client_secret`** - Reddit application client secret
- **Type**: String or `None`
- **Default**: `None`
- **⚠️ CRITICAL SECURITY**: This is your API password! Use `REDDIT_CLIENT_SECRET` env var
- **Exposure Risk**: CRITICAL - Full API access
- **If Compromised**: Attacker can use your Reddit app credentials
- **NEVER commit this to version control**
- **Environment Variable** (REQUIRED for any shared/public code):
```bash
export REDDIT_CLIENT_SECRET='your_secret_here'
```
* **`username`** - Your Reddit username
- **Type**: String or `None`
- **Default**: `None`
- **Security Level**: Medium (username is public anyway, but indicates which account)
- **Environment Variable** (RECOMMENDED):
```bash
export REDDIT_USERNAME='your_username'
```
- **Note**: Less critical than password, but still recommended to use env var
* **`password`** - Your Reddit account password
- **Type**: String or `None`
- **Default**: `None`
- **⚠️ MAXIMUM SECURITY RISK**: YOUR REDDIT PASSWORD!
- **Exposure = Account Takeover**: Someone with this can log into your Reddit account
- **2FA Format**: If you have two-factor authentication: `your_password:123456`
- **NEVER EVER put this in settings.ini** unless isolated system
- **Environment Variable** (ABSOLUTELY REQUIRED):
```bash
export REDDIT_PASSWORD='your_password'
# With 2FA:
export REDDIT_PASSWORD='your_password:123456'
```
##### Credential Priority (How Reddit Stash Checks):
1. **Environment Variables** (checked first) ← USE THIS
2. **settings.ini values** (fallback) ← AVOID THIS
If environment variable exists, settings.ini value is **completely ignored**.
##### Safe settings.ini Configuration:
**✅ SAFE - Keep credentials as None:**
```ini
[Configuration]
client_id = None
client_secret = None
username = None
password = None
```
**❌ UNSAFE - Credentials in file:**
```ini
[Configuration]
client_id = abc123def456 # ⚠️ WILL BE EXPOSED
client_secret = secret_key_here # ⚠️ CRITICAL RISK
username = your_username # ⚠️ ACCOUNT IDENTIFIER
password = your_password # ⚠️ ACCOUNT TAKEOVER RISK
```
##### When settings.ini Credentials Might Be Acceptable:
Only use settings.ini for credentials if **ALL** of these are true:
- ✅ System is completely isolated (no internet, air-gapped)
- ✅ Filesystem is encrypted
- ✅ Only you have access (no shared users)
- ✅ File will never be committed to version control
- ✅ File will never be backed up to cloud
- ✅ File will never be shared with anyone
- ✅ You fully understand the security implications
- ✅ You're using test/throwaway credentials only
**If you're unsure about even ONE of these, use environment variables!**
##### How to Verify Your Setup is Secure:
```bash
# Check if credentials are in settings.ini
grep -E "client_id|client_secret|username|password" settings.ini
# Safe output (all None):
client_id = None
client_secret = None
username = None
password = None
# Unsafe output (has values):
client_id = abc123 # ⚠️ FIX THIS
```
##### Additional Security Best Practices:
1. **Add settings.ini to .gitignore** (even with None values, for safety)
2. **Use .env files** for local development (also in .gitignore)
3. **Rotate credentials** if you suspect exposure
4. **Use GitHub Secrets** for GitHub Actions (already encrypted)
5. **Never screenshot** settings with credentials
6. **Check git history** if you accidentally committed credentials
##### Setup Guide:
For detailed instructions on setting up environment variables properly, see [Setting Up Reddit Environment Variables](#setting-up-reddit-environment-variables)
#### `[Media]` - Advanced Media Download System
```ini
[Media]
# Global media download controls
download_enabled = true # Master switch for all media downloads
download_images = true # Enable image downloads
download_videos = true # Enable video downloads
download_audio = true # Enable audio downloads
# Image processing settings
thumbnail_size = 800 # Thumbnail dimensions in pixels
max_image_size = 5242880 # Max image file size (5MB in bytes)
create_thumbnails = true # Generate thumbnails for large images
# Video settings
video_quality = high # Video quality preference
max_video_size = 209715200 # Max video file size (200MB in bytes)
# Album settings
download_albums = true # Process image albums/galleries
max_album_images = 50 # Max images per album (0 = unlimited)
# Performance and resource limits
max_concurrent_downloads = 3 # Parallel download streams
download_timeout = 30 # Download timeout in seconds
max_daily_storage_mb = 1024 # Daily storage limit in MB
```
#### Media Settings Explained:
**Global Controls:**
* **`download_enabled`** - Master switch for all media downloads
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**:
- `true`: Enables media download system, attempts to download images/videos/audio
- `false`: Completely disables media downloads, only saves text content (markdown)
- **Performance Impact**:
- Enabled: Adds 5-30 minutes per 1000 posts (depends on media count)
- Disabled: ~80% faster processing, minimal storage usage
- **Storage Impact**:
- Enabled: 50MB-5GB+ depending on content (images are heavy!)
- Disabled: <50MB for 10,000 text-only posts
- **Examples**:
```ini
download_enabled = true # Enable media downloads
download_enabled = false # Text-only archiving
```
* **`download_images`** - Control image downloads
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **Requires**: `download_enabled = true`
- **What It Downloads**:
- Direct image links (i.redd.it, i.imgur.com)
- Reddit galleries
- External images (direct URLs)
- **Success Rate**: ~80% with modern web compatibility
- **Examples**:
```ini
download_images = true # Download all images
download_images = false # Skip images
```
* **`download_videos`** - Control video downloads
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **Requires**: `download_enabled = true`
- **What It Downloads**:
- v.redd.it videos (with audio merging via ffmpeg if available)
- Direct video links (.mp4, .webm, .mov)
- Embedded videos from supported hosts
- **Dependencies**:
- ffmpeg recommended for v.redd.it audio merging
- Without ffmpeg: video-only (no audio track)
- **Storage Warning**: Videos are large! Single video can be 50-500MB
- **Examples**:
```ini
download_videos = true # Download videos
download_videos = false # Skip videos (save storage)
```
* **`download_audio`** - Control audio downloads
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **Requires**: `download_enabled = true`
- **What It Downloads**:
- Audio-only posts (podcasts, music)
- Audio tracks from videos (when separated)
- **Note**: Rarely used by Reddit, most audio is embedded in video
- **Examples**:
```ini
download_audio = true # Download audio files
download_audio = false # Skip audio
```
**Image Processing:**
* **`thumbnail_size`** - Maximum thumbnail dimension in pixels
- **Type**: Integer (positive)
- **Default**: `800`
- **Valid Range**: 100-4096 (recommended: 300-1200)
- **What It Means**:
- Thumbnails are resized to fit within this width × height box
- Aspect ratio is preserved
- If image is 1920×1080 and thumbnail_size=800: → 800×450
- **Storage Impact Per Image**:
- 300px: ~50KB (very small)
- 800px: ~150KB (balanced)
- 1200px: ~300KB (high quality)
- **Use Cases**:
- 300-500: Quick previews, storage-constrained
- 800-1000: Balanced quality/size (recommended)
- 1200-2000: High quality, display on large screens
- **Examples**:
```ini
thumbnail_size = 400 # Small previews
thumbnail_size = 800 # Default, good balance
thumbnail_size = 1200 # High quality
```
* **`max_image_size`** - Maximum image file size in bytes
- **Type**: Integer (positive)
- **Default**: `5242880` (5MB)
- **Valid Range**: 100,000 (100KB) to 52,428,800 (50MB)
- **What It Does**:
- Images larger than this are SKIPPED (not downloaded)
- Not resized - full skip to save bandwidth/storage
- Applies BEFORE download (uses Content-Length header if available)
- **Size Conversion Guide**:
```
1 MB = 1048576 bytes
2 MB = 2097152 bytes
5 MB = 5242880 bytes (default)
10 MB = 10485760 bytes
20 MB = 20971520 bytes
50 MB = 52428800 bytes
```
- **Trade-offs**:
- Small limit (1-2MB): Saves storage, may miss high-res images
- Medium limit (5-10MB): Balanced, gets most images
- Large limit (20-50MB): Complete archive, heavy storage
- **Examples**:
```ini
max_image_size = 1048576 # 1MB limit (minimal)
max_image_size = 5242880 # 5MB limit (default)
max_image_size = 20971520 # 20MB limit (comprehensive)
```
* **`create_thumbnails`** - Generate thumbnail versions of images
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**:
- `true`: Creates smaller preview version alongside original
- `false`: Only saves original full-size image
- **File Naming**:
- Original: `abc123_media.jpg`
- Thumbnail: `abc123_media_thumb.jpg`
- **Storage Cost**: +10-20% additional space
- **Benefits**:
- Fast loading in file browsers
- Preview without opening large files
- Useful for browsing archives offline
- **Processing Cost**: +2-5 seconds per image
- **Examples**:
```ini
create_thumbnails = true # Generate previews
create_thumbnails = false # Original only
```
**Video Processing:**
* **`video_quality`** - Download quality preference for videos
- **Type**: String (enum)
- **Default**: `high`
- **Valid Values**: `high`, `low` (case-insensitive)
- **What It Means**:
| Quality | Resolution | Bitrate | File Size (per min) | Use Case |
|---------|-----------|---------|---------------------|----------|
| **`high`** | 720p-1080p+ | High | 10-50MB/min | Best quality, storage available |
| **`low`** | 360p-480p | Lower | 3-10MB/min | Save bandwidth/storage |
- **How It Works**:
- v.redd.it: Selects from available quality levels
- YouTube: Requests specific quality (if available)
- Gfycat: Downloads HD vs SD version
- **Important**: Not all sources provide multiple qualities
- **Examples**:
```ini
video_quality = high # Best quality (default)
video_quality = low # Save storage
video_quality = HIGH # Case doesn't matter
```
* **`max_video_size`** - Maximum video file size in bytes
- **Type**: Integer (positive)
- **Default**: `209715200` (200MB)
- **Valid Range**: 1,048,576 (1MB) to 1,073,741,824 (1GB)
- **What It Does**: Videos larger than this limit are SKIPPED
- **Size Conversion Guide**:
```
10 MB = 10485760 bytes
50 MB = 52428800 bytes
100 MB = 104857600 bytes
200 MB = 209715200 bytes (default)
500 MB = 524288000 bytes
1 GB = 1073741824 bytes
```
- **Typical Video Sizes**:
- Short clip (30s): 5-20MB
- Medium video (2-5 min): 20-100MB
- Long video (10+ min): 100-500MB
- HD long video: 500MB-2GB
- **Recommendations By Use Case**:
- Storage-constrained: 50-100MB
- Balanced: 200-300MB (default range)
- Complete archive: 500MB-1GB
- **Examples**:
```ini
max_video_size = 52428800 # 50MB (minimal)
max_video_size = 209715200 # 200MB (default)
max_video_size = 524288000 # 500MB (comprehensive)
```
**Album Handling:**
* **`download_albums`** - Process multi-image posts (albums/galleries)
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Handles**:
- Imgur albums (imgur.com/a/XXXXX)
- Reddit galleries (multiple images in one post)
- Other multi-image services
- **File Naming**: `{post_id}_media_001.jpg`, `{post_id}_media_002.jpg`, etc.
- **Storage Impact**: Albums can be 2-100+ images each
- **Examples**:
```ini
download_albums = true # Download all album images
download_albums = false # Skip albums entirely
```
* **`max_album_images`** - Limit images per album
- **Type**: Integer (non-negative)
- **Default**: `50`
- **Valid Values**: `0` (unlimited) or positive integer (1-1000+)
- **What It Does**:
- `0`: Download ALL images in album (no limit)
- `Positive number`: Download only first N images, skip rest
- **Why Limit?**:
- Some albums have 100+ images (massive storage)
- Prevent single post from consuming too much space
- Faster processing
- **Behavior Example**:
- Album has 80 images
- `max_album_images = 50`: Download first 50, skip remaining 30
- `max_album_images = 0`: Download all 80
- **Recommendations**:
- Conservative: 20-30 images
- Balanced: 50 images (default)
- Complete: 0 (unlimited, use with caution)
- **Examples**:
```ini
max_album_images = 10 # First 10 only
max_album_images = 50 # First 50 (default)
max_album_images = 0 # No limit
```
**Performance Controls:**
* **`max_concurrent_downloads`** - Parallel download streams
- **Type**: Integer (positive)
- **Default**: `3`
- **Valid Range**: 1-20 (recommended: 1-10)
- **What It Does**: Number of media files downloaded simultaneously
- **Trade-offs**:
| Value | Speed | CPU Usage | Memory Usage | Network Load | Best For |
|-------|-------|-----------|--------------|--------------|----------|
| **1** | Slow | Low | Low | Light | Slow connections, low-power devices |
| **3** | Moderate | Medium | Medium | Moderate | Default, balanced |
| **5-7** | Fast | High | High | Heavy | Fast connections, powerful machines |
| **10+** | Fastest | Very High | Very High | Very Heavy | Server environments, very fast connections |
- **Limiting Factors**:
- GitHub Actions: 3-5 recommended (shared resources)
- Home internet: Based on bandwidth (3-5 typical)
- Fast connection: 5-10
- **Warning**: Too high can trigger rate limits!
- **Examples**:
```ini
max_concurrent_downloads = 1 # One at a time (safest)
max_concurrent_downloads = 3 # Default (balanced)
max_concurrent_downloads = 5 # Fast (requires good connection)
```
* **`download_timeout`** - Per-file download timeout in seconds
- **Type**: Integer (positive)
- **Default**: `30`
- **Valid Range**: 5-600 (recommended: 15-120)
- **What It Does**: Max time to wait for single file download
- **What Happens on Timeout**:
- Download cancelled
- Item added to retry queue
- Script continues to next file
- **Recommendations By File Type**:
- Images only: 15-30 seconds
- Mixed (images + small videos): 30-60 seconds (default range)
- Large videos: 60-300 seconds
- **Network Speed Considerations**:
- Slow connection (<1 Mbps): 60-120 seconds
- Medium (1-10 Mbps): 30-60 seconds
- Fast (10+ Mbps): 15-30 seconds
- **Examples**:
```ini
download_timeout = 15 # Fast timeout (risk more failures)
download_timeout = 30 # Default (balanced)
download_timeout = 120 # Patient (for large files/slow networks)
```
* **`max_daily_storage_mb`** - Daily storage consumption limit in megabytes
- **Type**: Integer (positive)
- **Default**: `1024` (1GB)
- **Valid Range**: 10-100,000+ (10MB to 100GB+)
- **What It Does**:
- Tracks total storage used for media per run
- Stops downloading media when limit reached
- Text content (markdown) still saved
- Resets on next script run (not calendar day)
- **Why Use It**:
- Prevent unexpected storage exhaustion
- Control cloud storage costs (Dropbox)
- GitHub Actions storage limits
- Predictable resource usage
- **Size Planning Guide**:
```
100 MB: ~500-1000 images OR ~2-5 short videos
500 MB: ~2500-5000 images OR ~10-25 videos
1 GB: ~5000-10000 images OR ~20-50 videos (default)
5 GB: ~25000+ images OR 100+ videos
10 GB: Complete large archive
```
- **Recommendations By Use Case**:
- Testing: 100-200MB
- GitHub Actions (free): 500-1000MB
- Home backup: 2000-5000MB (2-5GB)
- Complete archive: 10000+ MB (10GB+)
- **Examples**:
```ini
max_daily_storage_mb = 100 # Testing/minimal
max_daily_storage_mb = 1024 # 1GB default
max_daily_storage_mb = 5120 # 5GB comprehensive
max_daily_storage_mb = 0 # No limit (use with caution!)
```
#### `[Imgur]` - Imgur API Configuration
```ini
[Imgur]
# Optional: Comma-separated client IDs for rate limit rotation
client_ids = None # Multiple Imgur client IDs
client_secrets = None # Corresponding client secrets
recover_deleted = true # Attempt recovery of deleted content
```
#### Imgur Settings Explained:
⚠️ **CRITICAL NOTE**: Imgur API registration permanently closed to new users in May 2024. These settings only work if you already have existing Imgur application credentials.
* **`client_ids`** - Imgur application client IDs for API access
- **Type**: String (comma-separated list in settings.ini, single value in env var)
- **Default**: `None`
- **Valid Values**:
- `None`: No Imgur API access (uses fallback methods)
- Single ID: `abc123def456`
- Multiple IDs (settings.ini only): `id1,id2,id3` (for rate limit rotation)
- **Format Rules**:
- No spaces around commas: `id1,id2,id3` ✅
- With spaces (invalid): `id1, id2, id3` ❌
- Each ID is alphanumeric, typically 15 characters
- **What It Does**:
- Enables official Imgur API access
- Higher rate limits (12,500 requests/day per app)
- Album support and metadata
- Multiple IDs (settings.ini only) rotate to avoid single-app rate limits
- **Without API Access**:
- Falls back to direct HTTP downloads
- Lower success rate (~30-40% vs ~70-80% with API)
- Frequent 429 rate limit errors (expected and normal)
- No album support
- **Configuration Methods**:
| Method | Single ID | Multiple IDs | Best For |
|--------|-----------|--------------|----------|
| **Environment Variable** | ✅ Yes | ❌ No | Most users, single app |
| **settings.ini** | ✅ Yes | ✅ Yes | Advanced: rate limit rotation |
- **How to Get** (only if you registered before May 2024):
1. Go to https://imgur.com/account/settings/apps
2. Select your application
3. Copy the "Client ID" value
- **Examples**:
```ini
# settings.ini - Multiple IDs supported
client_ids = None # No API (most users)
client_ids = abc123def456 # Single app
client_ids = abc123,def456,ghi789 # Multiple apps (rotation) ⚠️ settings.ini only
```
```bash
# Environment variable - Single ID only
export IMGUR_CLIENT_ID='abc123def456' # Single app only
```
- **⚠️ Important**: Multiple client ID rotation is **only supported in settings.ini**, not via environment variables. If you need rotation across multiple Imgur apps, you must use settings.ini configuration.
* **`client_secrets`** - Imgur application client secrets
- **Type**: String (comma-separated list in settings.ini, single value in env var)
- **Default**: `None`
- **Valid Values**:
- `None`: No Imgur API access
- Comma-separated secrets matching `client_ids` order (settings.ini only)
- **MUST MATCH** `client_ids`:
- If `client_ids` has 3 IDs, `client_secrets` must have 3 secrets
- Order matters: `client_ids[0]` pairs with `client_secrets[0]`
- **Format Rules**:
- Same as client_ids: no spaces
- Each secret is alphanumeric, typically 40 characters
- Keep these SECRET (never commit to version control!)
- **Security Warning**:
- These are sensitive credentials
- Environment variables recommended for single app
- Never share or expose publicly
- **Configuration Methods**:
| Method | Single Secret | Multiple Secrets | Recommended |
|--------|--------------|------------------|-------------|
| **Environment Variable** | ✅ Yes | ❌ No | ✅ Most secure |
| **settings.ini** | ✅ Yes | ✅ Yes | ⚠️ Only if multiple apps |
- **Examples**:
```ini
# settings.ini - Multiple secrets supported
client_secrets = None # No API
client_secrets = abcdef1234567890abcdef1234567890abcdef12 # Single
client_secrets = secret1_40chars,secret2_40chars,secret3_40chars # Multiple ⚠️ settings.ini only
```
```bash
# Environment variable - Single secret only (RECOMMENDED)
export IMGUR_CLIENT_SECRET='abcdef1234567890abcdef1234567890abcdef12'
```
- **Validation**: Script checks that count matches `client_ids`
- **⚠️ Note**: Multiple client secrets are **only supported in settings.ini**. For single app (most users), use environment variable for better security.
* **`recover_deleted`** - Attempt recovery of deleted/unavailable Imgur content
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**:
- `true`: When Imgur returns 404, triggers content recovery cascade
- `false`: Skip recovery, treat as permanent failure
- **Recovery Process**:
1. Imgur returns 404 (not found) or 429 (rate limited)
2. System tries Wayback Machine for archived copy
3. Falls back to Reddit preview URLs
4. Checks other recovery providers
5. Caches result (success or failure) to avoid re-trying
- **Success Rates** (for deleted Imgur content):
- Recent deletions (<1 month): ~40-60% recovery
- Older deletions (1-6 months): ~20-40% recovery
- Very old (>6 months): ~10-20% recovery
- Popular images: Higher success (more likely archived)
- **Performance Impact**:
- Adds 5-15 seconds per failed Imgur image
- Only activates on failures (no cost for successful downloads)
- Results cached (subsequent failures instant)
- **When to Disable**:
- You don't care about deleted content
- Want faster processing (skip recovery attempts)
- Already know most Imgur links are dead
- **Examples**:
```ini
recover_deleted = true # Try to recover (default)
recover_deleted = false # Skip recovery, faster
```
#### `[Recovery]` - Content Recovery System
```ini
[Recovery]
# Recovery providers (4-provider cascade)
use_wayback_machine = true # Internet Archive Wayback Machine
use_pushshift_api = true # PullPush.io (Pushshift successor)
use_reddit_previews = true # Reddit's preview/thumbnail system
use_reveddit_api = true # Reveddit deleted content recovery
# Performance settings
timeout_seconds = 10 # Per-provider timeout
cache_duration_hours = 24 # Cache recovery results
# Cache management
max_cache_entries = 10000 # Maximum cached recovery results
max_cache_size_mb = 100 # Cache size limit in MB
cleanup_interval_minutes = 60 # Cache cleanup frequency
enable_background_cleanup = true # Automatic cache maintenance
```
#### Content Recovery Explained:
Reddit Stash includes a sophisticated 4-provider cascade system that attempts to recover deleted, removed, or unavailable content.
#### Recovery Provider Settings:
* **`use_wayback_machine`** - Use Internet Archive Wayback Machine
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**: Archives web snapshots going back to 1996
- **Best For**: Popular content, older deletions, historical preservation
- **Success Rate**: 60-80% for popular content, 20-40% for obscure content
- **Coverage**: Billions of web pages, extensive image archives
- **Rate Limit**: 60 requests/minute (respectful, non-blocking)
- **Response Time**: 2-10 seconds average
- **Examples**:
```ini
use_wayback_machine = true # Enable (recommended)
use_wayback_machine = false # Disable to save time
```
* **`use_pushshift_api`** - Use PullPush.io (Pushshift successor)
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**: Reddit-specific archive of posts and comments
- **Best For**: Deleted/removed Reddit text content, metadata
- **Success Rate**: 40-70% for Reddit content (higher for older content)
- **Coverage**: Reddit posts/comments from 2005-present
- **Rate Limit**: 12 requests/minute (conservative, respects soft limit of 15)
- **Response Time**: 1-3 seconds average
- **Note**: Sometimes slower or down (community-run service)
- **Examples**:
```ini
use_pushshift_api = true # Enable (recommended for Reddit content)
use_pushshift_api = false # Disable if service unavailable
```
* **`use_reddit_previews`** - Use Reddit's preview/thumbnail system
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**: Reddit's own cached preview images
- **Best For**: Recent posts with images, when original host is down
- **Success Rate**: 20-50% (only works if Reddit generated preview)
- **Coverage**: Images from posts where Reddit created thumbnails
- **Quality**: Usually lower resolution (preview quality, not original)
- **Rate Limit**: 30 requests/minute
- **Response Time**: <1 second (very fast)
- **Limitations**:
- Only works for posts Reddit previewed
- Lower quality than originals
- May not work for very old posts
- **Examples**:
```ini
use_reddit_previews = true # Enable (fast fallback)
use_reddit_previews = false # Disable if quality matters
```
* **`use_reveddit_api`** - Use Reveddit deleted content recovery
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**: Specialized service for recovering deleted Reddit content
- **Best For**: Recently deleted posts/comments (within days/weeks)
- **Success Rate**: 30-60% for recent deletions, lower for older
- **Coverage**: Reddit posts and comments deleted by users or moderators
- **Rate Limit**: 20 requests/minute
- **Response Time**: 2-5 seconds average
- **Note**: Most effective for recent deletions (<30 days)
- **Examples**:
```ini
use_reveddit_api = true # Enable (good for recent deletions)
use_reveddit_api = false # Disable to save time
```
#### Recovery Performance Settings:
* **`timeout_seconds`** - Maximum wait time per provider attempt
- **Type**: Integer (positive)
- **Default**: `10`
- **Valid Range**: 3-120 seconds (recommended: 5-30)
- **What It Does**: Max time to wait for each recovery provider to respond
- **Behavior on Timeout**:
- Provider attempt cancelled
- Moves to next provider in cascade
- Failure logged but doesn't stop processing
- **Trade-offs**:
| Timeout | Success Rate | Speed | Best For |
|---------|-------------|-------|----------|
| **5s** | Lower (~50%) | Fast | Quick pass, fast networks |
| **10s** | Good (~70%) | Moderate | Default, balanced |
| **20-30s** | Higher (~85%) | Slow | Thorough recovery, slow networks |
- **Cascade Example** (timeout=10s, 4 providers):
- Wayback: Try for 10s → Success/Fail → Next
- PullPush: Try for 10s → Success/Fail → Next
- Reddit Preview: Try for 10s → Success/Fail → Next
- Reveddit: Try for 10s → Success/Fail → Give up
- Total: 0-40 seconds per item (stops at first success)
- **Examples**:
```ini
timeout_seconds = 5 # Fast, may miss some content
timeout_seconds = 10 # Default, balanced
timeout_seconds = 30 # Thorough, slow
```
* **`cache_duration_hours`** - How long to cache recovery results
- **Type**: Integer (positive)
- **Default**: `24`
- **Valid Range**: 1-720 hours (1 hour to 30 days)
- **What It Does**: Stores recovery results (success/failure) to avoid re-trying
- **What Gets Cached**:
- Successful recoveries: URL → recovered content location
- Failed attempts: URL → "not found" (avoid retrying same failure)
- **Cache Database**: `.recovery_cache.db` in save directory (SQLite)
- **Why Cache?**:
- Avoid re-querying same URL across runs
- Respect provider rate limits
- Speed up subsequent runs (instant cache hits)
- Reduce network usage
- **Duration Recommendations**:
| Duration | Use Case | Behavior |
|----------|----------|----------|
| **1-6 hours** | Testing, rapidly changing content | Short-term cache |
| **24-48 hours** | Normal use, daily runs | Default, balanced |
| **168 hours (1 week)** | Weekly runs, stable content | Longer persistence |
| **720 hours (30 days)** | Monthly runs, archival | Maximum persistence |
- **Auto-Cleanup**: Expired entries automatically removed based on `cleanup_interval_minutes`
- **Examples**:
```ini
cache_duration_hours = 6 # 6 hours (short-term)
cache_duration_hours = 24 # 24 hours (default)
cache_duration_hours = 168 # 1 week (long-term)
```
#### Recovery Cache Management:
* **`max_cache_entries`** - Maximum number of cached results
- **Type**: Integer (positive)
- **Default**: `10000`
- **Valid Range**: 100-1,000,000+
- **What It Does**: Limits total number of entries in cache database
- **Cleanup Behavior**:
- When limit reached: Oldest entries removed first (FIFO)
- Expired entries removed first, then oldest
- Cleanup triggered automatically
- **Storage Per Entry**: ~200-500 bytes average
- **Total Storage Examples**:
```
1,000 entries = ~0.5 MB
10,000 entries = ~5 MB (default)
100,000 entries = ~50 MB
1,000,000 entries = ~500 MB
```
- **Recommendations**:
- Small archive (<1000 posts): 1,000-5,000 entries
- Medium archive (1000-10000): 10,000-50,000 entries (default range)
- Large archive (10000+): 50,000-500,000 entries
- **Examples**:
```ini
max_cache_entries = 1000 # Minimal cache
max_cache_entries = 10000 # Default
max_cache_entries = 100000 # Large cache
```
* **`max_cache_size_mb`** - Cache size limit in megabytes
- **Type**: Integer (positive)
- **Default**: `100`
- **Valid Range**: 1-10,000 MB (1MB to 10GB)
- **What It Does**: Limits total disk space used by cache database
- **Cleanup Trigger**: When cache file exceeds this size
- **Relationship with `max_cache_entries`**:
- Both limits enforced independently
- Whichever limit reached first triggers cleanup
- Typically entries limit hits first
- **Recommendations**:
```
10 MB: Testing, minimal cache
100 MB: Default, sufficient for most users
500 MB: Large archives, lots of recovery attempts
1000 MB: Very large archives, maximum persistence
```
- **Examples**:
```ini
max_cache_size_mb = 10 # Minimal
max_cache_size_mb = 100 # Default
max_cache_size_mb = 500 # Large
```
* **`cleanup_interval_minutes`** - How often to run cache cleanup
- **Type**: Integer (positive)
- **Default**: `60`
- **Valid Range**: 5-1440 minutes (5 minutes to 24 hours)
- **What It Does**: Automatic cleanup of expired cache entries
- **What Gets Cleaned**:
- Entries older than `cache_duration_hours`
- Excess entries beyond `max_cache_entries`
- If size exceeds `max_cache_size_mb`
- **Trigger Timing**: Based on wall clock time
- **Performance Impact**: Minimal (<1 second per cleanup)
- **Recommendations**:
```
15-30 min: Frequent runs, tight control
60 min: Default, balanced (hourly cleanup)
120-240 min: Infrequent runs, less overhead
1440 min: Once per day (very infrequent runs)
```
- **Examples**:
```ini
cleanup_interval_minutes = 30 # Every 30 minutes
cleanup_interval_minutes = 60 # Every hour (default)
cleanup_interval_minutes = 1440 # Daily
```
* **`enable_background_cleanup`** - Automatic cache maintenance
- **Type**: Boolean
- **Default**: `true`
- **Valid Values**: `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`
- **What It Does**:
- `true`: Runs cleanup automatically based on `cleanup_interval_minutes`
- `false`: Only cleans up when limits exceeded (manual mode)
- **Background Mode** (`true`):
- Periodic automatic cleanup
- Prevents cache bloat
- Recommended for most users
- **Manual Mode** (`false`):
- Only cleans when forced (size/entry limits hit)
- Slightly less overhead
- Cache may grow larger before cleanup
- Use if you want maximum cache retention
- **Examples**:
```ini
enable_background_cleanup = true # Automatic (recommended)
enable_background_cleanup = false # Manual only
```
#### `[Retry]` - Retry Queue Configuration
```ini
[Retry]
# Retry behavior
max_retries = 5 # Maximum retry attempts per item
base_retry_delay_high = 5 # Base delay for high priority (seconds)
base_retry_delay_medium = 10 # Base delay for medium priority (seconds)
base_retry_delay_low = 15 # Base delay for low priority (seconds)
# Exponential backoff
exponential_base_delay = 60 # Base delay for exponential backoff
max_retry_delay = 86400 # Maximum delay (24 hours in seconds)
# Dead letter queue
dead_letter_threshold_days = 7 # Days before moving to dead letter queue
```
#### Retry System Explained:
The retry system ensures failed downloads are automatically retried across multiple runs with intelligent priority-based backoff strategies. Failed items are queued in a persistent SQLite database (`.retry_queue.db`) and retried on subsequent script runs.
#### Retry Attempt Settings:
* **`max_retries`** - Maximum retry attempts before giving up
- **Type**: Integer (positive)
- **Default**: `5`
- **Valid Range**: 1-50 (recommended: 3-10)
- **What It Does**: How many times to retry a failed download before moving to dead letter queue
- **Retry Counter**:
- Attempt 1: Initial download (not a retry)
- Attempts 2-6: Actual retries (if max_retries=5)
- After attempt 6: Move to dead letter queue
- **What Triggers Retries**:
- Network timeouts
- HTTP errors (403, 429, 500, 502, 503, 504)
- Temporary service unavailability
- Rate limit errors
- **What Doesn't Retry**:
- 404 Not Found (permanent, triggers content recovery instead)
- Invalid URLs
- File too large (exceeds size limits)
- **Recommendations By Use Case**:
```
3 retries: Quick processing, accept some failures
5 retries: Default, balanced persistence
10 retries: Maximum persistence, thorough recovery
20+ retries: Extreme cases, very unstable networks
```
- **Examples**:
```ini
max_retries = 3 # Quick, fewer attempts
max_retries = 5 # Default, balanced
max_retries = 10 # Persistent, thorough
```
#### Priority-Based Delay Settings:
Failed items are assigned priorities that determine their retry delays:
* **Priority Assignment** (automatic):
- **High Priority**: Small files (<1MB), recently saved posts, first-time failures
- **Medium Priority**: Medium files (1-10MB), standard content
- **Low Priority**: Large files (>10MB), repeated failures, low-value content
* **`base_retry_delay_high`** - Base delay for high-priority items (seconds)
- **Type**: Integer (positive)
- **Default**: `5`
- **Valid Range**: 1-300 seconds
- **When Used**: Small, recent, important content
- **Examples**:
```ini
base_retry_delay_high = 1 # Retry almost immediately
base_retry_delay_high = 5 # Default, 5 second delay
base_retry_delay_high = 30 # More patient
```
* **`base_retry_delay_medium`** - Base delay for medium-priority items (seconds)
- **Type**: Integer (positive)
- **Default**: `10`
- **Valid Range**: 5-600 seconds
- **When Used**: Standard content, typical failures
- **Examples**:
```ini
base_retry_delay_medium = 5 # Quick retry
base_retry_delay_medium = 10 # Default
base_retry_delay_medium = 60 # Patient retry
```
* **`base_retry_delay_low`** - Base delay for low-priority items (seconds)
- **Type**: Integer (positive)
- **Default**: `15`
- **Valid Range**: 10-1800 seconds
- **When Used**: Large files, repeated failures
- **Examples**:
```ini
base_retry_delay_low = 10 # Relatively quick
base_retry_delay_low = 15 # Default
base_retry_delay_low = 120 # Very patient (2 minutes)
```
**Priority Delay Example**:
```
High priority item (attempt 1): Wait 5 seconds
Medium priority item (attempt 1): Wait 10 seconds
Low priority item (attempt 1): Wait 15 seconds
```
#### Exponential Backoff Settings:
After the base delay, subsequent retries use exponential backoff to avoid hammering failing services.
* **`exponential_base_delay`** - Base delay for exponential backoff calculation
- **Type**: Integer (positive)
- **Default**: `60`
- **Valid Range**: 10-3600 seconds (10 seconds to 1 hour)
- **Formula**: `delay = exponential_base_delay × 2^(attempt_number - 1)`
- **Backoff Examples** (base=60):
```
Attempt 1: 60 × 2^0 = 60 seconds (1 minute)
Attempt 2: 60 × 2^1 = 120 seconds (2 minutes)
Attempt 3: 60 × 2^2 = 240 seconds (4 minutes)
Attempt 4: 60 × 2^3 = 480 seconds (8 minutes)
Attempt 5: 60 × 2^4 = 960 seconds (16 minutes)
```
- **Purpose**: Gradual backoff reduces load on failing services, increases success chance
- **Recommendations**:
```
30s: Quick backoff, impatient
60s: Default, balanced (1 minute base)
300s: Slow backoff, very patient (5 minute base)
```
- **Examples**:
```ini
exponential_base_delay = 30 # Quick backoff
exponential_base_delay = 60 # Default (1 min)
exponential_base_delay = 300 # Slow backoff (5 min)
```
* **`max_retry_delay`** - Maximum delay between retries (cap)
- **Type**: Integer (positive)
- **Default**: `86400` (24 hours)
- **Valid Range**: 60-604800 seconds (1 minute to 7 days)
- **What It Does**: Caps exponential backoff to prevent extremely long delays
- **Without Cap**: Attempt 10 with base=60 would be 30,720 seconds (8.5 hours!)
- **With Cap** (86400): Any delay >24 hours is capped at 24 hours
- **Delay Capping Example** (base=60, max=86400):
```
Attempt 1: 60s (1 min)
Attempt 2: 120s (2 min)
Attempt 3: 240s (4 min)
Attempt 4: 480s (8 min)
Attempt 5: 960s (16 min)
Attempt 6: 1920s (32 min)
Attempt 7: 3840s (64 min)
Attempt 8: 7680s (128 min ≈ 2 hours)
Attempt 9: 15360s (256 min ≈ 4 hours)
Attempt 10: 30720s (512 min ≈ 8.5 hours)
Attempt 11: 61440s → CAPPED at 86400s (24 hours)
```
- **Recommendations**:
```
3600s (1 hour): Quick turnaround, frequent runs
86400s (24 hours): Default, daily runs
604800s (7 days): Weekly runs, maximum patience
```
- **Examples**:
```ini
max_retry_delay = 3600 # 1 hour max
max_retry_delay = 86400 # 24 hours (default)
max_retry_delay = 604800 # 7 days max
```
#### Dead Letter Queue Settings:
Items that exceed retry limits are moved to a "dead letter queue" for manual review or permanent archiving.
* **`dead_letter_threshold_days`** - Days before giving up permanently
- **Type**: Integer (positive)
- **Default**: `7`
- **Valid Range**: 1-365 days (1 day to 1 year)
- **What It Does**: After this many days of retrying, item moves to dead letter queue
- **Dead Letter Queue Behavior**:
- Items marked as "permanently failed"
- No longer retried automatically
- Kept in database for manual review
- Can be manually cleared or re-queued
- **Calculation**:
- Based on first_failure_timestamp, not retry count
- Example: Item first fails on Jan 1, threshold=7 days → Moves to DLQ on Jan 8
- **What Happens After DLQ**:
- Item logged as permanent failure
- Visible in retry queue status
- Manual intervention required to retry
- Can be cleared to reduce database size
- **Recommendations**:
```
1-2 days: Quick cleanup, aggressive pruning
7 days: Default, balanced (1 week)
30 days: Patient, thorough recovery attempts
365 days: Maximum persistence (1 year)
```
- **Examples**:
```ini
dead_letter_threshold_days = 1 # Move to DLQ after 1 day
dead_letter_threshold_days = 7 # Default (1 week)
dead_letter_threshold_days = 30 # Patient (1 month)
```
**Dead Letter Queue Management**:
- View DLQ items: Check `.retry_queue.db` with SQLite browser
- Clear DLQ: Delete database file (creates fresh on next run)
- Re-queue items: Manual SQL UPDATE statements
- Monitor: Check logs for "moved to dead letter queue" messages
**Complete Retry Flow Example** (max_retries=5, threshold=7 days):
```
Day 1, Run 1: Download fails → Queue (attempt 1/5, high priority, 5s delay)
Day 1, Run 2: Retry fails → Queue (attempt 2/5, 60s backoff)
Day 2, Run 1: Retry fails → Queue (attempt 3/5, 120s backoff)
Day 3, Run 1: Retry fails → Queue (attempt 4/5, 240s backoff)
Day 4, Run 1: Retry fails → Queue (attempt 5/5, 480s backoff)
Day 5, Run 1: Final retry fails → Max retries exceeded, keep in queue
Day 8: Item in queue for >7 days → Move to dead letter queue
```
---
### Settings Index (Alphabetical)
Quick alphabetical reference of all 43 settings with links to detailed documentation:
#### A-C
- **`base_retry_delay_high`** (Integer, default: 5) - High-priority retry delay | [→ Retry Section](#priority-based-delay-settings)
- **`base_retry_delay_low`** (Integer, default: 15) - Low-priority retry delay | [→ Retry Section](#priority-based-delay-settings)
- **`base_retry_delay_medium`** (Integer, default: 10) - Medium-priority retry delay | [→ Retry Section](#priority-based-delay-settings)
- **`cache_duration_hours`** (Integer, default: 24) - Cache recovery results duration | [→ Recovery Section](#recovery-performance-settings)
- **`check_type`** (String, default: LOG) - File existence checking method | [→ Settings Section](#core-settings-explained)
- **`cleanup_interval_minutes`** (Integer, default: 60) - Cache cleanup frequency | [→ Recovery Section](#recovery-cache-management)
- **`client_id`** (String, default: None) - Reddit API client ID | [→ Configuration Section](#api-configuration-settings)
- **`client_ids`** (String, default: None) - Imgur client IDs (multiple in settings.ini only) | [→ Imgur Section](#imgur-settings-explained)
- **`client_secret`** (String, default: None) - Reddit API client secret | [→ Configuration Section](#api-configuration-settings)
- **`client_secrets`** (String, default: None) - Imgur client secrets (multiple in settings.ini only) | [→ Imgur Section](#imgur-settings-explained)
- **`create_thumbnails`** (Boolean, default: true) - Generate thumbnail versions | [→ Media Section](#media-settings-explained)
#### D-I
- **`dead_letter_threshold_days`** (Integer, default: 7) - Days before moving to DLQ | [→ Retry Section](#dead-letter-queue-settings)
- **`download_albums`** (Boolean, default: true) - Process multi-image posts | [→ Media Section](#media-settings-explained)
- **`download_audio`** (Boolean, default: true) - Control audio downloads | [→ Media Section](#media-settings-explained)
- **`download_enabled`** (Boolean, default: true) - Master media download switch | [→ Media Section](#media-settings-explained)
- **`download_images`** (Boolean, default: true) - Control image downloads | [→ Media Section](#media-settings-explained)
- **`download_timeout`** (Integer, default: 30) - Per-file download timeout | [→ Media Section](#media-settings-explained)
- **`download_videos`** (Boolean, default: true) - Control video downloads | [→ Media Section](#media-settings-explained)
- **`dropbox_directory`** (String, default: /reddit) - Dropbox cloud storage path | [→ Settings Section](#core-settings-explained)
- **`enable_background_cleanup`** (Boolean, default: true) - Automatic cache maintenance | [→ Recovery Section](#recovery-cache-management)
- **`exponential_base_delay`** (Integer, default: 60) - Exponential backoff base delay | [→ Retry Section](#exponential-backoff-settings)
- **`ignore_tls_errors`** (Boolean, default: false) - Bypass SSL certificate validation | [→ Settings Section](#core-settings-explained)
#### M-P
- **`max_album_images`** (Integer, default: 50) - Limit images per album | [→ Media Section](#media-settings-explained)
- **`max_cache_entries`** (Integer, default: 10000) - Maximum cached recovery results | [→ Recovery Section](#recovery-cache-management)
- **`max_cache_size_mb`** (Integer, default: 100) - Cache size limit in MB | [→ Recovery Section](#recovery-cache-management)
- **`max_concurrent_downloads`** (Integer, default: 3) - Parallel download streams | [→ Media Section](#media-settings-explained)
- **`max_daily_storage_mb`** (Integer, default: 1024) - Daily storage limit in MB | [→ Media Section](#media-settings-explained)
- **`max_image_size`** (Integer, default: 5242880) - Max image file size in bytes | [→ Media Section](#media-settings-explained)
- **`max_retries`** (Integer, default: 5) - Maximum retry attempts | [→ Retry Section](#retry-attempt-settings)
- **`max_retry_delay`** (Integer, default: 86400) - Maximum retry delay cap | [→ Retry Section](#exponential-backoff-settings)
- **`max_video_size`** (Integer, default: 209715200) - Max video file size in bytes | [→ Media Section](#media-settings-explained)
- **`password`** (String, default: None) - Reddit account password | [→ Configuration Section](#api-configuration-settings)
- **`process_api`** (Boolean, default: true) - Fetch content from Reddit API | [→ Settings Section](#core-settings-explained)
- **`process_gdpr`** (Boolean, default: false) - Process GDPR export files | [→ Settings Section](#core-settings-explained)
#### R-U
- **`recover_deleted`** (Boolean, default: true) - Attempt Imgur content recovery | [→ Imgur Section](#imgur-settings-explained)
- **`save_directory`** (String, default: reddit/) - Local save directory | [→ Settings Section](#core-settings-explained)
- **`save_type`** (String, default: ALL) - What content to download | [→ Settings Section](#core-settings-explained)
- **`thumbnail_size`** (Integer, default: 800) - Thumbnail dimensions in pixels | [→ Media Section](#media-settings-explained)
- **`timeout_seconds`** (Integer, default: 10) - Per-provider recovery timeout | [→ Recovery Section](#recovery-performance-settings)
- **`unsave_after_download`** (Boolean, default: false) - Auto-unsave after download | [→ Settings Section](#core-settings-explained)
- **`use_pushshift_api`** (Boolean, default: true) - Use PullPush.io provider | [→ Recovery Section](#recovery-provider-settings)
- **`use_reddit_previews`** (Boolean, default: true) - Use Reddit preview system | [→ Recovery Section](#recovery-provider-settings)
- **`use_reveddit_api`** (Boolean, default: true) - Use Reveddit provider | [→ Recovery Section](#recovery-provider-settings)
- **`use_wayback_machine`** (Boolean, default: true) - Use Internet Archive | [→ Recovery Section](#recovery-provider-settings)
- **`username`** (String, default: None) - Reddit username | [→ Configuration