{"id":27054787,"url":"https://github.com/bitkarrot/pfpkhatru","last_synced_at":"2025-04-05T09:17:04.728Z","repository":{"id":281275062,"uuid":"944751598","full_name":"bitkarrot/pfpkhatru","owner":"bitkarrot","description":"a LRU Profile Pic cache based on Khatru Relay Framework","archived":false,"fork":false,"pushed_at":"2025-03-08T00:58:06.000Z","size":9424,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-08T01:26:21.098Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitkarrot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-07T22:42:12.000Z","updated_at":"2025-03-08T00:58:08.000Z","dependencies_parsed_at":"2025-03-08T01:36:27.538Z","dependency_job_id":null,"html_url":"https://github.com/bitkarrot/pfpkhatru","commit_stats":null,"previous_names":["bitkarrot/pfpkhatru"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitkarrot%2Fpfpkhatru","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitkarrot%2Fpfpkhatru/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitkarrot%2Fpfpkhatru/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitkarrot%2Fpfpkhatru/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitkarrot","download_url":"https://codeload.github.com/bitkarrot/pfpkhatru/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247312057,"owners_count":20918344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-05T09:17:04.225Z","updated_at":"2025-04-05T09:17:04.676Z","avatar_url":"https://github.com/bitkarrot.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Khatru Profile Picture Cache Relay\n\nA Nostr relay that caches profile pictures for fast access. It stores metadata events (kind 0) and serves profile pictures from a local cache.\n\n## Features\n\n- Caches profile pictures from Nostr profiles\n- Provides a simple HTTP API to fetch cached profile pictures\n- Automatically fetches profile pictures from multiple relays\n- Supports batch caching of profile pictures\n- Follows caching to pre-cache profile pictures of users that a given user follows\n- Media cache with configurable expiration\n- LRU (Least Recently Used) cache management to automatically remove old files when cache size limit is reached\n- Image resizing to optimize storage and bandwidth usage\n\n## TODO\n- make this work with LMDB, sqlite was just for testing\n\n## Installation\n\n### Prerequisites\n\n- [Go](https://golang.org/doc/install) (version 1.18 or later recommended)\n- Git (optional, for cloning the repository)\n\n### Steps\n\n1. Clone or download the repository:\n   ```bash\n   # Using Git\n   git clone https://github.com/bitkarrot/khatru.git\n   cd khatru/pfpcache\n   \n   # Or download and extract the ZIP file for the pfpcache directory only\n   ```\n\n2. Make the run script executable:\n   ```bash\n   chmod +x run.sh\n   ```\n\n3. Run the application:\n   ```bash\n   ./run.sh\n   ```\n\n   This script will:\n   - Create a default configuration file if it doesn't exist\n   - Create necessary data directories\n   - Build the Go code\n   - Start the relay on http://localhost:8080\n\n### Building from Source Manually\n\nIf you prefer not to use the run.sh script, you can build and run the application manually:\n\n1. Create the necessary directories:\n   ```bash\n   mkdir -p ./data/media_cache\n   ```\n\n2. Create a config.json file (if it doesn't exist):\n   ```bash\n   cat \u003e ./config.json \u003c\u003c EOF\n   {\n     \"listen_addr\": \":8080\",\n     \"database_path\": \"./data/pfpcache.db\",\n     \"media_cache_path\": \"./data/media_cache\",\n     \"upstream_relays\": [\n       \"wss://damus.io\",\n       \"wss://primal.net\",\n       \"wss://nos.lol\"\n     ],\n     \"max_concurrent\": 20,\n     \"cache_expiration_days\": 7\n   }\n   EOF\n   ```\n\n3. Build the application:\n   ```bash\n   go build -o pfpcache-relay main.go\n   ```\n\n4. Run the application:\n   ```bash\n   ./pfpcache-relay\n   ```\n\n## Usage\n\n### Starting the Relay\n\n```bash\n./run.sh\n```\n\n### Endpoints\n\n#### Profile Picture\n\n```\nGET /profile-pic/{pubkey}\n```\n\nReturns the profile picture for the given pubkey. If the profile picture is not cached, it will be fetched from the upstream relays and cached.\n\n**Example:**\n\n```html\n\u003c!-- In HTML --\u003e\n\u003cimg src=\"http://localhost:8080/profile-pic/32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245\" alt=\"Profile picture\"\u003e\n```\n\n```bash\n# Using curl\ncurl -o profile.jpg http://localhost:8080/profile-pic/32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245\n```\n\n#### Batch Cache\n\n```\nPOST /batch-cache\n```\n\nRequest body:\n```json\n{\n  \"pubkeys\": [\"pubkey1\", \"pubkey2\", ...]\n}\n```\n\nStarts a background job to cache profile pictures for the given pubkeys. Returns immediately with a status message.\n\n**Example:**\n\n```bash\n# Using curl\ncurl -X POST -H \"Content-Type: application/json\" -d '{\"pubkeys\": [\"32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245\", \"3878d95db7b854c3a0d3b2d6b7bf9bf28b36162be64326f5521ba71cf3b45a69\"]}' http://localhost:8080/batch-cache\n```\n\n**Response:**\n\n```json\n{\n  \"message\": \"Started caching 2 profile pictures\",\n  \"count\": 2\n}\n```\n\n#### Cache Follows\n\n```\nGET /cache-follows/{pubkey}?limit=100\n```\n\nFetches the follows (contact list) for the given pubkey and caches their profile pictures. The `limit` parameter is optional and defaults to 500.\n\n**Example:**\n\n```bash\n# Cache profile pictures for up to 100 follows of a user\ncurl -X GET http://localhost:8080/cache-follows/32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245?limit=100\n```\n\n**Response:**\n\n```json\n{\n  \"message\": \"Started caching profile pictures for 100 follows\",\n  \"count\": 100,\n  \"pubkey\": \"32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245\"\n}\n```\n\n#### Purge Cache\n\n```\nPOST /purge-cache/all\nPOST /purge-cache/profile-pics\n```\n\nPurges all cached media or just profile pictures.\n\n**Examples:**\n\n```bash\n# Purge all cached media\ncurl -X POST http://localhost:8080/purge-cache/all\n\n# Purge only profile pictures\ncurl -X POST http://localhost:8080/purge-cache/profile-pics\n```\n\n**Response:**\n\n```json\n{\n  \"status\": \"success\",\n  \"message\": \"Profile picture cache purged successfully\"\n}\n```\n\n#### Purge Single Profile Picture\n\n```\nPOST /purge-profile-pic/{pubkey}\n```\n\nPurges the cached profile picture for the given pubkey.\n\n**Example:**\n\n```bash\n# Purge a specific profile picture\ncurl -X POST http://localhost:8080/purge-profile-pic/32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245\n```\n\n**Response:**\n\n```json\n{\n  \"status\": \"success\",\n  \"message\": \"Profile picture for 32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245 purged successfully\"\n}\n```\n\n## Configuration\n\nThe relay can be configured using a JSON configuration file. The default configuration file is `config.json` in the current directory.\n\n```json\n{\n  \"listen_addr\": \":8080\",\n  \"database_path\": \"./data/pfpcache.db\",\n  \"media_cache_path\": \"./data/media_cache\",\n  \"upstream_relays\": [\n    \"wss://damus.io\",\n    \"wss://primal.net\",\n    \"wss://nos.lol\",\n    \"wss://purplepag.es\"\n  ],\n  \"max_concurrent\": 20,\n  \"cache_expiration_days\": 30,\n  \"max_cache_size_mb\": 1024,\n  \"lru_check_interval\": 60,\n  \"resize_images\": true,\n  \"max_image_size\": 200,\n  \"image_quality\": 85\n}\n```\n\n| Parameter | Description |\n| --- | --- |\n| `listen_addr` | The address to listen on |\n| `database_path` | The path to the SQLite database |\n| `media_cache_path` | The path to the media cache directory |\n| `upstream_relays` | A list of upstream relays to fetch profiles from |\n| `max_concurrent` | The maximum number of concurrent requests to upstream relays |\n| `cache_expiration_days` | The number of days after which cached media expires (default: 30 days) |\n| `max_cache_size_mb` | The maximum size of the cache in megabytes (default: 1024 MB) |\n| `lru_check_interval` | The interval in minutes to check and clean the LRU cache (default: 60 minutes) |\n| `resize_images` | Whether to resize profile images to optimize storage and bandwidth (default: true) |\n| `max_image_size` | The maximum width/height for resized profile images in pixels (default: 200) |\n| `image_quality` | The JPEG quality for resized images (1-100, default: 85) |\n\n## Running in Production\n\nFor production environments, you may want to:\n\n1. Customize the configuration in `config.json` based on your needs\n2. Use a process manager like systemd, supervisor, or PM2 to keep the service running\n3. Set up a reverse proxy (like Nginx) if you want to expose the service publicly\n\nExample systemd service file (`/etc/systemd/system/pfpcache.service`):\n\n```ini\n[Unit]\nDescription=Khatru Profile Picture Cache Relay\nAfter=network.target\n\n[Service]\nType=simple\nUser=yourusername\nWorkingDirectory=/path/to/pfpcache\nExecStart=/path/to/pfpcache/pfpcache-relay\nRestart=on-failure\nRestartSec=5s\n\n[Install]\nWantedBy=multi-user.target\n```\n\nAfter creating the service file:\n```bash\nsudo systemctl daemon-reload\nsudo systemctl enable pfpcache\nsudo systemctl start pfpcache\n```\n\n## Example Client\n\nSee `profile-pic-example.html` for an example of how to use the profile picture endpoint in a web page.\n\n## LRU Cache Management\n\nThe relay implements a Least Recently Used (LRU) cache mechanism to automatically manage the cache size:\n\n- The cache size is limited by the `max_cache_size_mb` configuration parameter\n- The system tracks when each file was last accessed\n- When the cache size exceeds the limit, the least recently used files are removed first\n- The cache is checked periodically based on the `lru_check_interval` configuration parameter\n\nThis ensures that:\n1. The cache doesn't grow indefinitely\n2. The most frequently accessed profile pictures remain in the cache\n3. Older, unused profile pictures are automatically removed\n\nYou can still manually purge the cache using the purge endpoints if needed.\n\n## Use Cases\n\n### 1. Fast Profile Picture Loading in Web Apps\n\nWeb applications can use the profile picture endpoint to quickly load profile pictures without having to query Nostr relays directly:\n\n```javascript\n// Example JavaScript\nfunction loadProfilePicture(pubkey) {\n  const img = document.createElement('img');\n  img.src = `http://localhost:8080/profile-pic/${pubkey}`;\n  img.alt = 'Profile picture';\n  document.getElementById('profile-container').appendChild(img);\n}\n```\n\n### 2. Preloading Profile Pictures for a Feed\n\nBefore displaying a feed of posts, you can preload all the profile pictures:\n\n```javascript\n// Example JavaScript\nasync function preloadProfilePictures(pubkeys) {\n  const response = await fetch('http://localhost:8080/batch-cache', {\n    method: 'POST',\n    headers: {\n      'Content-Type': 'application/json',\n    },\n    body: JSON.stringify({ pubkeys }),\n  });\n  \n  console.log('Preloading profile pictures:', await response.json());\n}\n```\n\n### 3. Caching Profile Pictures for a User's Network\n\nWhen a user logs in, cache profile pictures for all their follows:\n\n```javascript\n// Example JavaScript\nasync function cacheNetworkProfilePics(userPubkey) {\n  const response = await fetch(`http://localhost:8080/cache-follows/${userPubkey}`);\n  console.log('Caching network profile pictures:', await response.json());\n}\n```\n\n### 4. Clearing Cache During Development\n\nDuring development or testing, you might want to clear the cache:\n\n```bash\n# Clear all cached media\ncurl -X POST http://localhost:8080/purge-cache/all\n\n# Clear only profile pictures\ncurl -X POST http://localhost:8080/purge-cache/profile-pics\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitkarrot%2Fpfpkhatru","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitkarrot%2Fpfpkhatru","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitkarrot%2Fpfpkhatru/lists"}