{"id":32662518,"url":"https://github.com/graze-social/iftta","last_synced_at":"2026-03-04T21:34:46.914Z","repository":{"id":321328530,"uuid":"1043001273","full_name":"graze-social/iftta","owner":"graze-social","description":"AT Protocol automation service written in Rust","archived":false,"fork":false,"pushed_at":"2025-10-29T03:11:43.000Z","size":1451,"stargazers_count":58,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-29T04:24:18.743Z","etag":null,"topics":["atprotocol","automation","bluesky"],"latest_commit_sha":null,"homepage":"https://app.ifthisthen.at","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/graze-social.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-23T00:20:31.000Z","updated_at":"2025-10-29T03:11:46.000Z","dependencies_parsed_at":"2025-10-29T04:34:31.428Z","dependency_job_id":null,"html_url":"https://github.com/graze-social/iftta","commit_stats":null,"previous_names":["graze-social/iftta"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/graze-social/iftta","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graze-social%2Fiftta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graze-social%2Fiftta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graze-social%2Fiftta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graze-social%2Fiftta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/graze-social","download_url":"https://codeload.github.com/graze-social/iftta/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graze-social%2Fiftta/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30094001,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T20:42:30.420Z","status":"ssl_error","status_checked_at":"2026-03-04T20:42:30.057Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atprotocol","automation","bluesky"],"created_at":"2025-10-31T20:00:38.414Z","updated_at":"2026-03-04T21:34:46.905Z","avatar_url":"https://github.com/graze-social.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# ifthisthenat (IFTTA)\n![Snowflake Designs from Public Domain Archive, https://pdimagearchive.org/images/585d6186-d316-4672-87f1-f15ca7eb37b6/](https://bdefzwcumgzjwllsnaej.supabase.co/storage/v1/render/image/public/minilink-user-assets/0199628b-92ab-7cc3-b5c5-5fcb87301262?width=3840\u0026quality=75)\n**If This Then AT** - A high-performance AT Protocol automation service written in Rust. Process events from Jetstream (Bluesky's firehose), webhooks, and periodic schedules using a blueprint-based system with DataLogic expressions for rule evaluation.\n\n**Version**: 1.0.0-rc.1\n**Repository**: [github.com/graze-social/iftta](https://github.com/graze-social/iftta)\n\n## ⚠️ Release Candidate\n\n**Version 1.0.0-rc.1** - This release candidate includes comprehensive error handling, production features, and extensive testing. While suitable for production use, thorough testing is recommended for your specific use case.\n\n## Performance\n\nifthisthenat is designed for high throughput and low latency:\n\n- **Binary serialization** reduces cache storage by ~40% compared to JSON\n- **Multi-layer caching** with in-memory, Redis, and PostgreSQL support\n- **Work shedding** prevents unbounded queue growth\n- **Configurable TTLs** for fine-tuning cache freshness vs. performance\n- **Connection pooling** for Redis and PostgreSQL minimizes overhead\n- **Metrics support** via StatsD for production monitoring\n\nBenchmarks:\n- Condition node evaluation: ~30-50μs per evaluation\n- Sentiment analysis: ~300-500μs per analysis\n- Blueprint caching: 40% storage reduction with binary serialization\n- Jetstream processing: Supports partitioning for horizontal scaling\n\n## Features\n\n- **Blueprint System**: Define automation workflows with ordered node evaluation\n- **Entry Nodes**:\n  - `jetstream_entry`: Process AT Protocol firehose events\n  - `webhook_entry`: Handle external webhook triggers\n  - `periodic_entry`: Cron-based scheduled tasks\n  - `zap_entry`: Zapier integration support\n- **Processing Nodes**:\n  - `condition`: Boolean flow control with DataLogic expressions\n  - `transform`: Data manipulation and transformation\n  - `facet_text`: Process text with AT Protocol facets/mentions\n  - `sentiment_analysis`: Analyze text sentiment\n  - `parse_aturi`: Parse AT-URI strings\n  - `get_record`: Retrieve AT Protocol records\n- **Action Nodes**:\n  - `publish_record`: Create AT Protocol records\n  - `publish_webhook_direct`: Synchronous webhook delivery\n  - `publish_webhook_queue`: Async webhooks with retries\n  - `debug_action`: Development logging\n- **Production Features**:\n  - OAuth 2.0/2.1/OIDC authentication with AT Protocol\n  - Multi-instance support with Redis coordination\n  - Distributed leadership election\n  - Comprehensive error handling with structured codes\n  - Binary serialization for 40% storage reduction\n  - Sentry error tracking integration\n  - Health checks and graceful shutdown\n- **Queue \u0026 Throttling**:\n  - Multiple adapter support (MPSC, Redis)\n  - Blueprint throttling (per-authority, per-AT-URI)\n  - Work shedding for overload protection\n  - Denylist system for blocking bad actors\n\n## Building\n\n### Prerequisites\n\n- Rust 1.70 or later\n- PostgreSQL 14+ (required)\n- Redis 6.2+ (optional, for multi-instance deployments)\n- Docker \u0026 Docker Compose (for development)\n\n### Build Commands\n\n```bash\n# Clone the repository\ngit clone https://github.com/graze-social/iftta.git\ncd ifthisthenat\n\n# Start development dependencies\ndocker-compose up -d\n\n# Run database migrations\nsqlx migrate run\n\n# Build the project\ncargo build --release\n\n# Run tests\ncargo test\n\n# Run with debug logging\nRUST_LOG=debug cargo run\n```\n\n## Minimum Configuration\n\nifthisthenat requires minimal configuration to run. Configuration is validated at startup, and the service will exit with specific error codes if validation fails.\n\n### Required\n\n- `EXTERNAL_BASE`: External hostname for service endpoints (e.g., `https://iftta.example.com`)\n- `HTTP_COOKIE_KEY`: Cookie encryption key (generate with `openssl rand -base64 64`)\n- `ISSUER_DID`: Service DID (must be valid did:plc or did:web)\n- `AIP_HOSTNAME`: AT Protocol OAuth hostname\n- `AIP_CLIENT_ID`: OAuth client ID\n- `AIP_CLIENT_SECRET`: OAuth client secret\n\n### Example Minimal Setup\n\n```bash\n# Generate cookie key\nexport HTTP_COOKIE_KEY=$(openssl rand -base64 64)\n\n# Run with minimal config\nEXTERNAL_BASE=http://localhost:8080 \\\nHTTP_COOKIE_KEY=$HTTP_COOKIE_KEY \\\nISSUER_DID=did:plc:test123 \\\nAIP_HOSTNAME=test.com \\\nAIP_CLIENT_ID=test \\\nAIP_CLIENT_SECRET=test \\\nDATABASE_URL=postgres://user:pass@localhost:5432/ifthisthenat \\\ncargo run\n```\n\nThis will start ifthisthenat with:\n- HTTP server on port 8080 (default)\n- In-memory caching only\n- MPSC queue adapter for async processing\n- Jetstream consumer enabled (default)\n- Scheduler enabled for periodic tasks (default)\n- Noop throttling and denylist (default)\n\n## Architecture\n\n```\nJetstream → Consumer → Blueprint Cache → Evaluator → Queue → Actions\n    ↓           ↓            ↓              ↓          ↓        ↓\nWebhooks    Partitions   Memory/Redis   DataLogic   Redis   Publish/\nSchedules                                          /Memory   Webhook\n```\n\n### Core Components\n\n1. **Blueprint System** (`src/storage/blueprint.rs`, `src/processor.rs`)\n   - Blueprints define automation workflows\n   - Each blueprint has an AT-URI identifier\n   - Ordered node evaluation with early termination\n\n2. **Node Types** (`src/engine/`)\n   - Entry nodes: `jetstream_entry`, `webhook_entry`, `periodic_entry`, `zap_entry`\n   - Processing nodes: `condition`, `transform`, `facet_text`, `sentiment_analysis`, `parse_aturi`, `get_record`\n   - Action nodes: `publish_record`, `publish_webhook_direct`, `publish_webhook_queue`, `debug_action`\n\n3. **Evaluation Engine** (`src/engine/evaluator.rs`)\n   - DataLogic expression evaluation\n   - Sequential node processing\n   - Async evaluation via queue adapters\n\n4. **Consumer System** (`src/consumer.rs`)\n   - Jetstream event processing with partitioning\n   - Multi-worker thread support\n   - Redis or file-based cursor persistence\n\n5. **Task System** (`src/tasks/`)\n   - Blueprint evaluation with queue adapters\n   - Webhook delivery with retry logic\n   - Periodic task scheduling\n   - Task lifecycle management\n\n6. **Storage Layer** (`src/storage/`)\n   - PostgreSQL for persistent data\n   - Redis for caching and queues (optional)\n   - Binary serialization for efficiency\n   - Blueprint and node caching with multiple backends\n   - Session management for OAuth\n   - Evaluation result storage (filesystem or noop)\n\n## Configuration\n\n### Environment Variables\n\nConfiguration follows the 12-factor app methodology using environment variables exclusively.\n\n#### Required Variables\n- `EXTERNAL_BASE`: External base URL\n- `HTTP_COOKIE_KEY`: Cookie encryption key (generate with `openssl rand -base64 64`)\n- `ISSUER_DID`: Service DID (valid did:plc or did:web)\n- `AIP_HOSTNAME`: AT Protocol OAuth hostname\n- `AIP_CLIENT_ID`: OAuth client ID\n- `AIP_CLIENT_SECRET`: OAuth client secret\n\n#### Core Service (Optional)\n- `HTTP_PORT`: Server port (default: 8080)\n- `HTTP_STATIC_PATH`: Static file path (default: `static`)\n- `HTTP_CLIENT_TIMEOUT`: Client timeout in seconds (default: 8)\n- `USER_AGENT`: Custom user agent (default: `ifthisthenat/VERSION`)\n- `DATABASE_URL`: PostgreSQL URL (default: `postgres://username:password@localhost:5432/ifthisthenat`)\n\n#### OAuth/Authentication\n- `AIP_OAUTH_SCOPE`: OAuth scope (default: `openid email profile atproto account:email`)\n- `ADMIN_DIDS`: Semicolon-separated admin DIDs\n- `ALLOWED_IDENTITIES`: Comma-separated allowed DIDs\n\n#### Jetstream Configuration\n- `JETSTREAM_ENABLED`: Enable Jetstream consumer (default: true)\n- `JETSTREAM_COLLECTIONS`: Comma-separated collections to monitor\n- `JETSTREAM_WORKER_THREADS`: Number of worker threads (default: 1)\n- `JETSTREAM_PARTITION`: Partition config (format: `INDEX:TOTAL`)\n- `JETSTREAM_PARTITION_STRATEGY`: Partitioning strategy (default: `metrohash`)\n- `JETSTREAM_CURSOR_PATH`: File path for cursor persistence\n\n#### Webhook Queue\n- `WEBHOOK_QUEUE_ENABLED`: Enable webhook queue (default: false)\n- `WEBHOOK_MAX_CONCURRENT`: Max concurrent webhooks (default: 10)\n- `WEBHOOK_DEFAULT_TIMEOUT_MS`: Default timeout in ms (default: 30000)\n- `WEBHOOK_MAX_RETRIES`: Max retry attempts (default: 3)\n- `WEBHOOK_RETRY_DELAY_MS`: Delay between retries (default: 1000)\n- `WEBHOOK_LOG_BODIES`: Log webhook bodies (default: false)\n\n#### Blueprint Processing\n- `BLUEPRINT_QUEUE_ADAPTER`: Queue adapter type (`mpsc` or `redis`, default: `mpsc`)\n- `BLUEPRINT_QUEUE_BUFFER_SIZE`: MPSC buffer size (default: 10000)\n- `BLUEPRINT_QUEUE_MAX_RETRIES`: Max retries (default: 3)\n- `BLUEPRINT_THROTTLER`: Throttler type (`noop`, `redis`, `redis-per-identity`, default: `noop`)\n- `BLUEPRINT_THROTTLER_AUTHORITY_LIMIT`: Authority rate limit\n- `BLUEPRINT_THROTTLER_AUTHORITY_WINDOW`: Authority window in seconds\n- `BLUEPRINT_THROTTLER_ATURI_LIMIT`: AT-URI rate limit\n- `BLUEPRINT_THROTTLER_ATURI_WINDOW`: AT-URI window in seconds\n\n#### Scheduler\n- `SCHEDULER_ENABLED`: Enable periodic scheduler (default: true)\n- `SCHEDULER_CHECK_INTERVAL_SECS`: Check interval (default: 60)\n- `SCHEDULER_CACHE_RELOAD_SECS`: Cache reload interval (default: 300)\n- `SCHEDULER_MAX_CONCURRENT`: Max concurrent evaluations (default: 10)\n\n#### Monitoring\n- `SENTRY_DSN`: Sentry DSN for error tracking\n- `SENTRY_ENVIRONMENT`: Environment name (default: `development`)\n- `SENTRY_TRACES_SAMPLE_RATE`: Trace sampling rate (0.0-1.0, default: 0.01)\n- `SENTRY_DEBUG`: Enable Sentry debug mode (default: false)\n\n#### Advanced\n- `REDIS_URL`: Redis connection URL (enables advanced features)\n- `REDIS_CURSOR_TTL_SECONDS`: TTL for Redis cursors (default: 86400)\n- `DENYLIST_TYPE`: Denylist type (`noop` or `postgres`, default: `noop`)\n- `DISABLED_NODE_TYPES`: Comma-separated node types to disable\n- `EVALUATION_STORAGE_TYPE`: Storage type (`noop` or `filesystem`, default: `noop`)\n- `EVALUATION_STORAGE_DIRECTORY`: Directory for filesystem storage\n\n### Production Configuration Examples\n\n#### Single-Instance Production\n\n```bash\nEXTERNAL_BASE=https://iftta.example.com \\\nHTTP_PORT=3000 \\\nHTTP_COOKIE_KEY=$COOKIE_KEY \\\nISSUER_DID=did:plc:yourservice \\\nAIP_HOSTNAME=your-oauth.com \\\nAIP_CLIENT_ID=your-client-id \\\nAIP_CLIENT_SECRET=your-client-secret \\\nDATABASE_URL=postgres://user:pass@localhost/iftta \\\nJETSTREAM_ENABLED=true \\\nWEBHOOK_QUEUE_ENABLED=true \\\nDENYLIST_TYPE=postgres \\\nSENTRY_DSN=your-sentry-dsn \\\nSENTRY_ENVIRONMENT=production \\\nRUST_LOG=info \\\n./target/release/ifthisthenat\n```\n\n#### Multi-Instance with Redis (High Availability)\n\n```bash\nEXTERNAL_BASE=https://iftta.example.com \\\nHTTP_PORT=3000 \\\nHTTP_COOKIE_KEY=$COOKIE_KEY \\\nISSUER_DID=did:plc:yourservice \\\nAIP_HOSTNAME=your-oauth.com \\\nAIP_CLIENT_ID=your-client-id \\\nAIP_CLIENT_SECRET=your-client-secret \\\nDATABASE_URL=postgres://user:pass@pghost/iftta \\\nREDIS_URL=redis://redishost:6379 \\\nJETSTREAM_ENABLED=true \\\nJETSTREAM_WORKER_THREADS=4 \\\nJETSTREAM_PARTITION=0:3 \\\nBLUEPRINT_QUEUE_ADAPTER=redis \\\nBLUEPRINT_THROTTLER=redis-per-identity \\\nWEBHOOK_QUEUE_ENABLED=true \\\nDENYLIST_TYPE=postgres \\\nSENTRY_DSN=your-sentry-dsn \\\nSENTRY_ENVIRONMENT=production \\\nRUST_LOG=info \\\n./target/release/ifthisthenat\n```\n\n## API Endpoints\n\n### XRPC Endpoints\n- `GET /xrpc/tools.graze.ifthisthenat.getBlueprints` - List blueprints\n- `GET /xrpc/tools.graze.ifthisthenat.getBlueprint` - Get specific blueprint\n- `POST /xrpc/tools.graze.ifthisthenat.updateBlueprint` - Create/update blueprint\n- `POST /xrpc/tools.graze.ifthisthenat.deleteBlueprint` - Delete blueprint\n\n### Health \u0026 Monitoring\n- `GET /_health` - Basic health check\n- `GET /_health/ready` - Readiness check (all components)\n- `GET /_health/live` - Liveness check\n- `GET /_health/components` - Individual component status\n\n### OAuth Flow\n- `GET /oauth/authorize` - Start OAuth flow\n- `GET /oauth/callback` - OAuth callback\n- `POST /oauth/refresh` - Refresh tokens\n\n### Static Files\n- `GET /.well-known/atproto-did` - AT Protocol DID document\n- `GET /.well-known/did.json` - DID document\n- `GET /` - Landing page\n\n## Docker Deployment\n\n### Quick Docker Setup\n\n```bash\n# Build the image\ndocker build -t ifthisthenat:latest .\n\n# Run with environment file\ndocker run -d \\\n  --name ifthisthenat \\\n  --env-file .env \\\n  -p 8080:8080 \\\n  ifthisthenat:latest\n```\n\n### Docker Compose\n\n```yaml\nversion: '3.8'\nservices:\n  app:\n    build: .\n    ports:\n      - \"8080:8080\"\n    env_file: .env\n    depends_on:\n      - postgres\n      - redis\n    \n  postgres:\n    image: postgres:16-alpine\n    environment:\n      POSTGRES_DB: ifthisthenat\n      POSTGRES_USER: iftta\n      POSTGRES_PASSWORD: secret\n    volumes:\n      - postgres_data:/var/lib/postgresql/data\n  \n  redis:\n    image: redis:7-alpine\n    volumes:\n      - redis_data:/data\n\nvolumes:\n  postgres_data:\n  redis_data:\n```\n\n## DataLogic Expression System\n\nThe engine uses DataLogic for dynamic evaluation. Common patterns:\n\n```json\n// Value extraction\n{\"val\": [\"path\", \"to\", \"field\"]}\n\n// Comparisons\n{\"==\": [{\"val\": [\"field\"]}, \"value\"]}\n\n// String operations\n{\"cat\": [\"prefix-\", {\"val\": [\"field\"]}, \"-suffix\"]}\n\n// Time functions\n{\"now\": []}\n{\"datetime\": [1234567890]}\n\n// Complex conditions\n{\"and\": [\n  {\"==\": [{\"val\": [\"kind\"]}, \"commit\"]},\n  {\"contains\": [{\"val\": [\"collection\"]}, \"app.bsky.feed.post\"]}\n]}\n\n// Hash functions for unique IDs\n{\"metrohash\": [\"string1\", \"string2\"]}\n\n// Parse AT-URI components\n{\"parse_aturi\": [\"at://did:plc:example/app.bsky.feed.post/abc\"]}\n\n// Process text with facets/mentions\n{\"facet_text\": [\"Hello @user.bsky.social!\"]}\n```\n\n## Development\n\n### Project Structure\n\n```\nsrc/\n├── bin/              # Entry point\n├── engine/           # Node evaluation engine\n├── storage/          # Database and cache layers\n├── http/             # HTTP server and handlers\n├── tasks/            # Background task system\n│   ├── blueprint.rs          # Blueprint evaluation task\n│   ├── blueprint_adapter.rs  # Queue adapter for blueprints\n│   ├── webhook.rs            # Webhook delivery task\n│   ├── scheduler.rs          # Periodic task scheduler\n│   └── manager.rs            # Task lifecycle management\n├── consumer.rs       # Jetstream consumer\n├── processor.rs      # Blueprint processor\n├── metrics.rs        # Metrics publishing\n├── serialization.rs  # Binary serialization\n└── errors.rs         # Error definitions\n```\n\n### Running Tests\n\n```bash\n# Run all tests\ncargo test\n\n# Run specific test\ncargo test test_name\n\n# Run with output\ncargo test -- --nocapture\n\n# Run integration tests\ncargo test --test '*'\n```\n\n### Development Tools\n\n```bash\n# Check code\ncargo check\n\n# Format code\ncargo fmt\n\n# Lint code\ncargo clippy\n\n# Watch for changes\ncargo watch -x test -x run\n```\n\n## Documentation\n\n- [API Documentation](API.md) - Blueprint structure and node payloads\n- [Development Guide](CLAUDE.md) - Architecture details and patterns\n- [Contributing Guide](CONTRIBUTING.md) - How to contribute\n- [Changelog](CHANGELOG.md) - Release history and changes\n\n## Contributing\n\nContributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.\n\n## License\n\nThis project is licensed under the MIT License. Copyright © 2025 Nick Gerakines and Graze Social. See the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- AT Protocol team for the protocol and specifications\n- Bluesky team for Jetstream and the firehose\n- Rust community for excellent libraries and tooling\n- Contributors and early adopters for feedback and improvements\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraze-social%2Fiftta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgraze-social%2Fiftta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraze-social%2Fiftta/lists"}