{"id":30766661,"url":"https://github.com/george-swift/expender-backend","last_synced_at":"2026-05-08T07:31:34.285Z","repository":{"id":309113888,"uuid":"1029996926","full_name":"george-swift/expender-backend","owner":"george-swift","description":"Serverless AI-powered expense tracker with receipt scanning, multi-tier quotas, and real-time processing. Built on AWS","archived":false,"fork":false,"pushed_at":"2025-08-12T22:54:58.000Z","size":422,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"feature/mvp","last_synced_at":"2025-09-04T19:44:56.079Z","etag":null,"topics":["ai","aws","expense-tracker","graphql","ocr","python","serverless","terraform"],"latest_commit_sha":null,"homepage":"https://expender.vercel.app","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/george-swift.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-31T23:29:24.000Z","updated_at":"2025-08-12T22:55:02.000Z","dependencies_parsed_at":"2025-08-10T00:14:02.372Z","dependency_job_id":"ca77eb72-93ae-40cd-a67a-8c4e098aea52","html_url":"https://github.com/george-swift/expender-backend","commit_stats":null,"previous_names":["george-swift/expender-backend"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/george-swift/expender-backend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-swift%2Fexpender-backend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-swift%2Fexpender-backend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-swift%2Fexpender-backend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-swift%2Fexpender-backend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/george-swift","download_url":"https://codeload.github.com/george-swift/expender-backend/tar.gz/refs/heads/feature/mvp","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-swift%2Fexpender-backend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32770980,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T02:36:36.067Z","status":"ssl_error","status_checked_at":"2026-05-08T02:36:07.210Z","response_time":54,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","aws","expense-tracker","graphql","ocr","python","serverless","terraform"],"created_at":"2025-09-04T19:31:31.233Z","updated_at":"2026-05-08T07:31:34.272Z","avatar_url":"https://github.com/george-swift.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Expender - Serverless Expense Management Platform\n\nA comprehensive serverless expense tracking and management platform built on AWS. Expender combines AI-powered receipt scanning (SmartScan) with intelligent expense categorization, multi-tier user quotas, and real-time processing capabilities.\n\n## Features\n\n### Core Functionality\n\n- **📊 Expense Management**: Create, read, update, and delete individual expenses with full CRUD operations\n- **🗂️ Batch Operations**: Process multiple expenses simultaneously for bulk import/export workflows\n- **📈 Data Export**: Export expenses to CSV format with customizable field selection\n- **🔍 Smart Filtering**: Query expenses by date ranges, categories, and other criteria using DynamoDB GSI\n\n### SmartScan Technology\n\n- **🤖 AI-Powered Receipt Scanning**: Automated receipt processing using AWS Textract for data extraction\n- **🧠 Intelligent Categorization**: OpenAI GPT-powered expense categorization based on merchant and line items\n- **📷 Multi-Format Support**: Process images (JPEG, PNG) and PDF receipts up to 5MB\n- **💱 Multi-Currency Detection**: Automatic currency recognition across 18+ global currencies\n- **⚡ Real-Time Processing**: GraphQL subscriptions for live SmartScan result updates\n\n### User Management \u0026 Security\n\n- **🔐 Clerk Authentication**: Secure user authentication and authorization via Clerk\n- **📋 Multi-Tier Quotas**: Free (30 scans/month) and Pro (unlimited) plan management\n- **🚫 Rate Limiting**: API Gateway throttling with tier-specific limits for optimal performance\n- **🗂️ Data Isolation**: Complete user data separation with secure deletion workflows\n\n### Infrastructure \u0026 Monitoring\n\n- **⚡ Serverless Architecture**: 100% serverless with auto-scaling Lambda functions\n- **📊 CloudWatch Integration**: Comprehensive logging and monitoring with correlation\n- **🔄 Event-Driven Design**: EventBridge, DynamoDB Streams, and S3 triggers for reactive processing\n- **🌍 Global CDN**: CloudFront distribution for optimized file delivery\n- **🛡️ Production-Ready**: Comprehensive error handling and middleware for reliability\n\n## Architecture Overview\n\n![Expender AWS Architecture Diagram](./expender_architecture.png)\n\n_Comprehensive AWS serverless architecture for Expender, showing the complete data flow from user authentication, expense management, receipt processing and real-time notifications through to account lifecycle management. Created with Amazon Q_\n\n### Serverless Components\n\n**API Layer:**\n\n- **AWS API Gateway**: RESTful API with Lambda authorization and comprehensive rate limiting\n- **AWS Lambda**: Event-driven functions for business logic processing\n- **AWS Chalice**: Python framework for rapid serverless API development\n\n**Data Storage:**\n\n- **DynamoDB**: NoSQL database with streams for real-time event processing\n  - `expenses`: Main expense records with date/category GSIs\n  - `smartscans`: Temporary SmartScan results with TTL\n  - `quotas`: User plan management and usage tracking\n- **S3**: Secure file storage for receipt images and documents\n- **CloudFront**: Global CDN for optimized file delivery\n\n**AI \u0026 Processing:**\n\n- **AWS Textract**: Automated document text and data extraction\n- **OpenAI GPT**: Intelligent expense categorization with structured JSON responses\n- **AppSync GraphQL**: Real-time subscriptions for SmartScan processing updates\n\n**Event-Driven Architecture:**\n\n- **EventBridge**: Custom event bus for user lifecycle management\n- **DynamoDB Streams**: Real-time data change processing with filtering\n- **Step Functions**: Orchestrated user data deletion workflows\n- **S3 Event Notifications**: Automatic receipt processing triggers\n\n**Monitoring \u0026 Security:**\n\n- **AWS X-Ray**: Distributed tracing for performance monitoring\n- **CloudWatch Logs**: Centralized logging with structured correlation\n- **IAM**: Fine-grained permissions with least-privilege access\n- **Clerk Webhooks**: Secure user lifecycle event handling\n\n## Prerequisites\n\nBefore setting up Expender, ensure you have the following installed:\n\n- **Python 3.12+**: Required for AWS Chalice and application code\n- **AWS CLI**: Configured with appropriate IAM permissions\n- **Terraform**: Version 1.0.0+ for infrastructure provisioning\n- **AWS Chalice**: Python serverless framework (`pip install chalice`)\n- **Node.js**: For integration with [the frontend](https://github.com/george-swift/expender) (optional)\n\n### Required AWS Services Access\n\n- Lambda, API Gateway, DynamoDB, S3, CloudFront\n- Textract, EventBridge, Step Functions, AppSync\n- IAM, CloudWatch, X-Ray\n\n### External Service Requirements\n\n- **Clerk Account**: For user authentication ([clerk.com](https://clerk.com/))\n- **OpenAI API Key**: For AI-powered categorization ([openai.com](https://openai.com/))\n\n## Installation\n\n### 1. Clone the Repository\n\n```bash\ngit clone https://github.com/george-swift/expender-backend.git\ncd expender-backend\n```\n\n### 2. Set Up Python Environment\n\n```bash\n# Create virtual environment\npython3 -m venv venv\n\n# Activate virtual environment\nsource venv/bin/activate  # On macOS/Linux\n# OR\nvenv\\Scripts\\activate     # On Windows\n\n# Install dependencies\npip install -r requirements.txt\npip install -r requirements-dev.txt\n```\n\n### 3. Install Terraform\n\n```bash\n# macOS (using Homebrew)\nbrew install terraform\n\n# Or download from: https://www.terraform.io/downloads\n```\n\n### 4. Configure AWS CLI\n\n```bash\naws configure\n# Provide your AWS Access Key ID, Secret, and default region\n```\n\n## Configuration\n\n### Environment Variables\n\nCreate Terraform variable files for each environment:\n\n#### `vars_dev.tfvars`\n\n```hcl\n# AWS Configuration\naws_region     = \"us-east-1\"\nenvironment    = \"dev\"\n\n# Frontend URLs\nfrontend_app_url     = \"https://dev.app.url.here\"\nfrontend_dev_app_url = \"http://localhost:PORT\"\n\n# External Service Keys (use AWS Secrets Manager in production)\nclerk_secret_key              = \"your-clerk-secret-key\"\nclerk_webhook_signing_secret  = \"your-clerk-webhook-secret\"\nopenai_api_key               = \"your-openai-api-key\"\nsmartscan_encryption_key     = \"your-32-character-encryption-key\"\n```\n\n#### `vars_prod.tfvars`\n\n```hcl\n# AWS Configuration\naws_region     = \"us-east-1\"\nenvironment    = \"prod\"\n\n# Frontend URLs\nfrontend_app_url     = \"https://prod.app.url.here\"\nfrontend_dev_app_url = \"https://dev.app.url.here\"\n\n# External Service Keys (AWS Secrets Manager preferably)\nclerk_secret_key              = \"your-production-clerk-secret-key\"\nclerk_webhook_signing_secret  = \"your-production-clerk-webhook-secret\"\nopenai_api_key               = \"your-production-openai-api-key\"\nsmartscan_encryption_key     = \"your-production-32-character-encryption-key\"\n```\n\n### Chalice Configuration\n\nThe application uses Chalice's automatic configuration. See full settings in `.chalice/config.json`:\n\n```json\n{\n  \"version\": \"2.0\",\n  \"app_name\": \"expender\",\n  \"manage_iam_role\": false,\n  \"iam_role_arn\": \"${aws_iam_role.api_service_role.arn}\",\n  \"automatic_layer\": true,\n  \"xray\": true,\n  \"stages\": {\n    \"dev\": {},\n    \"prod\": {}\n  }\n}\n```\n\n### Security Configuration\n\n**Environment Variables (Never commit to Git):**\n\n- Store sensitive values in `*.tfvars` files (excluded by `.gitignore`)\n- Use AWS Secrets Manager for production environments (PREFERRED)\n- Rotate keys regularly and follow security best practices\n\n## Deployment\n\n### Development Environment\n\nDeploy to development using the provided Makefile:\n\n```bash\n# Deploy to development environment\nmake deploy-dev\n```\n\nThis command:\n\n1. Formats Python code with Black\n2. Packages the Chalice application for Terraform\n3. Applies infrastructure patches for multi-environment compatibility\n4. Deploys infrastructure using Terraform\n\n### Production Environment\n\n```bash\n# Deploy to production environment\nmake deploy-prod\n```\n\n### Manual Deployment Steps\n\nIf you prefer manual deployment:\n\n```bash\n# 1. Format code\nblack .\n\n# 2. Package Chalice application\nchalice package --pkg-format terraform . --stage dev\n\n# 3. Patch Terraform configuration\npython3 scripts/patch_chalice_tf.py\n\n# 4. Initialize Terraform (first time only)\nterraform init\n\n# 5. Plan deployment\nterraform plan -var-file=vars_dev.tfvars\n\n# 6. Apply infrastructure\nterraform apply -var-file=vars_dev.tfvars\n```\n\n### Clean Up Resources\n\n```bash\n# Remove generated files\nmake clean\n\n# Destroy infrastructure (be careful!)\nterraform destroy -var-file=vars_dev.tfvars\n```\n\n## Usage\n\n### API Endpoints\n\nThe API is documented in the Swagger specification (`expender-dev-swagger-postman.json`). Key endpoints include:\n\n#### Expense Management\n\n```bash\n# List user expenses\nGET /expenses\nAuthorization: {clerk-jwt-token}\n\n# Create single expense\nPOST /expenses\nAuthorization: {clerk-jwt-token}\nContent-Type: application/json\n{\n  \"amount\": 25.99,\n  \"merchant\": \"Coffee Shop\",\n  \"category\": \"Meals and Entertainment\",\n  \"date\": \"2025-08-07\",\n  \"currency\": \"USD\",\n  \"description\": \"Team meeting coffee\"\n}\n\n# Batch create expenses\nPOST /expenses/batch\nAuthorization: {clerk-jwt-token}\nContent-Type: application/json\n{\n  \"expenses\": [\n    {\n      \"amount\": 15.50,\n      \"currency\": \"USD\",\n      \"merchant\": \"Amazing Lunch Place\",\n      \"category\": \"Meals and Entertainment\"\n    },\n    {\n      \"amount\": 8.99,\n      \"currency\": \"USD\",\n      \"merchant\": \"Awesome Office Supplies\",\n      \"category\": \"Office Supplies\"\n    }\n  ]\n}\n\n# Update expense\nPUT /expenses/{expense_id}\nAuthorization: {clerk-jwt-token}\nContent-Type: application/json\n{\n  \"amount\": 27.99,\n  \"category\": \"Professional Services\"\n}\n\n# Delete expense\nDELETE /expenses/{expense_id}\nAuthorization: {clerk-jwt-token}\n\n# Export expenses to CSV\nPOST /expenses/export\nAuthorization: {clerk-jwt-token}\nContent-Type: application/json\n{\n  \"format\": \"csv\",\n  \"attributes\": [\"date\", \"merchant\", \"amount\", \"category\"]\n}\n```\n\n#### SmartScan Operations\n\n```bash\n# Initiate SmartScan (get upload URL)\nPOST /smartscans\nAuthorization: {clerk-jwt-token}\nContent-Type: application/json\n{\n  \"contentType\": \"image/jpeg\"\n}\n\n# Response includes presigned S3 upload URL and fields\n# Upload receipt image using the provided URL and fields\n# Results will be available via GraphQL subscription\n```\n\n#### User Quotas\n\n```bash\n# Check user quota and usage\nGET /quotas\nAuthorization: {clerk-jwt-token}\n```\n\n### GraphQL API (SmartScan Results)\n\n```graphql\n# Subscribe to SmartScan results\nsubscription GetSmartScanResult($userId: String!, $scanId: String!) {\n  getSmartScanResult(userId: $userId, scanId: $scanId) {\n    userId\n    scanId\n    result {\n      merchant\n      amount\n      currency\n      date\n      category\n      confidence\n      lineItems {\n        description\n        amount\n        quantity\n      }\n    }\n    objectKey\n    createdAt\n  }\n}\n```\n\n### Rate Limits\n\nThe API implements tiered rate limiting:\n\n**Free Tier Users:**\n\n- 1,000 requests/day\n- 10 requests/second (burst: 20)\n- 30 SmartScans/month\n\n**Pro Tier Users:**\n\n- 10,000 requests/day\n- 50 requests/second (burst: 100)\n- Unlimited SmartScans\n\n**Endpoint-Specific Limits:**\n\n- SmartScan: 5-10 requests/second (resource intensive)\n- Export: 2-5 requests/second (memory intensive)\n- Webhooks: 50-100 requests/second (external callbacks)\n\n## Troubleshooting\n\n### Common Issues\n\n#### 1. Deployment Failures\n\n**CloudWatch Logs Role Error:**\n\n```\nError: BadRequestException: CloudWatch Logs role ARN must be set\n```\n\n**Solution:** The deployment automatically creates the required CloudWatch role. Ensure your AWS account has API Gateway permissions.\n\n#### 2. Authentication Issues\n\n**Unauthorized API Calls:**\n\n```json\n{\n  \"message\": \"Unauthorized\"\n}\n```\n\n**Solution:** Ensure you're passing a valid Clerk JWT token in the Authorization header.\n\n#### 3. SmartScan Processing\n\n**Textract Analysis Failures:**\n\n- Ensure uploaded files are under 5MB\n- Supported formats: JPEG, PNG, PDF\n- Check file is properly uploaded to S3 before processing\n\n**AI Categorization Issues:**\n\n- Verify OpenAI API key is valid and has sufficient credits\n- Check CloudWatch logs for detailed error messages\n\n#### 4. Rate Limiting\n\n**Too Many Requests (429):**\n\n- Check your user tier and current usage\n- Implement exponential backoff in client applications\n- Consider upgrading to Pro tier for higher limits\n\n### Debug Tools\n\n**CloudWatch Logs:**\n\n```bash\n# View API logs\naws logs tail /aws/lambda/expender-dev --follow\n\n# View specific function logs\naws logs tail /aws/lambda/expender-dev-bucket_event_handler --follow\n```\n\n**X-Ray Tracing:**\n\n- Visit AWS X-Ray console for distributed trace analysis\n- Track request flows across Lambda functions and AWS services\n\n**DynamoDB Monitoring:**\n\n- Monitor table metrics in CloudWatch\n- Check stream processing in DynamoDB console\n\n## Contributing\n\nContributions to Expender are welcome! Please see [Contributing Guidelines](./CONTRIBUTING.md) for detailed information about:\n\n- Code of conduct\n- Development workflow\n- Pull request process\n- Coding standards and best practices\n- Testing requirements\n\n## License\n\nThis project is licensed under the terms specified in the [LICENSE.md](./LICENSE.md) file. Please review the license before using or contributing to this project.\n\n## Contact Information \u0026 Support\n\n### Getting Help\n\n**🐛 Bug Reports \u0026 Feature Requests:**\n\n- [Open an issue](https://github.com/george-swift/expender-backend/issues) on GitHub\n- Include detailed reproduction steps and environment information as outlined in [Contributing Guidelines](./CONTRIBUTING.md)\n\n**📧 Direct Support:**\n\n- Email: [support@expender.app](mailto:support@expender.app)\n- Response time: 24-48 hours for general inquiries\n\n**📚 Documentation:**\n\n- API Documentation: Import `expender-swagger-postman.json` in Postman to see the collection.\n- Architecture Details: See inline code documentation and comments\n- Infrastructure: Comprehensive Terraform module documentation\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeorge-swift%2Fexpender-backend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeorge-swift%2Fexpender-backend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeorge-swift%2Fexpender-backend/lists"}