https://github.com/cansinacarer/maillistshield-com
Mail List Shield - SaaS
https://github.com/cansinacarer/maillistshield-com
continuous-deployment continuous-integration dev-containers docker event-driven-microservices flask full-stack-web-development github-actions loki object-relational-mapping rabbitmq sphinx sphinx-autoapi stripe
Last synced: 3 months ago
JSON representation
Mail List Shield - SaaS
- Host: GitHub
- URL: https://github.com/cansinacarer/maillistshield-com
- Owner: cansinacarer
- License: other
- Created: 2024-10-21T01:53:02.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-12-03T23:22:54.000Z (6 months ago)
- Last Synced: 2025-12-07T04:16:37.492Z (6 months ago)
- Topics: continuous-deployment, continuous-integration, dev-containers, docker, event-driven-microservices, flask, full-stack-web-development, github-actions, loki, object-relational-mapping, rabbitmq, sphinx, sphinx-autoapi, stripe
- Language: Python
- Homepage: https://maillistshield.com
- Size: 10.7 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- Changelog: CHANGELOG.md
- License: LICENSE.md
Awesome Lists containing this project
README
An email validation SaaS built with microservices architecture
## Architecture
```mermaid
sequenceDiagram
participant flask as 🌶️ 1. Flask Application
participant db as 🐘 Postgres Database
participant s3 as ☁️ S3 Bucket
participant fis as 📁 2. File Intake Service
participant f2vqp as 📤 3. File to Validation Queue Publisher
participant rabbit as 🐰 RabbitMQ
participant vo as ⚙️ 5. Validation Orchestrator
participant evw as ✉️ 4. Email Validation Worker
participant rfg as 📊 6. Results File Generator
flask ->> s3: Uploads a batch validation job to validation/uploaded/
flask ->> db: Records the job as pending_start
s3 ->> fis: Clean up the file, calculate cost
fis ->> db: Deduct credits from user, update status to file_accepted
fis ->> s3: Upload cleaned file to validation/in-progress/
s3 ->> f2vqp: Parse the cleaned file
f2vqp ->> rabbit: Create a queue per file, publish each row as a message
f2vqp ->> db: Update status to file_queued
rabbit ->> vo: Consume N message per queue with Round-Robin
vo ->> evw: API call to send each message, retrieve result
vo ->> db: Update progress
db ->> flask: Update progress in the UI
vo ->> rabbit: Enqueue the results in each files' queue
rabbit ->> rfg: Drain results queue of the file when expected message count is reached, build result file
rfg ->> s3: Upload the result file to /validation/completed
rfg ->> db: Set status to file_validation_in_progress or file_completed, save the name of the results file
db ->> flask: Generate download link for results file
```
This application consists of 6 event driven services:
1. [Flask SaaS](https://github.com/cansinacarer/maillistshield-com) (this repository)
2. [File Intake Service](https://github.com/cansinacarer/maillistshield-file-intake-service)
3. [File to Validation Queue Publisher](https://github.com/cansinacarer/maillistshield-file-to-validation-queue-publisher)
4. [Email Validation Worker](https://github.com/cansinacarer/maillistshield-validation-worker)
5. [Validation Orchestrator](https://github.com/cansinacarer/maillistshield-validation-orchestrator)
6. [Results File Generator](https://github.com/cansinacarer/maillistshield-results-file-generator)
[See a more detailed architecture diagram →](https://app.diagrams.net/#Uhttps://raw.githubusercontent.com/cansinacarer/maillistshield-com/main/docs/drawio/mls-service-architecture.drawio)
## Tech Stack
| Category | Technologies |
|----------|-------------|
| **Backend** | Python, Flask |
| **Database** | PostgreSQL, SQLAlchemy ORM |
| **Message Queue** | RabbitMQ |
| **Infrastructure** | Docker, AWS S3, CapRover (PaaS deployment) |
| **Security** | OAuth 2.0, TOTP 2FA, reCAPTCHA, CSRF/XSS protection |
| **Observability** | Loki, Grafana, Uptime Kuma |
| **CI/CD** | GitHub Actions, Semantic Release |
| **Payments** | Stripe (subscriptions + one-time purchases) |
## Database Model
See the ER diagram
```mermaid
erDiagram
Users {
int id PK
string email
string password
string role
string stripe_customer_id
int tier_id FK
bigint credits
datetime cancel_at
string firstName
string lastName
int newsletter
datetime member_since
datetime last_login
string email_confirmation_code
datetime last_confirmation_codes_sent
int number_of_email_confirmation_codes_sent
int email_confirmed
string google_avatar_url
boolean avatar_uploaded
string totp_secret
int totp_enabled
}
Tiers {
int id PK
string name
string label
string stripe_price_id
}
APIKeys {
int id PK
int user_id FK
string key_hash
string label
datetime created_at
datetime expires_at
datetime last_used
boolean is_active
}
BatchJobs {
int id PK
string uid
int user_id FK
string status
string original_file_name
string uploaded_file
string accepted_file
string results_file
int row_count
bigint last_pick_row
datetime last_pick_time
string source
int header_row
string email_column
datetime uploaded
datetime started
datetime finished
string result
}
Users }o--|| Tiers : "has"
Users ||--o{ APIKeys : "owns"
Users ||--o{ BatchJobs : "creates"
```
## Features of the Flask Application
### 🧑💻 Developer Experience
- Dev containers:
- **Flask** container with pre-configured with:
- VSCode launch.json for debugging the Flask app,
- Prettier for HTML, CSS, and JS formatting,
- Pre-commit hooks for code quality checks,
- Markdownlint for Markdown formatting,
- Black for Python code formatting,
- Commitlint for commit message linting.
- **Postgres** as a development database,
- **pgAdmin** pre-connected to the development,
- **docs** serving the built html files of the Sphinx documentation..
- CI/CD pipelines with GitHub Actions to:
- Run pre-commit hooks,
- Run tests,
- Automate semantic release for versioning and changelog generation,
- Build and deploy the documentation,
- Build and deploy the app to production.
### ☁️ Deployment
- 🐳 Dockerized Flask for stateless continuous deployment for scalability,
- 🗄️ Database model abstracted with ORM,
- 📦 S3 object storage with pre-signed URLs.
### 💳 Stripe Integrations
- Subscriptions,
- Different subscription tiers,
- Billing page with Invoices,
- Integration mechanism:
- To begin a subscription, we send the user to Stripe with a checkout session,
- Then listen to Stripe webhook events to process the results,
- We set the Products in Stripe, then insert their prices into the Tiers table.
- One-off credit purchases for pre-paid metered usage.
### 🔒 Authentication
- Sign up flow,
- Sign up with Google option,
- Email validation requirement,
- Two factor authentication (TOTP only),
- Forgot password flow,
- reCAPTCHA v2 for sign up and login forms,
- Account details page where the user can:
- Upload a profile picture (stored in S3),
- Change profile details like first & last name.
### 📧 Transactional Emails with SMTP
- About Stripe subscription changes:
- Confirmation,
- Cancellation,
- Expiration.
- Email verification on registration,
- Forgot password.
### 🚨 Security
- Cross-Site Request Forgery (CSRF) protection in all forms,
- Rate limiting: App-wide and form specific limits,
- Cross-Site Scripting (XSS) protection,
- Cross-Origin Resource Sharing (CORS) protection.