{"id":43614632,"url":"https://github.com/pyrex41/p_schedule","last_synced_at":"2026-02-04T12:23:48.090Z","repository":{"id":297518302,"uuid":"997050053","full_name":"pyrex41/p_schedule","owner":"pyrex41","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-05T23:11:21.000Z","size":535,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-05T23:20:50.658Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyrex41.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-05T21:52:56.000Z","updated_at":"2025-06-05T23:11:23.000Z","dependencies_parsed_at":"2025-06-05T23:31:48.708Z","dependency_job_id":null,"html_url":"https://github.com/pyrex41/p_schedule","commit_stats":null,"previous_names":["pyrex41/p_schedule"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pyrex41/p_schedule","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyrex41%2Fp_schedule","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyrex41%2Fp_schedule/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyrex41%2Fp_schedule/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyrex41%2Fp_schedule/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyrex41","download_url":"https://codeload.github.com/pyrex41/p_schedule/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyrex41%2Fp_schedule/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29084188,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T03:31:03.593Z","status":"ssl_error","status_checked_at":"2026-02-04T03:29:50.742Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-04T12:23:48.015Z","updated_at":"2026-02-04T12:23:48.085Z","avatar_url":"https://github.com/pyrex41.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Email Scheduling System\n\nA sophisticated, domain-specific language (DSL) based email scheduling system for multi-state compliance with anniversary-based and campaign-based emails.\n\n## Overview\n\nThis email scheduling system manages automated email and SMS campaigns for multiple organizations with up to 3 million contacts. It uses a sophisticated rule engine to determine when to send different types of communications based on contact information, state-specific regulations, and timing constraints.\n\n### Key Features\n\n- **State-specific compliance**: Automatically handles different exclusion rules for different states\n- **Anniversary-based emails**: Birthday, effective date, and AEP emails\n- **Flexible campaign system**: Support for unlimited campaign types with configurable behavior\n- **Load balancing**: Sophisticated smoothing to prevent email clustering\n- **Domain-specific language**: Declarative configuration for business rules\n- **Scalable architecture**: Handles up to 3 million contacts with batch processing\n- **Audit trail**: Comprehensive logging and recovery capabilities\n\n## Architecture\n\n### Core Components\n\n1. **Email Scheduler** (`scheduler.py`) - Main orchestrator\n2. **Campaign Scheduler** (`campaign_scheduler.py`) - Handles campaign-based emails\n3. **State Rules Engine** - Manages state-specific exclusion rules\n4. **Database Manager** - Handles all database operations\n5. **Load Balancer** - Distributes email volume evenly\n6. **Configuration System** - YAML-based declarative configuration\n\n### Email Types\n\n#### Anniversary-Based Emails\n- **Birthday**: Sent 14 days before contact's birthday\n- **Effective Date**: Sent 30 days before policy effective date anniversary\n- **AEP**: Annual Enrollment Period emails (typically September 15)\n- **Post Window**: Catch-up emails sent after exclusion windows end\n\n#### Campaign-Based Emails\n- **Rate Increase**: Advance notification of premium changes\n- **Seasonal Promotions**: Configurable marketing campaigns\n- **Initial Blast**: System introduction emails\n- **Regulatory Notices**: Compliance-required communications\n- **Custom Campaigns**: Flexible campaigns for any purpose\n\n## Installation and Setup\n\n### Prerequisites\n\n- Python 3.8+\n- SQLite3\n- PyYAML\n\n### Install Dependencies\n\n```bash\n# On Ubuntu/Debian\nsudo apt install python3-yaml sqlite3\n\n# On other systems, you may need:\npip install PyYAML\n```\n\n### Database Setup\n\nThe scheduler automatically creates and updates the database schema. No manual setup required.\n\n## Usage\n\n### Basic Usage\n\n```bash\n# Run full scheduling for all contacts\npython3 scheduler.py --db your-database.sqlite3 --run-full\n\n# Use custom configuration\npython3 scheduler.py --db your-database.sqlite3 --config scheduler_config.yaml --run-full\n\n# Enable debug logging\npython3 scheduler.py --db your-database.sqlite3 --run-full --debug\n```\n\n### Campaign Management\n\n```bash\n# Set up sample campaigns\npython3 campaign_scheduler.py --db your-database.sqlite3 --setup-samples\n```\n\n### Configuration\n\nCreate a `scheduler_config.yaml` file to customize behavior:\n\n```yaml\ntiming_constants:\n  send_time: \"08:30:00\"\n  birthday_email_days_before: 14\n  effective_date_days_before: 30\n\nload_balancing:\n  daily_send_percentage_cap: 0.07\n  ed_daily_soft_limit: 15\n\ncampaign_types:\n  rate_increase:\n    respect_exclusion_windows: true\n    enable_followups: true\n    days_before_event: 14\n    priority: 1\n```\n\n## State Exclusion Rules\n\nThe system implements state-specific exclusion windows where no emails should be sent:\n\n### Year-Round Exclusion States\n- **CT** (Connecticut)\n- **MA** (Massachusetts) \n- **NY** (New York)\n- **WA** (Washington)\n\n### Birthday-Based Exclusion Windows\n- **CA**: 30 days before to 60 days after birthday\n- **ID**: 0 days before to 63 days after birthday\n- **KY**: 0 days before to 60 days after birthday\n- **MD**: 0 days before to 30 days after birthday\n- **NV**: 0 days before to 60 days after birthday (uses month start)\n- **OK**: 0 days before to 60 days after birthday\n- **OR**: 0 days before to 31 days after birthday\n- **VA**: 0 days before to 30 days after birthday\n\n### Effective Date-Based Exclusion Windows\n- **MO**: 30 days before to 33 days after effective date anniversary\n\nAll exclusion windows include a 60-day pre-window extension.\n\n## Campaign System\n\n### Campaign Types (Reusable Patterns)\n\nCampaign types define reusable behavior patterns:\n\n```python\nrate_increase = CampaignType(\n    name=\"rate_increase\",\n    respect_exclusion_windows=True,\n    enable_followups=True,\n    timing_rule=TimingRule(days_before_event=14),\n    priority=1\n)\n```\n\n### Campaign Instances (Specific Executions)\n\nCampaign instances represent specific executions with unique templates:\n\n```python\nq1_rate_increase = CampaignInstance(\n    campaign_type=\"rate_increase\",\n    instance_name=\"rate_increase_q1_2025\",\n    email_template=\"rate_increase_standard_v2\",\n    sms_template=\"rate_increase_sms_v1\",\n    active_start_date=date(2025, 1, 1),\n    active_end_date=date(2025, 3, 31)\n)\n```\n\n### Contact Targeting\n\nContacts are linked to campaigns through the `contact_campaigns` table:\n\n```sql\nINSERT INTO contact_campaigns \n(contact_id, campaign_instance_id, trigger_date, status)\nVALUES (123, 1, '2025-07-15', 'pending');\n```\n\n## Database Schema\n\n### Core Tables\n\n#### contacts\n```sql\nid, first_name, last_name, email, zip_code, state, \nbirth_date, effective_date, phone_number\n```\n\n#### email_schedules\n```sql\nid, contact_id, email_type, scheduled_send_date, scheduled_send_time,\nstatus, skip_reason, priority, campaign_instance_id, email_template,\nsms_template, scheduler_run_id, event_year, event_month, event_day\n```\n\n#### campaign_types\n```sql\nname, respect_exclusion_windows, enable_followups, days_before_event,\ntarget_all_contacts, priority, active\n```\n\n#### campaign_instances\n```sql\nid, campaign_type, instance_name, email_template, sms_template,\nactive_start_date, active_end_date, metadata\n```\n\n#### contact_campaigns\n```sql\nid, contact_id, campaign_instance_id, trigger_date, status, metadata\n```\n\n## Load Balancing and Smoothing\n\nThe system implements sophisticated load balancing to prevent email clustering:\n\n### Daily Volume Caps\n- **Organizational Cap**: Maximum 7% of total contacts per day\n- **Effective Date Soft Limit**: 15 emails per day (configurable)\n- **Over-Limit Detection**: Days exceeding 120% of cap trigger redistribution\n\n### Effective Date Smoothing\nSpreads clustered effective date emails across a ±2 day window using deterministic hashing.\n\n### Global Daily Cap Enforcement\nMigrates excess emails to following days when daily caps are exceeded.\n\n## Domain-Specific Language (DSL)\n\nThe system uses DSL components to make business rules declarative:\n\n### Timing Rules\n```python\nbirthday_timing = TimingRule(days_before_event=14)\nsend_date = birthday_timing.calculate_send_date(anniversary_date)\n```\n\n### Exclusion Windows\n```python\nca_birthday_window = ExclusionWindow(\n    rule_type=ExclusionRuleType.BIRTHDAY_WINDOW,\n    window_before_days=30,\n    window_after_days=60,\n    pre_window_extension_days=60\n)\n```\n\n### State Rules\n```python\nca_rule = StateRule(\n    state_code=\"CA\",\n    exclusion_windows=[ca_birthday_window]\n)\n```\n\n## Performance Optimization\n\n### For Large Datasets (Up to 3M contacts)\n\n1. **Batch Processing**: Processes contacts in configurable batches (default: 10,000)\n2. **Streaming**: Uses database cursors to avoid memory exhaustion\n3. **Optimized Indexes**: Database indexes for common query patterns\n4. **Connection Pooling**: Efficient database connection management\n\n### Recommended Indexes\n```sql\nCREATE INDEX idx_contacts_state_birthday ON contacts(state, birthday);\nCREATE INDEX idx_contacts_state_effective ON contacts(state, effective_date);\nCREATE INDEX idx_email_schedules_status_date ON email_schedules(status, scheduled_send_date);\n```\n\n## Monitoring and Observability\n\n### Key Metrics Tracked\n- Contacts processed per run\n- Emails scheduled vs skipped\n- Daily volume distribution\n- Exclusion window hit rate\n- Campaign effectiveness metrics\n- Processing time per batch\n\n### Audit Trail\n- Scheduler checkpoints for recovery\n- Configuration version tracking\n- Campaign change logs\n- Detailed error logging\n\n## API Integration\n\n### External System Integration Points\n\n1. **Contact Management**: Import/update contact data\n2. **Campaign Triggers**: External systems can trigger campaigns\n3. **Template Management**: Integration with email/SMS sending systems\n4. **Webhook Handling**: Process delivery notifications\n5. **Analytics**: Export scheduling and delivery metrics\n\n## Example Workflows\n\n### 1. Setting Up a Rate Increase Campaign\n\n```python\n# 1. Create campaign type (one-time setup)\ncampaign_manager = CampaignManager(\"database.sqlite3\")\nrate_increase_type = CampaignType(\n    name=\"rate_increase\",\n    respect_exclusion_windows=True,\n    enable_followups=True,\n    timing_rule=TimingRule(days_before_event=14),\n    priority=1\n)\ncampaign_manager.create_campaign_type(rate_increase_type)\n\n# 2. Create campaign instance\nq2_instance = CampaignInstance(\n    id=None,\n    campaign_type=\"rate_increase\",\n    instance_name=\"rate_increase_q2_2025\",\n    email_template=\"rate_increase_v3\",\n    active_start_date=date(2025, 4, 1),\n    active_end_date=date(2025, 6, 30)\n)\ninstance_id = campaign_manager.create_campaign_instance(q2_instance)\n\n# 3. Add contacts to campaign\ncontact_campaigns = [\n    ContactCampaign(\n        contact_id=123,\n        campaign_instance_id=instance_id,\n        trigger_date=date(2025, 7, 1),  # Rate increase date\n        status='pending'\n    )\n]\ncampaign_manager.add_contacts_to_campaign(instance_id, contact_campaigns)\n\n# 4. Run scheduler\nscheduler = EmailScheduler(\"database.sqlite3\")\nscheduler.run_full_schedule()\n```\n\n### 2. Running Daily Scheduling\n\n```bash\n#!/bin/bash\n# Daily cron job script\n\n# Run scheduler with error handling\npython3 /path/to/scheduler.py \\\n    --db /path/to/production.sqlite3 \\\n    --config /path/to/production_config.yaml \\\n    --run-full\n\n# Check exit code\nif [ $? -eq 0 ]; then\n    echo \"Scheduler completed successfully\"\nelse\n    echo \"Scheduler failed - check logs\"\n    exit 1\nfi\n```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Missing ZIP codes/states**: Contacts with invalid data are skipped with warnings\n2. **No campaign emails**: Check that campaign instances have valid active dates\n3. **All emails skipped**: Verify state exclusion rules aren't too restrictive\n4. **Performance issues**: Reduce batch size or enable streaming mode\n\n### Debug Mode\n\n```bash\npython3 scheduler.py --db database.sqlite3 --run-full --debug\n```\n\n### Checking Scheduler Results\n\n```sql\n-- View scheduled emails\nSELECT email_type, status, COUNT(*) \nFROM email_schedules \nWHERE scheduler_run_id = 'your-run-id'\nGROUP BY email_type, status;\n\n-- Check campaign emails\nSELECT * FROM email_schedules \nWHERE email_type LIKE 'campaign_%' \nLIMIT 10;\n\n-- View exclusion reasons\nSELECT skip_reason, COUNT(*) \nFROM email_schedules \nWHERE status = 'skipped'\nGROUP BY skip_reason;\n```\n\n## Contributing\n\n### Adding New State Rules\n\n1. Update `StateRulesEngine._load_default_rules()`\n2. Add configuration to `scheduler_config.yaml`\n3. Test with contacts from the new state\n\n### Adding New Campaign Types\n\n1. Define campaign type in configuration\n2. Create campaign instances as needed\n3. Add contacts to campaigns via API or direct database insertion\n\n### Extending the DSL\n\n1. Create new dataclasses for business concepts\n2. Implement validation and calculation methods\n3. Update scheduler to use new DSL components\n\n## License\n\nThis system implements complex business logic for email scheduling compliance. Please ensure you understand and comply with all applicable regulations in your jurisdiction.\n\n## Support\n\nFor questions about the business logic implementation, refer to the original `business_logic.md` document which contains the complete specification this system implements.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyrex41%2Fp_schedule","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpyrex41%2Fp_schedule","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyrex41%2Fp_schedule/lists"}