{"id":43762154,"url":"https://github.com/kcenon/screener_system","last_synced_at":"2026-02-05T15:08:06.596Z","repository":{"id":323306487,"uuid":"1092647579","full_name":"kcenon/screener_system","owner":"kcenon","description":"Advanced stock screening platform for Korean markets (KOSPI/KOSDAQ) with 200+ financial indicators","archived":false,"fork":false,"pushed_at":"2025-12-02T06:35:26.000Z","size":5077,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-03T08:32:48.430Z","etag":null,"topics":["apache-airflow","fastapi","financial-analysis","korean-stocks","kosdaq","kospi","postgresql","react","stock-screening","timescaledb"],"latest_commit_sha":null,"homepage":"","language":"PLpgSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kcenon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing/TSDOC_TEMPLATES.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/SECURITY_AUDIT.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-09T02:50:13.000Z","updated_at":"2025-12-02T06:35:28.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kcenon/screener_system","commit_stats":null,"previous_names":["kcenon/screener_system"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kcenon/screener_system","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcenon%2Fscreener_system","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcenon%2Fscreener_system/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcenon%2Fscreener_system/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcenon%2Fscreener_system/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kcenon","download_url":"https://codeload.github.com/kcenon/screener_system/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcenon%2Fscreener_system/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29124793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T14:05:12.718Z","status":"ssl_error","status_checked_at":"2026-02-05T14:03:53.078Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-airflow","fastapi","financial-analysis","korean-stocks","kosdaq","kospi","postgresql","react","stock-screening","timescaledb"],"created_at":"2026-02-05T15:08:06.487Z","updated_at":"2026-02-05T15:08:06.573Z","avatar_url":"https://github.com/kcenon.png","language":"PLpgSQL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Stock Screening Platform\n\n[![CI Pipeline](https://github.com/kcenon/screener_system/actions/workflows/ci.yml/badge.svg)](https://github.com/kcenon/screener_system/actions/workflows/ci.yml)\n[![CD Pipeline](https://github.com/kcenon/screener_system/actions/workflows/cd.yml/badge.svg)](https://github.com/kcenon/screener_system/actions/workflows/cd.yml)\n[![Documentation](https://github.com/kcenon/screener_system/actions/workflows/docs.yml/badge.svg)](https://github.com/kcenon/screener_system/actions/workflows/docs.yml)\n[![PR Checks](https://github.com/kcenon/screener_system/actions/workflows/pr-checks.yml/badge.svg)](https://github.com/kcenon/screener_system/actions/workflows/pr-checks.yml)\n[![codecov](https://codecov.io/gh/kcenon/screener_system/branch/main/graph/badge.svg)](https://codecov.io/gh/kcenon/screener_system)\n[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n[![Docs Status](https://img.shields.io/badge/docs-live-success)](https://kcenon.github.io/screener_system)\n[![Project Status](https://img.shields.io/badge/status-production--ready-success)]()\n[![Completion](https://img.shields.io/badge/tickets-54%2F54%20(100%25)-brightgreen)]()\n\nA comprehensive stock analysis and screening platform for Korean markets (KOSPI/KOSDAQ) with 200+ financial and technical indicators.\n\n\u003e **🎉 Project Status**: MVP Complete! All 54 planned tickets completed (~358 hours). Production-ready with full documentation, testing, and CI/CD automation.\n\n## 🎯 Features\n\n### ✅ Implemented (Production-Ready)\n\n- **Advanced Stock Screening**: Filter 2,400+ stocks using 200+ technical and financial indicators\n  - Complex multi-criteria filtering with AND/OR logic\n  - Filter presets (save/load/delete custom filters)\n  - Export results to CSV/JSON\n  - URL sharing for filter configurations\n- **Real-time Market Data**: WebSocket-based live updates\n  - Real-time price streaming with JWT authentication\n  - Order book visualization (10-level bid/ask)\n  - Multi-instance support via Redis Pub/Sub\n  - Auto-reconnection with exponential backoff\n- **User Authentication \u0026 Authorization**: JWT-based security\n  - User registration and login\n  - Role-based access control (RBAC)\n  - Refresh token support\n  - Rate limiting by user tier\n- **Stock Detail Pages**: Comprehensive stock information\n  - Real-time price data and charts\n  - Technical indicators visualization\n  - Historical data analysis\n  - Financial ratios and fundamentals\n- **Data Pipeline**: Automated data ingestion and processing\n  - Apache Airflow orchestration\n  - KIS API integration for real-time data\n  - Daily price ingestion (2,400+ stocks)\n  - 200+ indicator calculations\n  - Error handling and retry logic\n- **Monitoring \u0026 Observability**: Production-grade monitoring\n  - Prometheus metrics collection\n  - Grafana dashboards\n  - Performance baselines established\n  - Alert configuration\n- **Documentation**: Comprehensive auto-generated docs\n  - Docusaurus documentation platform\n  - Python API docs (Sphinx)\n  - TypeScript API docs (TypeDoc)\n  - OpenAPI/Swagger UI\n  - User guides and tutorials\n\n### 🚧 Planned (Future Enhancements)\n\n- **Portfolio Management**: Track holdings and performance vs benchmarks\n- **Price Alerts**: Customizable notifications for price movements\n- **Mobile App**: React Native mobile application\n- **Advanced Analytics**: Backtesting and strategy builder\n- **Social Features**: Community discussions and shared analyses\n\n## 📖 Documentation\n\nComprehensive documentation is automatically built and deployed at **[kcenon.github.io/screener_system](https://kcenon.github.io/screener_system)**\n\nThe documentation site includes:\n- **Getting Started Guide** - Setup and installation\n- **API Reference** - Backend (Python) and Frontend (TypeScript) APIs\n- **User Guides** - Feature documentation\n- **Architecture** - System design and components\n- **Contributing** - Development guidelines\n\n### Building Documentation Locally\n\n```bash\n# Build all documentation\ncd docs-site\nnpm install\nnpm start  # Opens http://localhost:3000\n\n# Build Python API docs (Sphinx)\ncd docs/api/python\nsphinx-build -b html . _build/html\n\n# Build Frontend API docs (TypeDoc)\ncd frontend\nnpm run docs:generate\n```\n\n### Documentation Pipeline\n\nDocumentation is automatically built and deployed on every push to `main`:\n- **Sphinx** generates Python API documentation\n- **TypeDoc** generates TypeScript API documentation\n- **Docusaurus** builds the main documentation site\n- **GitHub Pages** hosts at https://kcenon.github.io/screener_system\n\nSee [CI/CD Setup Guide](docs/CI_CD_SETUP.md) for details.\n\n## 📊 Tech Stack\n\n### Frontend\n- **Framework**: React 18 + TypeScript\n- **Build Tool**: Vite\n- **State Management**: Zustand\n- **Data Fetching**: TanStack Query (React Query)\n- **UI Components**: Radix UI + Tailwind CSS\n- **Charts**: TradingView Lightweight Charts + Recharts\n\n### Backend\n- **API Framework**: FastAPI (Python)\n- **Database**: PostgreSQL 16 + TimescaleDB\n- **Caching**: Redis 7\n- **Task Queue**: Celery\n- **Authentication**: JWT (FastAPI-Users)\n\n### Infrastructure\n- **Containerization**: Docker + Docker Compose\n- **Orchestration**: Kubernetes\n- **CI/CD**: GitHub Actions\n- **Monitoring**: Grafana + Prometheus\n- **Logging**: ELK Stack\n\n### Data Pipeline\n- **Workflow Orchestration**: Apache Airflow\n- **Data Processing**: Pandas + NumPy\n- **Data Sources**: KRX API, F\u0026Guide API\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- Docker \u0026 Docker Compose\n- Node.js 18+\n- Python 3.9+\n- PostgreSQL 16+ (with TimescaleDB extension)\n- Redis 7+\n\n### Installation\n\n**Option 1: Docker Compose (Recommended)**\n\n```bash\n# Clone repository\ngit clone https://github.com/kcenon/screener_system.git\ncd screener_system\n\n# Setup environment\ncp .env.example .env\n# Edit .env with your configuration\n\n# Start all services with Docker Compose\ndocker-compose up -d\n\n# Check service health\ndocker-compose ps\n\n# View logs\ndocker-compose logs -f backend\n```\n\n**Option 2: Local Development**\n\n```bash\n# Clone repository\ngit clone https://github.com/kcenon/screener_system.git\ncd screener_system\n\n# Setup environment\ncp .env.example .env\n\n# Start database and cache only\ndocker-compose up -d postgres redis\n\n# Backend setup\ncd backend\npip install -r requirements.txt\nuvicorn app.main:app --reload\n\n# Frontend setup (in another terminal)\ncd frontend\nnpm install\nnpm run dev\n\n# Data pipeline setup (optional, in another terminal)\ncd data_pipeline\n# Follow data_pipeline/README.md for setup\n```\n\nAccess the application:\n- **Frontend**: http://localhost:5173\n- **Backend API**: http://localhost:8000\n- **API Docs**: http://localhost:8000/docs\n- **Airflow UI**: http://localhost:8080\n\n### Loading Sample Data\n\nAfter starting the services, load sample stock data for testing:\n\n```bash\n# Load pre-generated seed data (150 stocks, 252 days of prices)\ndocker exec -i screener_postgres psql -U screener_user -d screener_db \\\n  \u003c database/seeds/seed_data.sql\n\n# Verify data loaded\ndocker exec screener_postgres psql -U screener_user -d screener_db \\\n  -c \"SELECT COUNT(*) FROM stocks; SELECT COUNT(*) FROM daily_prices;\"\n```\n\n**Sample Data Includes:**\n- 150 stocks (100 KOSPI + 50 KOSDAQ)\n- 27,000 daily prices (~252 trading days per stock)\n- 600 financial statements (4 quarters per stock)\n- 150 calculated indicators\n\n\u003e **Note:** This is test data for development only. For production, use real KRX API data via Airflow DAGs.\n\nSee [Data Loading Guide](docs/DATA_LOADING.md) for detailed instructions and customization options.\n\n## 📁 Project Structure\n\n```\nscreener_system/\n├── frontend/                 # React SPA\n│   ├── src/\n│   │   ├── components/      # Reusable UI components\n│   │   ├── pages/           # Page components\n│   │   ├── hooks/           # Custom React hooks\n│   │   ├── services/        # API client services\n│   │   ├── store/           # Zustand state management\n│   │   └── utils/           # Utility functions\n│   ├── public/              # Static assets\n│   └── package.json\n│\n├── backend/                  # FastAPI application\n│   ├── app/\n│   │   ├── api/             # API endpoints\n│   │   ├── core/            # Core functionality\n│   │   ├── db/              # Database models\n│   │   ├── schemas/         # Pydantic schemas\n│   │   └── services/        # Business logic\n│   ├── tests/               # Unit \u0026 integration tests\n│   └── requirements.txt\n│\n├── database/                 # Database schema \u0026 migrations\n│   ├── migrations/          # SQL migration files\n│   ├── seeds/               # Seed data\n│   ├── scripts/             # Utility scripts\n│   └── README.md\n│\n├── data_pipeline/            # Apache Airflow DAGs\n│   ├── dags/                # DAG definitions\n│   ├── plugins/             # Custom Airflow plugins\n│   ├── config/              # Configuration files\n│   └── README.md\n│\n├── api/                      # API specification\n│   ├── openapi.yaml         # OpenAPI 3.0 spec\n│   └── README.md\n│\n├── infrastructure/           # Infrastructure as Code\n│   ├── docker/              # Dockerfiles\n│   ├── kubernetes/          # K8s manifests\n│   ├── terraform/           # Cloud infrastructure\n│   └── monitoring/          # Grafana dashboards\n│\n├── docs/                     # Documentation\n│   ├── PRD.md               # Product Requirements\n│   ├── architecture.md      # System architecture\n│   ├── api-guide.md         # API usage guide\n│   └── deployment.md        # Deployment guide\n│\n├── tests/                    # End-to-end tests\n│   ├── e2e/                 # Cypress tests\n│   └── load/                # k6 load tests\n│\n├── .github/                  # GitHub workflows\n│   └── workflows/\n│       ├── ci.yml           # Continuous Integration\n│       └── cd.yml           # Continuous Deployment\n│\n├── docker-compose.yml        # Local development stack\n├── .env.example              # Environment variables template\n├── .gitignore\n└── README.md                 # This file\n```\n\n## 🔧 Development\n\n### Running Tests\n\n```bash\n# Frontend tests\ncd frontend\nnpm test\n\n# Backend tests\ncd backend\npytest\n\n# End-to-end tests\ncd tests/e2e\nnpx cypress run\n```\n\n### Code Quality\n\n```bash\n# Frontend linting\ncd frontend\nnpm run lint\n\n# Backend linting\ncd backend\nruff check .\nmypy app/\n\n# Format code\ncd frontend \u0026\u0026 npm run format\ncd backend \u0026\u0026 ruff format .\n```\n\n### Database Migrations\n\n```bash\n# Create new migration\nalembic revision --autogenerate -m \"description\"\n\n# Apply migrations\nalembic upgrade head\n\n# Rollback\nalembic downgrade -1\n```\n\n## 📈 Performance Benchmarks\n\nPerformance targets measured and validated during Sprint 4:\n\n| Metric | Target | Actual | Status |\n|--------|--------|--------|--------|\n| Screening Query Time (p99) | \u003c 500ms | ~220ms | ✅ 56% better |\n| API Response Time (p95) | \u003c 200ms | ~150ms | ✅ 25% better |\n| Page Load Time (p95) | \u003c 1.5s | ~1.2s | ✅ 20% better |\n| DAG Execution (Full) | \u003c 10 min | ~8.5 min | ✅ 15% better |\n| Cache Hit Rate | \u003e 80% | ~85% | ✅ |\n| Test Coverage (Backend) | \u003e 70% | 80% | ✅ |\n\n**Key Optimizations Applied:**\n- Window function optimization (45% query speedup)\n- Redis caching with 5-minute TTL\n- Database connection pooling\n- Materialized views for complex queries\n- Atomic Redis operations for rate limiting\n\n## 🚢 Deployment\n\n### Docker Production Build\n\n```bash\n# Build images\ndocker-compose -f docker-compose.prod.yml build\n\n# Deploy\ndocker-compose -f docker-compose.prod.yml up -d\n```\n\n### Kubernetes Deployment\n\n```bash\n# Apply manifests\nkubectl apply -f infrastructure/kubernetes/\n\n# Check status\nkubectl get pods -n screener\n\n# View logs\nkubectl logs -f deployment/backend -n screener\n```\n\n## 📊 Monitoring\n\n- **Application Metrics**: Grafana dashboard at http://localhost:3000 (development)\n- **API Health**: http://localhost:8000/health (development)\n- **Database Performance**: TimescaleDB monitoring\n- **Error Tracking**: Sentry integration\n\n\u003e **Note**: Production URLs will be configured during deployment phase.\n\n## 🔐 Security\n\n- JWT-based authentication (15-minute access tokens)\n- HTTPS only (TLS 1.3)\n- Rate limiting (100-2000 req/min based on tier)\n- SQL injection prevention (parameterized queries)\n- XSS protection (Content Security Policy)\n- CSRF protection (SameSite cookies)\n- Regular dependency scans (Dependabot)\n\n## 📝 API Documentation\n\nComprehensive API documentation is available in multiple formats:\n\n**Documentation Portal** (https://kcenon.github.io/screener_system):\n- **API Reference**: https://kcenon.github.io/screener_system/docs/api/intro\n  - Python Backend API (auto-generated with Sphinx)\n  - TypeScript Frontend API (auto-generated with TypeDoc)\n  - Architecture and design patterns\n  - Code examples and tutorials\n\n**Interactive API Docs** (Backend Server):\n- **Swagger UI**: http://localhost:8000/docs (development)\n- **ReDoc**: http://localhost:8000/redoc (development)\n- **OpenAPI Spec**: http://localhost:8000/openapi.json\n\n\u003e **Note**: Interactive API docs (Swagger/ReDoc) are served by the FastAPI backend. Production URLs will be configured during deployment.\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit changes (`git commit -m 'Add amazing feature'`)\n4. Push to branch (`git push origin feature/amazing-feature`)\n5. Open Pull Request\n\n## 📄 License\n\nThis project is licensed under the BSD 3-Clause License - see the [LICENSE](LICENSE) file for details.\n\n## 📞 Support\n\n- **Email**: kcenon@gmail.com\n- **Issue Tracker**: https://github.com/kcenon/screener_system/issues\n- **Documentation**: https://kcenon.github.io/screener_system\n\n## 📊 Project Statistics\n\n| Category | Metrics |\n|----------|---------|\n| **Development** | 54 tickets completed (100%) |\n| **Total Effort** | ~358 hours across 4 sprints |\n| **Test Coverage** | Backend: 80%, Frontend: 100% pass rate |\n| **Code Quality** | All linting/type checks passing |\n| **Documentation** | 100% API coverage, auto-generated |\n| **CI/CD** | Fully automated with GitHub Actions |\n| **Infrastructure** | Docker, Airflow, Monitoring ready |\n| **Security** | JWT auth, rate limiting, SQL injection prevention |\n\n## 🗺️ Development Timeline\n\n### ✅ Sprint 1-2: Foundation \u0026 Core Features (Complete)\n**Tickets**: 23 tickets, ~137 hours\n\n- ✅ **Infrastructure** (3 tickets)\n  - Docker Compose development environment\n  - CI/CD pipeline with GitHub Actions\n  - Production monitoring and logging\n- ✅ **Backend** (6 tickets)\n  - FastAPI project setup\n  - User authentication API\n  - Stock data API\n  - Stock screening API (96 tests)\n  - API rate limiting and throttling\n  - WebSocket real-time streaming\n- ✅ **Database** (5 tickets)\n  - PostgreSQL + TimescaleDB setup\n  - Schema migrations\n  - Indexes and materialized views\n  - Functions and triggers\n  - Order book schema\n- ✅ **Data Pipeline** (4 tickets)\n  - Apache Airflow environment\n  - Daily price ingestion DAG\n  - Indicator calculation DAG (200+ indicators)\n  - KIS API integration\n- ✅ **Frontend** (5 tickets)\n  - React + Vite project setup\n  - User authentication UI\n  - Stock screener page (139 tests)\n  - Stock detail page\n  - Order book visualization\n\n### ✅ Sprint 3: Polish \u0026 Advanced Features (Complete)\n**Tickets**: 12 tickets, ~114 hours\n\n- ✅ Technical debt resolution (5 tickets)\n- ✅ Security hardening (SQL injection fixes)\n- ✅ Performance optimization (45% query improvement)\n- ✅ Rate limiting documentation\n- ✅ Test coverage increase (59% → 80%)\n\n### ✅ Sprint 4: Quality Assurance \u0026 Documentation (Complete)\n**Tickets**: 19 tickets, ~107 hours\n\n- ✅ **Bug Fixes \u0026 Validation** (7 tickets)\n  - Docker environment runtime testing\n  - Airflow DAG runtime testing\n  - Performance baseline measurement\n  - CI/CD validation\n  - Acceptance criteria validation\n- ✅ **Documentation Sprint** (9 tickets)\n  - Docusaurus platform setup\n  - Python API documentation (Sphinx)\n  - TypeScript API documentation (TypeDoc)\n  - Documentation guidelines and templates\n  - CI/CD deployment pipeline\n- ✅ **Security \u0026 Data Quality** (3 tickets)\n  - Dependency vulnerability resolution (29 CVEs)\n  - Data quality validation\n  - Sample data generation\n\n### 🚀 Future Roadmap (Post-MVP)\n\n**Phase 1: Beta Launch** (Next 2 months)\n- [ ] User acceptance testing\n- [ ] Performance tuning under load\n- [ ] Security audit and penetration testing\n- [ ] Production deployment and monitoring\n\n**Phase 2: Feature Expansion** (Months 3-6)\n- [ ] Portfolio management and tracking\n- [ ] Price alerts and notifications\n- [ ] Advanced charting features\n- [ ] User watchlists and favorites\n- [ ] News integration\n\n**Phase 3: Advanced Features** (Months 7-12)\n- [ ] Mobile app (React Native)\n- [ ] Backtesting engine\n- [ ] Strategy builder\n- [ ] API access for developers\n- [ ] Premium subscription tiers\n\n**Phase 4: Scale \u0026 Innovation** (Year 2+)\n- [ ] AI-powered stock recommendations\n- [ ] Social features and community\n- [ ] International markets support\n- [ ] Institutional features\n- [ ] White-label solutions\n\n## 📚 Documentation\n\n- [Product Requirements Document (PRD)](docs/PRD.md)\n- [API Guide](api/README.md)\n- [Database Schema](database/README.md)\n- [Data Pipeline](data_pipeline/README.md)\n- [Architecture Overview](docs/architecture.md)\n- [Deployment Guide](docs/deployment.md)\n\n---\n\n**Made with ❤️ by 🍀☀🌕🌥 🌊**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkcenon%2Fscreener_system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkcenon%2Fscreener_system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkcenon%2Fscreener_system/lists"}