{"id":31688154,"url":"https://github.com/mirseo/mirseodb","last_synced_at":"2026-04-19T02:01:29.804Z","repository":{"id":316845886,"uuid":"1065076797","full_name":"mirseo/MirseoDB","owner":"mirseo","description":"MirseoDB - Painless database for everyone","archived":false,"fork":false,"pushed_at":"2025-09-29T11:03:56.000Z","size":220,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-14T08:39:10.683Z","etag":null,"topics":["api","async","data-management","database","db","db-engine","developer-tools","embedded-db","lightweight-db","new-database","open-source","query-language","rust","rust-database","storage","svelte","web-console"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mirseo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-27T02:53:42.000Z","updated_at":"2025-10-03T07:54:34.000Z","dependencies_parsed_at":"2025-09-27T05:36:57.890Z","dependency_job_id":"a2d8e69d-090d-476b-a75c-8c6935f8c634","html_url":"https://github.com/mirseo/MirseoDB","commit_stats":null,"previous_names":["mirseo/mirseodb"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mirseo/MirseoDB","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirseo%2FMirseoDB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirseo%2FMirseoDB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirseo%2FMirseoDB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirseo%2FMirseoDB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mirseo","download_url":"https://codeload.github.com/mirseo/MirseoDB/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirseo%2FMirseoDB/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31991720,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","async","data-management","database","db","db-engine","developer-tools","embedded-db","lightweight-db","new-database","open-source","query-language","rust","rust-database","storage","svelte","web-console"],"created_at":"2025-10-08T10:36:00.512Z","updated_at":"2026-04-19T02:01:29.753Z","avatar_url":"https://github.com/mirseo.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MirseoDB\n\nMirseoDB is a high-performance, lightweight database system written in Rust with built-in AnySQL HYPERTHINKING engine that automatically detects and supports multiple SQL dialects.\n\n**Warning**: This project is in early development, expect breaking changes.\n\n## Features\n\n- **AnySQL HYPERTHINKING Engine**: Automatically detects SQL dialects (Standard SQL, MS-SQL, MySQL/MariaDB, Oracle) without requiring users to specify the dialect\n- **Bloom Filter Optimization**: Column-based row skipping for improved query performance\n- **Composite Indexing**: Multi-column index support with query optimization\n- **Chunked Table Scanning**: Memory-efficient processing with early termination support\n- **Advanced Security**: SQL injection protection, two-factor authentication, and API token validation\n- **Web Console**: Modern SvelteKit-based administration interface\n- **Route Forwarding**: Built-in request routing and forwarding capabilities\n\n## Quick Start\n\n### Prerequisites\n\n- Rust toolchain (edition 2021)\n- Node.js 18+ with npm\n\n### Installation\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/your-username/mirseodb.git\n   cd mirseodb\n   ```\n\n2. Install console dependencies:\n   ```bash\n   cd console\n   npm install\n   cd ..\n   ```\n\n3. Run the server:\n   ```bash\n   cargo run\n   ```\n\nThe server will start on `http://127.0.0.1:3306/` with the web console proxied. The underlying SvelteKit dev server runs on `http://localhost:5173`.\n\n### Environment Variables\n\n- `MIRSEODB_SKIP_CONSOLE=1`: Disable web console startup\n- `MIRSEODB_API_TOKEN`: Set API authentication token\n- `SQL_INJECTION_PROTECT=1`: Enable SQL injection protection\n\n## API Usage\n\n### Authentication\n\nAll API requests require authentication via the `Authorization` header:\n\n```bash\ncurl -X POST http://127.0.0.1:3306/query \\\n  -H \"Authorization: Bearer YOUR_API_TOKEN\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"sql\": \"SELECT * FROM users\"}'\n```\n\n### Query Endpoints\n\n- `POST /query`: Execute SQL queries (JSON format)\n- `GET /query?sql=SELECT * FROM users`: Execute SQL queries (URL parameter)\n- `POST /api/query`: Alternative query endpoint\n- `GET /health`: Health check endpoint\n- `GET /api/health`: Alternative health check endpoint\n\n### Request Format\n\n```json\n{\n  \"sql\": \"CREATE TABLE users (id INTEGER, name TEXT)\",\n  \"auth_token\": \"optional_token_override\",\n  \"totp_token\": \"optional_2fa_token\",\n  \"email\": \"optional_user_email\"\n}\n```\n\n### Response Format\n\n```json\n{\n  \"success\": true,\n  \"data\": [...],\n  \"execution_time_ms\": 25,\n  \"rows_affected\": 1\n}\n```\n\n## Security Features\n\n### Authentication Failure Behavior\n\nWhen authentication fails on `/query` endpoints, the server returns random HTTP error codes (404, 403, 502, 500) to obscure the API's existence from unauthorized users.\n\n### SQL Injection Protection\n\nEnable with `SQL_INJECTION_PROTECT=1`. The system automatically sanitizes suspicious SQL patterns.\n\n### Two-Factor Authentication\n\nConfigure 2FA for sensitive operations like DROP TABLE, DROP DATABASE, and bulk DELETE/UPDATE operations.\n\n## Architecture\n\n### Core Components\n\n- **Engine** (`src/engine.rs`): Main database engine with CRUD operations\n- **AnySQL Parser** (`src/smart_parser.rs`): Intelligent SQL dialect detection and parsing\n- **Bloom Filters** (`src/bloom_filter.rs`): Column-based filtering and chunked scanning\n- **Indexing** (`src/indexing.rs`): Composite indexing and query optimization\n- **Security** (`src/auth.rs`, `src/two_factor_auth.rs`): Authentication and security features\n- **Server** (`src/server.rs`): HTTP server and API endpoints\n- **Persistence** (`src/persistence.rs`): File-based storage engine\n\n### Data Flow\n\n1. SQL Input → AnySQL Parser → SqlStatement enum\n2. SqlStatement → Database.execute() → Storage operations\n3. Results → Formatted output\n\n### Storage\n\n- Database files: `.mirseoDB/*.mdb` (binary serialized format)\n- In-memory operations with periodic disk persistence\n- Automatic backup and recovery\n\n## Development\n\n### Building\n\n```bash\ncargo build --release\n```\n\n### Testing\n\n```bash\ncargo test\n```\n\n### Development Mode\n\n```bash\ncargo run\n```\n\n### Disable Console\n\n```bash\nMIRSEODB_SKIP_CONSOLE=1 cargo run\n```\n\n## Performance Optimizations\n\n### Recent Optimizations\n\n1. **Parser Optimizations**:\n   - Dialect caching with LRU eviction\n   - Keyword hash matching with weighted detection\n   - Performance metrics tracking\n\n2. **Query Optimizations**:\n   - Composite index support for multi-column queries\n   - WHERE clause analysis for optimal index selection\n   - Index hint system (USE, FORCE, IGNORE)\n\n3. **Table Scan Optimizations**:\n   - Bloom filters for column-based row skipping\n   - Chunk-based processing for memory efficiency\n   - Early termination with LIMIT support\n\n### Benchmark Results\n\nThe optimizations provide significant performance improvements:\n- Parser: 90% reduction in repeated SQL parsing overhead\n- Indexing: Up to 10x faster multi-column queries\n- Scanning: 50-80% reduction in unnecessary row processing\n\n## Documentation\n\nComprehensive documentation is available in the `docs/` directory:\n\n- `docs/en/`: English documentation\n- `docs/ko/`: Korean documentation\n\nEach module has detailed documentation covering:\n- API reference\n- Implementation details\n- Usage examples\n- Performance considerations\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests\n5. Submit a pull request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Support\n\nFor questions, issues, or contributions, please visit the GitHub repository or create an issue.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirseo%2Fmirseodb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmirseo%2Fmirseodb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirseo%2Fmirseodb/lists"}