{"id":50472480,"url":"https://github.com/semcod/deta","last_synced_at":"2026-06-01T11:04:03.542Z","repository":{"id":354847827,"uuid":"1225561796","full_name":"semcod/deta","owner":"semcod","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-13T15:09:20.000Z","size":8031,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-24T16:09:35.463Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semcod.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-30T12:06:07.000Z","updated_at":"2026-05-13T15:16:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/semcod/deta","commit_stats":null,"previous_names":["semcod/deta"],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/semcod/deta","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fdeta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fdeta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fdeta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fdeta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semcod","download_url":"https://codeload.github.com/semcod/deta/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fdeta/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33771633,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-01T11:04:03.470Z","updated_at":"2026-06-01T11:04:03.531Z","avatar_url":"https://github.com/semcod.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# deta\n\n\n## AI Cost Tracking\n\n![PyPI](https://img.shields.io/badge/pypi-costs-blue) ![Version](https://img.shields.io/badge/version-0.2.52-blue) ![Python](https://img.shields.io/badge/python-3.9+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)\n![AI Cost](https://img.shields.io/badge/AI%20Cost-$1.98-orange) ![Human Time](https://img.shields.io/badge/Human%20Time-14.6h-blue) ![Model](https://img.shields.io/badge/Model-openrouter%2Fqwen%2Fqwen3--coder--next-lightgrey)\n\n- 🤖 **LLM usage:** $1.9758 (53 commits)\n- 👤 **Human dev:** ~$1457 (14.6h @ $100/h, 30min dedup)\n\nGenerated on 2026-05-26 using [openrouter/qwen/qwen3-coder-next](https://openrouter.ai/qwen/qwen3-coder-next)\n\n---\n\nInfrastructure anomaly detection and monitoring tool for development environments.\n\n![PyPI](https://img.shields.io/badge/pypi-deta-blue) ![Version](https://img.shields.io/badge/version-0.2.52-blue) ![Python](https://img.shields.io/badge/python-3.8+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)\n\n## Features\n\n- **Manifest Scanning**: Scans docker-compose, OpenAPI, package.json, and pyproject.toml files up to 3 layers deep\n- **Environment Interpolation**: Full `${VAR:-default}` support with `.env` file layering and compose override merging\n- **Port Parsing**: Resolves host/container port mappings with protocol and host-bind support\n- **Topology Building**: Builds service dependency graphs with cycle detection and hub analysis\n- **Anomaly Detection**: Detects missing healthchecks, port conflicts, dependency cycles, and hardcoded secrets\n- **Real-time Monitoring**: Watches for config changes, probes HTTP/TCP health checks with concurrent probing\n- **Web Dashboard**: Real-time FastAPI/WebSocket dashboard with topology visualization and event stream\n- **DSL Change Tracking**: Structured change commands for service/port/probe state transitions\n- **Topology Caching**: mtime-based cache with configurable TTL to avoid redundant rescans\n- **Multi-format Output**: JSON, YAML graph, Mermaid, PNG (graphviz), and Semcod toon format\n- **CLI Interface**: `scan`, `monitor`, `diff`, and `web` commands via argparse\n\n## Installation\n\n```bash\npip install deta\n```\n\nWith optional dependencies:\n\n```bash\n# web dashboard (FastAPI + uvicorn + WebSocket)\npip install 'deta[web]'\n\n# all extras\npip install deta[docker,toml,web]\n```\n\n## Usage\n\n### Scan infrastructure\n\n```bash\ndeta scan /path/to/project --depth 3 --output infra-map.json\n```\n\n### Generate graph outputs (YAML / Mermaid / PNG)\n\n```bash\n# text graph + mermaid + json\ndeta scan /path/to/project --formats json,yaml,mermaid\n\n# include online checks (localhost probes)\ndeta scan /path/to/project --formats json,yaml,mermaid --online\n\n# try PNG (requires graphviz python package + graphviz binary)\ndeta scan /path/to/project --formats png\n```\n\nGenerated files (configurable in `deta.yaml`):\n- `infra-map.json`\n- `infra-graph.yaml`\n- `infra-graph.mmd`\n- `infra-graph.png`\n- `infra.toon.yaml`\n\n### Monitor in real-time\n\n```bash\ndeta monitor /path/to/project --interval 30 --depth 3\ndeta monitor . --interval 30 --depth 3\n\n# realtime watch from scan command (regenerates outputs on each change)\ndeta scan /path/to/project --watch --formats json,yaml,mermaid --online\n```\n\n### Web Dashboard\n\n```bash\ndeta web /path/to/project --depth 3 --host 127.0.0.1 --port 8765\n```\n\nOpen `http://127.0.0.1:8765` — real-time topology view with WebSocket event push (service up/down, port changes).\n\nConfiguration in `deta.yaml`:\n\n```yaml\nweb:\n  enabled: true\n  host: 127.0.0.1\n  port: 8765\n  refresh_seconds: 5\n  cache_ttl_seconds: 30.0\n  debounce_seconds: 0.5\n  push_events:\n    - service_added\n    - service_removed\n    - service_up\n    - service_down\n```\n\n### Compare with baseline\n\n```bash\ndeta diff --baseline infra-map.json /path/to/project\n```\n\n### Example Workflow (e.g., for c2004 project)\n\nHere is a complete example of how to use `deta` to scan, monitor, and visualize the `c2004` project:\n\n1. **Scan and export topology:**\n   ```bash\n   deta scan ../maskservice/c2004 --output infra-map.json\n   ```\n2. **Detect changes against the baseline:**\n   ```bash\n   deta diff --baseline infra-map.json ../maskservice/c2004\n   ```\n3. **Monitor infrastructure in real-time:**\n   ```bash\n   deta monitor ../maskservice/c2004 --interval 5\n   ```\n4. **Launch the interactive Web Dashboard:**\n   ```bash\n   # Ensure the web extra is installed: pip install '.[web]'\n   deta web ../maskservice/c2004 --host 127.0.0.1 --port 8765\n   ```\n\n### Python API\n\n```python\nfrom pathlib import Path\nfrom deta import build_topology, scan_compose, scan_openapi, probe_all\n\n# Build topology from manifests\ntopology = build_topology(Path(\"/path/to/project\"), max_depth=3)\n\n# Detect anomalies\nanomalies = topology.detect_anomalies()\nfor anomaly in anomalies:\n    print(f\"{anomaly['severity']}: {anomaly['type']}\")\n\n# Cycle \u0026 hub analysis\ncycles = topology.detect_cycles()\nhubs = topology.find_hubs(min_degree=3)\n\n# Export to JSON\nimport json\noutput = json.loads(topology.to_json())\n\n# Probe services\nresults = probe_all(topology.services, max_concurrency=20)\n```\n\n## Architecture\n\n```\ndeta/\n├── scanner/          # Manifest parsing\n│   ├── compose.py    # docker-compose.yml (with override merging)\n│   ├── openapi.py    # OpenAPI specs\n│   ├── npm.py        # package.json\n│   ├── python.py     # pyproject.toml \u0026 requirements.txt\n│   ├── env.py        # .env file loading \u0026 interpolation\n│   └── ports.py      # Port binding parsing \u0026 resolution\n├── builder/          # Topology construction\n│   ├── topology.py   # Graph, anomaly detection, cycles, hubs\n│   └── cache.py      # mtime-based topology cache\n├── monitor/          # Real-time monitoring\n│   ├── watcher.py    # File change polling\n│   ├── prober.py     # HTTP/TCP health checks (concurrent)\n│   └── alerter.py    # Rich console output\n├── formatter/        # Output formats\n│   ├── graph.py      # YAML graph, Mermaid, PNG\n│   └── toon.py       # Semcod toon format\n├── dsl/              # Change tracking DSL\n│   └── commands.py   # service_up/down, port_added/removed\n├── web/              # Real-time dashboard\n│   └── app.py        # FastAPI + WebSocket (topology, probes, events)\n├── integration/      # Ecosystem hooks\n│   └── semcod.py     # sumd, pyqual, vallm, pre_deploy_check\n├── config.py         # deta.yaml configuration (Pydantic models)\n├── core.py           # Base Wup class\n└── cli.py            # CLI entry point (scan, monitor, diff, web)\n```\n\n## Configuration\n\nPlace `deta.yaml` in your project root. See [`deta.yaml.example`](deta.yaml.example) for all options:\n\n- **scan** — depth, include/exclude patterns\n- **watch** — file monitoring paths and patterns\n- **anomaly** — toggle checks, secret patterns, severity levels\n- **monitor** — interval, probe timeout/retries, concurrency\n- **output** — formats and file paths\n- **alert** — console colors, min severity\n- **web** — dashboard host/port, refresh, cache TTL, push events\n\n## Semcod Integration\n\ndeta integrates with the Semcod ecosystem:\n\n```python\nfrom pathlib import Path\nfrom deta.integration import generate_for_sumd, generate_for_pyqual, generate_for_vallm, pre_deploy_check\n\n# Generate toon for sumd\ngenerate_for_sumd(Path(\".\"), output=Path(\"infra.toon.yaml\"))\n\n# Quality analysis for pyqual\ngenerate_for_pyqual(Path(\".\"), depth=3)\n\n# Validation for vallm\ngenerate_for_vallm(Path(\".\"), depth=3)\n\n# Pre-deployment validation\npassed, issues = pre_deploy_check(Path(\".\"))\nif not passed:\n    print(\"Deployment blocked:\", issues)\n```\n\n## Tests\n\n```bash\npytest tests/ -v\n```\n\nTest coverage: cache invalidation, compose env interpolation, DSL commands, `.env` parsing, port parsing, monitor port display.\n\n## License\n\nLicensed under Apache-2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fdeta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemcod%2Fdeta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fdeta/lists"}