{"id":48372030,"url":"https://github.com/notyusheng/tracepcap","last_synced_at":"2026-04-05T17:01:58.903Z","repository":{"id":335706648,"uuid":"1146618872","full_name":"NotYuSheng/TracePcap","owner":"NotYuSheng","description":"Self-hosted LLM network packet analysis tool. Visualize network traffic patterns, analyze packet flows, generate intelligent filters, and gain insights through AI-powered analysis. Perfect for network troubleshooting, security analysis, protocol debugging, or educational purposes.","archived":false,"fork":false,"pushed_at":"2026-04-04T04:01:58.000Z","size":8247,"stargazers_count":5,"open_issues_count":9,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-04T04:26:58.461Z","etag":null,"topics":["ai","blue-team","cyber-defense","cybersecurity","data-visualization","llm","ndpi","network-forensics","network-monitoring","network-visualization","packet-analysis","pcap","pcap-analyzer","pcap4j","protocol-analysis","self-hosted","sgds","soc","tshark","wireshark"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NotYuSheng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-31T11:53:19.000Z","updated_at":"2026-04-04T04:01:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/NotYuSheng/TracePcap","commit_stats":null,"previous_names":["notyusheng/tracecap","notyusheng/tracepcap"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/NotYuSheng/TracePcap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotYuSheng%2FTracePcap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotYuSheng%2FTracePcap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotYuSheng%2FTracePcap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotYuSheng%2FTracePcap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NotYuSheng","download_url":"https://codeload.github.com/NotYuSheng/TracePcap/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NotYuSheng%2FTracePcap/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31442924,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T15:22:31.103Z","status":"ssl_error","status_checked_at":"2026-04-05T15:22:00.205Z","response_time":75,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","blue-team","cyber-defense","cybersecurity","data-visualization","llm","ndpi","network-forensics","network-monitoring","network-visualization","packet-analysis","pcap","pcap-analyzer","pcap4j","protocol-analysis","self-hosted","sgds","soc","tshark","wireshark"],"created_at":"2026-04-05T17:01:58.020Z","updated_at":"2026-04-05T17:01:58.889Z","avatar_url":"https://github.com/NotYuSheng.png","language":"Java","readme":"\u003ch1 align=\"center\"\u003eTracePcap\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eIntelligent PCAP file analysis with AI-powered insights and interactive network visualization\u003c/strong\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#features\"\u003eFeatures\u003c/a\u003e •\n  \u003ca href=\"#quick-start\"\u003eQuick Start\u003c/a\u003e •\n  \u003ca href=\"#usage\"\u003eUsage\u003c/a\u003e •\n  \u003ca href=\"#documentation\"\u003eDocumentation\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Spring_Boot-6DB33F?style=flat\u0026logo=springboot\u0026logoColor=white\" alt=\"Spring Boot\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/React-61DAFB?style=flat\u0026logo=react\u0026logoColor=black\" alt=\"React\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/PostgreSQL-4169E1?style=flat\u0026logo=postgresql\u0026logoColor=white\" alt=\"PostgreSQL\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Docker-2496ED?style=flat\u0026logo=docker\u0026logoColor=white\" alt=\"Docker\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/MinIO-C72E49?style=flat\u0026logo=minio\u0026logoColor=white\" alt=\"MinIO\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Java-21-007396?style=flat\u0026logo=openjdk\u0026logoColor=white\" alt=\"Java 21\"/\u003e\n\u003c/p\u003e\n\n---\n\nA comprehensive network packet analysis tool that runs entirely self-hosted. Upload PCAP files to visualize network traffic patterns, analyze packet flows, generate intelligent filters, and gain insights through AI-powered analysis. Perfect for network troubleshooting, security analysis, protocol debugging, or educational purposes.\n\n\u003cdiv align=\"center\"\u003e\n\n![TracePcap Demo](https://raw.githubusercontent.com/NotYuSheng/TracePcap/main/sample-files/TracePcap-Demo.gif)\n\n\u003c/div\u003e\n\n## Features\n\n| Feature | Description |\n|---------|-------------|\n| **PCAP Upload \u0026 Management** | Upload and manage PCAP/PCAPNG/CAP files (max 512MB) with MinIO object storage; duplicate detection and configurable upload limits |\n| **Network Visualization** | Interactive network topology using React Flow + ELK layout with a rich filter panel (IP, port, device type, protocol, risk), fullscreen toggle, layout controls, and clickable node detail panel |\n| **nDPI Security Detection** | Deep packet inspection via nDPI v5: application identification, traffic categories, risk/alert flags, JA3/JA3S TLS fingerprints, SNI extraction, and TLS certificate metadata per conversation |\n| **Conversation Tracking** | Paginated conversation list with advanced filtering (IP, port, protocol, app, risk, custom rules, device type, country, payload pattern), multi-column sorting, column picker, and bulk PCAP export |\n| **Session Reconstruction** | TCP/UDP application-layer payload decoding with a hex+ASCII viewer for inspecting raw packet payloads |\n| **File Extraction** | HTTP object extraction and raw TCP/UDP stream extraction via Aho-Corasick; automatic MIME type detection; bulk download |\n| **Geolocation \u0026 Device Classification** | Country/ASN enrichment for external IPs; automatic device-type prediction (Router, Server, IoT, Mobile, Laptop/Desktop) across all views |\n| **MAC Manufacturer Lookup** | Wireshark OUI database integration for vendor identification from MAC addresses |\n| **Timeline Analysis** | Chronological traffic visualization with configurable time granularity (auto or manual) and protocol breakdown |\n| **AI Filter Generator** | LLM-powered Wireshark/tcpdump filter generation from natural language queries with confidence scores and packet-level results |\n| **Story Mode** | AI-generated narrative reconstruction of network activities with an interactive LLM Q\u0026A chat, custom context input, and story timeline |\n| **Custom Signature Rules** | YAML-based detection rules matched against IP, CIDR, port, JA3, hostname, app, and protocol fields; live-reloaded without restart |\n| **Export Options** | PDF report (with live topology capture), per-conversation PCAP, bulk PCAP export, and CSV export |\n| **Real-time Processing** | Asynchronous analysis with detailed progress tracking |\n| **Multi-protocol Support** | TCP, UDP, ICMP, and application-layer protocols including TLS, HTTP, DNS, QUIC, and L2 protocols (ARP, STP, LLDP, CDP) |\n\n## Quick Start\n\n### Prerequisites\n\n| Software | Version | Purpose |\n|----------|---------|---------|\n| Docker | Latest | Container runtime |\n| Docker Compose | Latest | Multi-container orchestration |\n| LLM Server | Any OpenAI-compatible API | AI filter generation (e.g., LM Studio, Ollama, OpenAI) |\n\n**Minimum Hardware:**\n- RAM: 4GB (8GB+ recommended)\n- Storage: 10GB (for database, PCAP files, and object storage)\n\n### Installation\n\n**1. Clone and setup:**\n```bash\ngit clone https://github.com/NotYuSheng/TracePcap.git\ncd TracePcap\ncp .env.example .env\n```\n\n**2. Configure `.env`:**\n```env\n# Upload Configuration\nMAX_UPLOAD_SIZE_BYTES=536870912  # 512MB default\n\n# Nginx Port Configuration\nNGINX_PORT=80  # Change if port 80 is already in use\n\n# LLM Configuration (Local LLM Server)\nLLM_API_BASE_URL=http://localhost:1234/v1\nLLM_API_KEY=\nLLM_MODEL=Qwen2.5-14B-Coder-Instruct\nLLM_TEMPERATURE=0.7\nLLM_MAX_TOKENS=2000\n```\n\n**3. Start the application:**\n```bash\ndocker compose up -d\n```\n\n**4. Access TracePcap:**\n\nOpen **http://localhost:80** in your browser.\n\n\u003e **Note**: The application includes PostgreSQL for metadata storage and MinIO for PCAP file storage. First startup may take a few minutes while containers initialize.\n\n## Usage\n\n### Basic Workflow\n\n1. **Upload** — Drag-and-drop a PCAP/PCAPNG file; optionally enable nDPI analysis and file extraction before uploading\n2. **Analyze** — File is processed asynchronously with a detailed progress view\n3. **Overview** — Review detected applications, protocols, risk alerts, and custom signature matches\n4. **Visualize** — Explore the interactive network topology with filters, layout controls, and node detail panels\n5. **Conversations** — Review flows with advanced filtering, session reconstruction, and payload inspection\n6. **Story Mode** — Read the AI-generated narrative and ask follow-up questions via LLM chat\n7. **Extracted Files** — Browse and download files recovered from HTTP responses and raw streams\n8. **Generate Filters** — Use AI to create Wireshark/tcpdump filters from natural language (e.g., \"show all TLS traffic to external IPs\")\n9. **Export** — Download a PDF report, per-conversation or bulk PCAP, or CSV\n\n### Supported File Formats\n\nPCAP, PCAPNG, CAP (max 512MB default, configurable via `MAX_UPLOAD_SIZE_BYTES`)\n\n## Tech Stack\n\n| Component | Technology |\n|-----------|------------|\n| **Backend** | Spring Boot 3.2.1, Java 21, Maven, Lombok, MapStruct |\n| **Architecture** | Layered architecture (Controller → Service → Repository → Database) |\n| **Frontend** | React 19, Vite, TypeScript, Lucide Icons, SGDS React |\n| **Visualization** | React Flow + ELK (network topology), Recharts, D3.js |\n| **Reverse Proxy** | Nginx |\n| **Packet Parsing** | tshark / Wireshark, nDPI v5 (deep packet inspection) |\n| **Database** | PostgreSQL 15 with Flyway migrations |\n| **Object Storage** | MinIO (S3-compatible) |\n| **Containerization** | Docker, Docker Compose |\n| **API Documentation** | SpringDoc OpenAPI (Swagger UI) |\n\n## Documentation\n\nComprehensive documentation is available in the [`docs/`](docs/) directory:\n\n| Document | Description |\n|----------|-------------|\n| **[API Endpoints](docs/API_ENDPOINTS.md)** | Complete REST API documentation with request/response examples |\n| **[Backend Structure](docs/BACKEND_STRUCTURE.md)** | Backend architecture, package organization, and design patterns |\n\n## Common Tasks\n\n### View Logs\n\n```bash\n# All services\ndocker compose logs -f\n\n# Specific service\ndocker compose logs -f tracepcap-backend\ndocker compose logs -f postgres\ndocker compose logs -f minio\n```\n\n### Restart Services\n\n```bash\n# Restart all\ndocker compose restart\n\n# Restart backend only\ndocker compose restart tracepcap-backend\n```\n\n### Backup Data\n\n```bash\n# Backup database\ndocker exec tracepcap-postgres pg_dump -U tracepcap_user tracepcap \u003e backup.sql\n\n# Backup MinIO data (PCAP files)\ndocker exec tracepcap-minio mc mirror minio/tracepcap-files ./backup-pcaps/\n\n# Backup all volumes\nsudo tar -czf tracepcap_backup.tar.gz /var/lib/docker/volumes/tracepcap_*\n```\n\n### Access Database\n\n```bash\ndocker exec -it tracepcap-postgres psql -U tracepcap_user tracepcap\n```\n\n### Access MinIO Console\n\nNavigate to **http://localhost:9001** and login with:\n- Username: `minioadmin`\n- Password: `minioadmin`\n\n### Access Swagger API Documentation\n\nNavigate to **http://localhost:80/swagger-ui.html** to explore the API interactively.\n\n## Deployment\n\nTracePcap is designed for self-hosted deployment:\n\n- **Development**: Use built-in configuration with exposed ports\n- **Production**:\n  - Change default MinIO credentials in `docker-compose.yml`\n  - Update PostgreSQL password\n  - Configure reverse proxy with SSL/TLS\n  - Adjust `MAX_UPLOAD_SIZE_BYTES` based on your needs\n  - Set appropriate LLM configuration for your infrastructure\n\n## Security\n\n- **Local Processing**: PCAP analysis runs entirely on your server\n- **Data Privacy**: Network captures never leave your infrastructure (except LLM filter generation)\n- **Object Storage**: MinIO provides S3-compatible secure file storage\n- **Database**: PostgreSQL not exposed outside Docker network by default\n- **No Authentication**: Add authentication layer for multi-user deployments\n- **LLM Privacy**: Use local LLM servers (LM Studio, Ollama) to keep filter queries private\n\n## Performance\n\n- **Async Processing**: File analysis runs asynchronously to prevent blocking\n- **Object Storage**: MinIO provides scalable storage for large PCAP files\n- **Database Indexing**: Optimized queries with proper indexing for fast lookups\n- **Connection Pooling**: Efficient database connection management\n- **File Size Limits**: Configurable upload limits to prevent resource exhaustion\n\n## Architecture Highlights\n\n- **Layered Architecture**: Clean separation of concerns (API → Service → Repository → Database)\n- **DTO Pattern**: MapStruct for efficient object mapping between layers\n- **Database Migrations**: Flyway for version-controlled schema management\n- **Health Checks**: Built-in health checks for all services\n- **API-First Design**: OpenAPI specification with Swagger UI\n\n## Custom Signature Rules\n\nTracePcap supports user-defined detection rules that are matched against every conversation after nDPI analysis. Matched rule names appear as color-coded badges in the Conversations tab and Overview, alongside nDPI's built-in detections.\n\n### How it works\n\nRules live inside a Docker named volume (`config_data`) at `/app/config/signatures.yml` inside the backend container. The file is reloaded on every analysis run — no restart required after editing.\n\nClick **Custom Detection Rules** in the navbar to open the built-in YAML editor. Changes are saved immediately — no restart required.\n\n\u003e **`signatures.sample.yml`** in the repo root is a reference template with demo rules covering every match field. Paste it into the browser editor to get started.\n\n### Rule format\n\n```yaml\nsignatures:\n  - name: rule_name_shown_in_ui   # shown as a badge (use underscores, no spaces)\n    description: Human-readable description\n    severity: low                  # low | medium | high | critical\n    match:\n      ip: \"203.0.113.42\"           # exact match against srcIp OR dstIp\n```\n\nA rule fires when **all** specified match fields are satisfied. All fields are optional — mix and match as needed.\n\n### Match fields\n\n| Field | Type | Description | Example |\n|-------|------|-------------|---------|\n| `ip` | string | Exact match against srcIp or dstIp | `\"203.0.113.42\"` |\n| `cidr` | string | CIDR range match against srcIp or dstIp | `\"10.0.0.0/8\"` |\n| `srcPort` | number | Exact source port | `67` |\n| `dstPort` | number | Exact destination port | `4444` |\n| `ja3` | string | Exact JA3S fingerprint hash (nDPI 5.x) | `\"82f0d8a75fa483d1cfe4b7085b784d7e\"` |\n| `hostname` | string | Exact or wildcard SNI hostname — `*.evil.com` matches any subdomain at any depth | `\"*.evil.com\"` |\n| `app` | string | Case-insensitive nDPI application name | `\"Telegram\"`, `\"TOR\"`, `\"SIP\"`, `\"RTP\"` |\n| `protocol` | string | Case-insensitive transport protocol | `\"TCP\"`, `\"UDP\"`, `\"ICMP\"` |\n\n### Severity colors\n\n| Severity | Color |\n|----------|-------|\n| `critical` | Red |\n| `high` | Orange |\n| `medium` | Yellow |\n| `low` | Purple |\n\n### Examples\n\n```yaml\nsignatures:\n\n  # Flag a known C2 IP\n  - name: known_c2_ip\n    description: Known command-and-control server\n    severity: high\n    match:\n      ip: \"203.0.113.42\"\n\n  # Flag all traffic to a suspicious subnet\n  - name: flagged_subnet\n    severity: medium\n    match:\n      cidr: \"198.51.100.0/24\"\n\n  # Detect DNS over TCP (possible zone transfer or tunnelling)\n  - name: dns_over_tcp\n    severity: medium\n    match:\n      app: \"DNS\"\n      protocol: TCP\n\n  # Wildcard hostname match\n  - name: blocked_domain\n    severity: high\n    match:\n      hostname: \"*.malware.example.com\"\n\n  # JA3S fingerprint from a threat-intel feed\n  - name: suspicious_tls_fingerprint\n    severity: critical\n    match:\n      ja3: \"a0e9f5d64349fb13191bc781f81f42e1\"\n```\n\nA full set of 12 demo rules covering every match field type is available in [`signatures.sample.yml`](signatures.sample.yml). The script [`sample-files/gen_demo.py`](sample-files/gen_demo.py) generates a PCAP file that triggers all 12 rules at once.\n\n## Sample Files\n\nThe [`sample-files/`](sample-files/) directory contains example PCAP files:\n\n- `atm_capture1.cap` - ATM network traffic sample\n- `free5gc.pcap` - 5G core network traffic sample\n- `demo_all_rules.pcap` - Triggers all 12 custom signature demo rules (generated by `gen_demo.py`)\n\nThese files can be used to test the application's analysis capabilities.\n\n## License\n\nThis project is licensed under the MIT License. See [LICENSE](LICENSE) for details.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnotyusheng%2Ftracepcap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnotyusheng%2Ftracepcap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnotyusheng%2Ftracepcap/lists"}