{"id":30045534,"url":"https://github.com/saviobatista/sbs-logger","last_synced_at":"2026-04-18T11:02:53.666Z","repository":{"id":301540967,"uuid":"1008282496","full_name":"saviobatista/sbs-logger","owner":"saviobatista","description":"🚁 High-performance distributed SBS/ADS-B message processor with real-time aircraft tracking, flight session management, and TimescaleDB storage. Built with Go microservices and NATS messaging.","archived":false,"fork":false,"pushed_at":"2025-11-24T10:50:40.000Z","size":279,"stargazers_count":3,"open_issues_count":12,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-02T02:12:39.472Z","etag":null,"topics":["ads-b","aircraft-tracking","aviation","distributed-systems","docker","flight-data","flight-sessions","go","microservices","nats","real-time","redis","sbs","time-series","timescaledb"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/saviobatista.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-25T09:57:26.000Z","updated_at":"2025-10-23T18:30:20.000Z","dependencies_parsed_at":"2025-06-27T12:15:46.014Z","dependency_job_id":"4357d586-e9e0-4504-b8f4-c7f749eb04cb","html_url":"https://github.com/saviobatista/sbs-logger","commit_stats":null,"previous_names":["saviobatista/sbs-logger"],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/saviobatista/sbs-logger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saviobatista%2Fsbs-logger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saviobatista%2Fsbs-logger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saviobatista%2Fsbs-logger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saviobatista%2Fsbs-logger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/saviobatista","download_url":"https://codeload.github.com/saviobatista/sbs-logger/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saviobatista%2Fsbs-logger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31966217,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T00:39:45.007Z","status":"online","status_checked_at":"2026-04-18T02:00:07.018Z","response_time":103,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ads-b","aircraft-tracking","aviation","distributed-systems","docker","flight-data","flight-sessions","go","microservices","nats","real-time","redis","sbs","time-series","timescaledb"],"created_at":"2025-08-07T07:52:36.556Z","updated_at":"2026-04-18T11:02:53.660Z","avatar_url":"https://github.com/saviobatista.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SBS Logger\n\n[![Go Version](https://img.shields.io/badge/Go-1.24.5+-blue.svg)](https://golang.org)\n[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)\n[![Go Report Card](https://goreportcard.com/badge/github.com/saviobatista/sbs-logger)](https://goreportcard.com/report/github.com/saviobatista/sbs-logger)\n[![Code Coverage](https://codecov.io/gh/saviobatista/sbs-logger/branch/main/graph/badge.svg)](https://codecov.io/gh/saviobatista/sbs-logger)\n[![Security Scan](https://github.com/saviobatista/sbs-logger/workflows/CI%2FCD%20Pipeline/badge.svg)](https://github.com/saviobatista/sbs-logger/actions/workflows/ci.yml)\n[![Docker Build](https://img.shields.io/badge/Docker-Build%20Passing-brightgreen.svg)](https://github.com/saviobatista/sbs-logger/actions/workflows/ci.yml)\n[![Trivy Security](https://img.shields.io/badge/Trivy-Security%20Scan-brightgreen.svg)](https://github.com/saviobatista/sbs-logger/actions/workflows/ci.yml)\n[![Go Modules](https://img.shields.io/badge/Go%20Modules-Go%201.24.5+-blue.svg)](go.mod)\n[![Dependabot](https://img.shields.io/badge/Dependabot-Enabled-brightgreen.svg)](https://github.com/saviobatista/sbs-logger/security/dependabot)\n[![GitHub Issues](https://img.shields.io/github/issues/saviobatista/sbs-logger)](https://github.com/saviobatista/sbs-logger/issues)\n[![GitHub Pull Requests](https://img.shields.io/github/issues-pr/saviobatista/sbs-logger)](https://github.com/saviobatista/sbs-logger/pulls)\n\nA high-performance, distributed Go application for capturing, processing, and storing SBS (BaseStation) messages from ADS-B receivers. The system provides real-time aircraft tracking, flight session management, and comprehensive data persistence with TimescaleDB.\n\n## 🚀 Features\n\n- **Real-time SBS Message Ingestion**: Connects to multiple ADS-B receivers simultaneously\n- **Distributed Architecture**: Microservices-based design with NATS messaging\n- **Aircraft State Tracking**: Real-time position, altitude, speed, and flight data\n- **Flight Session Management**: Automatic flight detection and session tracking\n- **High-Performance Storage**: TimescaleDB for time-series data with automatic retention policies\n- **Redis Caching**: Fast access to active aircraft states and flight data\n- **Comprehensive Logging**: Daily log rotation with automatic compression\n- **Statistics \u0026 Monitoring**: Real-time system metrics and performance tracking\n- **Docker Support**: Complete containerized deployment with docker-compose\n\n## 🛡️ Quality \u0026 Security\n\nThis project maintains high code quality and security standards through automated checks:\n\n- **🔍 Code Quality**: Automated linting with golangci-lint\n- **🧪 Testing**: Comprehensive unit and integration tests with race condition detection\n- **📊 Code Coverage**: Continuous coverage tracking with Codecov\n- **🔒 Security Scanning**: Automated vulnerability scanning with Trivy\n- **🐳 Container Security**: Docker image scanning and multi-platform builds\n- **📋 Go Report Card**: Code quality analysis and grading\n- **🔄 CI/CD**: Automated testing, building, and deployment pipeline\n\n## 🏗️ Architecture\n\nThe system consists of several microservices that communicate via NATS:\n\n```\n┌─────────────┐    ┌─────────────┐    ┌─────────────┐\n│   Ingestor  │───▶│     NATS    │───▶│   Logger    │\n│             │    │             │    │             │\n└─────────────┘    └─────────────┘    └─────────────┘\n                           │\n                           ▼\n                   ┌─────────────┐\n                   │   Tracker   │\n                   │             │\n                   └─────────────┘\n                           │\n                    ┌──────┴──────┐\n                    ▼             ▼\n            ┌─────────────┐ ┌─────────────┐\n            │ TimescaleDB │ │    Redis    │\n            │             │ │             │\n            └─────────────┘ └─────────────┘\n```\n\n### Components\n\n- **Ingestor**: Connects to SBS sources and publishes messages to NATS\n- **Logger**: Subscribes to messages and writes to daily log files\n- **Tracker**: Processes messages, tracks aircraft states, manages flight sessions, and handles database migrations\n- **NATS**: Message broker for inter-service communication\n- **TimescaleDB**: Time-series database for aircraft states and statistics\n- **Redis**: Caching layer for active aircraft and flight data\n\n## 📋 Prerequisites\n\n- Go 1.24.5 or later\n- Docker and Docker Compose\n- PostgreSQL/TimescaleDB\n- Redis\n- NATS Server\n\n## 🛠️ Installation\n\n### Option 1: Docker Compose (Recommended)\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/saviobatista/sbs-logger.git\ncd sbs-logger\n```\n\n2. Configure environment variables:\n```bash\n# Copy the sample environment file\ncp .env.sample .env\n\n# Edit the .env file with your configuration\nnano .env\n```\n\n3. Update the key variables in `.env`:\n```bash\n# Your ADS-B receiver(s)\nSOURCES=your-adsb-receiver:30003,another-receiver:30003\n\n# Your receiver location (for Ultrafeeder)\nULTRAFEEDER_LAT=your_latitude\nULTRAFEEDER_LON=your_longitude\nULTRAFEEDER_ALT=your_altitude\n\n# Database credentials (change for production)\nPOSTGRES_PASSWORD=your_secure_password\n```\n\n4. Start the services:\n```bash\ndocker-compose up -d\n```\n\n### Option 2: Manual Build\n\n1. Install dependencies:\n```bash\ngo mod download\n```\n\n2. Build all components:\n```bash\ngo build ./cmd/ingestor\ngo build ./cmd/logger\ngo build ./cmd/tracker\n```\n\n3. Start the services:\n```bash\n# Migrations will run automatically when the tracker service starts\n./tracker\n```\n\nThe tracker service will automatically run database migrations on startup.\n\n## ⚙️ Configuration\n\n### Environment Setup\n\nThe project uses a `.env` file for configuration. Start by copying the sample file:\n\n```bash\ncp .env.sample .env\n```\n\n### Environment Variables\n\n#### Ingestor\n- `SOURCES`: Comma-separated list of SBS sources (e.g., `10.0.0.1:30003,10.0.0.2:30003`)\n- `NATS_URL`: NATS server URL (default: `nats://nats:4222`)\n\n#### Logger\n- `OUTPUT_DIR`: Directory for log files (default: `./logs`)\n- `NATS_URL`: NATS server URL (default: `nats://nats:4222`)\n\n#### Tracker\n- `NATS_URL`: NATS server URL (default: `nats://nats:4222`)\n- `DB_CONN_STR`: Database connection string\n- `REDIS_ADDR`: Redis server address (default: `redis:6379`)\n\n### Environment Variables Organization\n\nThe `.env.sample` file is organized into sections:\n\n- **Ultrafeeder Configuration**: ADS-B receiver settings and web interface\n- **SBS Services**: Configuration for ingestor, logger, and tracker\n- **Database Configuration**: TimescaleDB connection settings\n- **Security**: Optional authentication and SSL settings\n\n### Database Schema\n\nThe system uses TimescaleDB with the following main tables:\n\n- `aircraft_states`: Time-series table for aircraft position and state data\n- `flights`: Flight session information\n- `system_stats`: System performance and statistics\n\n### NATS Configuration\n\nNATS is configured with JetStream enabled for message persistence:\n\n```conf\nport: 4222\nhttp_port: 8222\n\njetstream {\n    store_dir: \"/data\"\n    max_memory_store: 1G\n    max_file_store: 10G\n}\n```\n\n## 📊 Data Processing\n\n### SBS Message Types\n\nThe system processes the following SBS message types:\n\n- **MSG,1**: Selection change\n- **MSG,2**: New aircraft\n- **MSG,3**: New ID\n- **MSG,4**: New callsign\n- **MSG,5**: New altitude\n- **MSG,6**: New ground speed\n- **MSG,7**: New track\n- **MSG,8**: New lat/lon (position)\n- **MSG,9**: New ground status\n\n### Aircraft State Tracking\n\nThe tracker maintains real-time state for each aircraft:\n\n- Position (latitude/longitude)\n- Altitude and vertical rate\n- Ground speed and track\n- Callsign and squawk\n- Ground status\n\n### Flight Sessions\n\nFlight sessions are automatically detected and tracked:\n\n- Session start/end times\n- Flight path (first/last position)\n- Maximum altitude and speed\n- Session statistics\n\n## 📈 Monitoring \u0026 Statistics\n\nThe system provides comprehensive statistics:\n\n- Message processing rates\n- Aircraft and flight counts\n- Processing performance metrics\n- Error rates and system health\n\nStatistics are logged every minute and persisted to the database every 5 minutes.\n\n## 🔧 Development\n\n### Project Structure\n\n```\nsbs-logger/\n├── cmd/                    # Application entry points\n│   ├── ingestor/          # SBS message ingestion\n│   ├── logger/            # Log file management\n│   └── tracker/           # Aircraft state tracking\n├── internal/              # Private application code\n│   ├── capture/           # Network capture logic\n│   ├── config/            # Configuration management\n│   ├── db/                # Database operations\n│   ├── nats/              # NATS client\n│   ├── parser/            # SBS message parsing\n│   ├── redis/             # Redis client\n│   ├── stats/             # Statistics tracking\n│   ├── storage/           # Storage abstractions\n│   └── types/             # Data structures\n├── config/                # Configuration files\n│   └── nats/              # NATS server configuration\n├── logs/                  # Log file output\n└── docker-compose.yml     # Container orchestration\n```\n\n### Running Tests\n\n```bash\ngo test ./...\n```\n\n### Building for Production\n\n```bash\n# Build all components\nmake build\n\n# Build individual components\nmake build-ingestor\nmake build-logger\nmake build-tracker\n```\n\n### Docker Hub Publishing\n\nThe project includes automated Docker Hub publishing. To set up:\n\n1. **Setup Docker Hub publishing**:\n```bash\nmake dockerhub-setup\n```\n\n2. **Test Docker builds locally**:\n```bash\nmake dockerhub-test\n```\n\n3. **Manual push to Docker Hub**:\n```bash\nmake dockerhub-push DOCKERHUB_USERNAME=youruser DOCKERHUB_TOKEN=yourtoken VERSION=v1.0.0\n```\n\n4. **Automated publishing**: Create a GitHub release to trigger automatic publishing\n\nFor detailed setup instructions, see [Docker Hub Setup Guide](docs/dockerhub-setup.md).\n\n## 🚀 Deployment\n\n### Docker Images\n\nThe project provides pre-built Docker images on multiple registries:\n\n#### GitHub Container Registry (GHCR)\n```bash\n# Pull images from GHCR\ndocker pull ghcr.io/saviobatista/sbs-logger/sbs-ingestor:latest\ndocker pull ghcr.io/saviobatista/sbs-logger/sbs-logger:latest\ndocker pull ghcr.io/saviobatista/sbs-logger/sbs-tracker:latest\n```\n\n#### Docker Hub\n```bash\n# Pull images from Docker Hub\ndocker pull saviobatista/sbs-ingestor:latest\ndocker pull saviobatista/sbs-logger:latest\ndocker pull saviobatista/sbs-tracker:latest\n```\n\n### Production Considerations\n\n1. **Scaling**: Run multiple ingestor instances for high availability\n2. **Storage**: Configure appropriate retention policies for TimescaleDB\n3. **Monitoring**: Set up monitoring for NATS, Redis, and TimescaleDB\n4. **Backup**: Implement regular database backups\n5. **Security**: Use TLS for NATS and database connections\n\n### Docker Deployment\n\n```bash\n# Production deployment\ndocker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d\n\n# Scale services\ndocker-compose up -d --scale ingestor=3\n```\n\n### Using Pre-built Images\n\nUpdate your `docker-compose.yml` to use pre-built images:\n\n```yaml\nservices:\n  ingestor:\n    image: saviobatista/sbs-ingestor:latest\n    # or: image: ghcr.io/saviobatista/sbs-logger/sbs-ingestor:latest\n    environment:\n      - SOURCES=your-adsb-receiver:30003\n      - NATS_URL=nats://nats:4222\n    depends_on:\n      - nats\n\n  logger:\n    image: saviobatista/sbs-logger:latest\n    environment:\n      - OUTPUT_DIR=/app/logs\n      - NATS_URL=nats://nats:4222\n    volumes:\n      - ./logs:/app/logs\n    depends_on:\n      - nats\n```\n\n## 📝 Logging\n\nLogs are written to daily files with automatic rotation:\n\n- Format: `sbs_YYYY-MM-DD.log`\n- Compression: Previous day's logs are automatically compressed\n- Location: `./logs/` directory (configurable)\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## 📄 License\n\nMIT License - see LICENSE file for details.\n\n## 🆘 Support\n\nFor issues and questions:\n\n1. Check the documentation\n2. Search existing issues\n3. Create a new issue with detailed information\n\n## 🔗 Related Projects\n\n- [ADS-B Exchange](https://www.adsbexchange.com/)\n- [FlightAware](https://flightaware.com/)\n- [OpenSky Network](https://opensky-network.org/) ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaviobatista%2Fsbs-logger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsaviobatista%2Fsbs-logger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaviobatista%2Fsbs-logger/lists"}