An open API service indexing awesome lists of open source software.

https://github.com/sipcapture/homer

HOMER - 100% Open-Source SIP, VoIP, RTC Packet Capture & Monitoring
https://github.com/sipcapture/homer

analytics callflow capture-agent cdr correlation encapsulation flow hep kamailio monitoring opensips packet-capture packet-sniffer pcap rtc sip statistics troubleshooting voip webrtc

Last synced: 14 days ago
JSON representation

HOMER - 100% Open-Source SIP, VoIP, RTC Packet Capture & Monitoring

Awesome Lists containing this project

README

          

# homer eleven
> 100% Opensource Telecom Observability since 2011

`homer` is the *all-in-one* HEP capture and API server monolith powering Homer 11.x data lake

homer11

## Features

- All-in-One Application *(Writer, Reader, Coordinator, Compactor, API)*
- Modern Codebase in golang for X64/ARM64 on Linux/MacOS
- Powered by DuckDB 1.5 and Apache Arrow/IPC/Parquet
- Datalake design based on DuckLake Catalog and Local/Object Storage
- End-to-End Columnar OTLP Design w/ on-demand query execution
- Linear Scaling to query over shared Object Storage catalog/pool
- Flexible Schema support for growing problems and protocols
- Backwards compatible with all HEPv3 Agents
- Easy to maintain, operate and scale *(down to zero!)*
- Cloud Native Design for K8s and standard deployments
- Built-In User Interface for Humans
- MCP support and LLM/Agent friendly design *(steal our boring jobs)*

## Architecture

Homer uses a modular architecture with four main components:

```
┌─────────────────────────────────────────────────────────────────────────────┐
│ Homer Core │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌──────────┐ ┌─────────────────┐ │
│ │ Ingest │ │ Storage │ │ Node │ │ Coordinator │ │
│ │ (HEP recv) │──▶│ (DuckLake) │──▶│ gRPC/HTTP│──▶│ (REST API) │ │
│ └─────────────┘ └─────────────┘ └──────────┘ └─────────────────┘ │
│ UDP/TCP/HTTP Parquet+S3 Airport :50051 HTTP :8080 │
│ FlightSQL :50055 (opt. proxy :32010) │
└─────────────────────────────────────────────────────────────────────────────┘
```

### Modules

| Module | Description |
|--------|-------------|
| **Ingest** | Receives HEP packets via UDP/TCP/TLS/HTTP/HTTPS |
| **Storage** | Writes data to DuckLake (Parquet + catalog) |
| **Node** | Airport gRPC + HTTP `/query`; optional Arrow FlightSQL for Grafana ([docs/FLIGHTSQL.md](docs/FLIGHTSQL.md)) |
| **Coordinator** | REST API gateway for UI and external applications |

## Quick Start

### Install Options

- Download a [release](https://github.com/sipcapture/homer/releases) and install the `homer` package on your OS/arch
- Use the official [docker builds](https://github.com/sipcapture/homer/pkgs/container/homer) and [compose examples](https://github.com/sipcapture/homer/tree/homer11/examples/docker)
- Build your own release locally _(keep reading)_

### All-in-One Deployment

```json
{
"ingest": {
"enable": true,
"udp": { "enable": true, "port": 9060 },
"tcp": { "enable": true, "port": 9061 },
"http": { "enable": true, "port": 9080 }
},
"storage": {
"enable": true,
"ducklake": {
"catalog_path": "/data/homer/homer_catalog.sqlite",
"data_path": "/data/homer/parquet"
}
},
"node": {
"enable": true,
"flight_server": { "port": 50051 }
},
"coordinator": {
"enable": true,
"http_server": { "port": 8080 }
}
}
```

> For more storage options consult the available [STORAGE POLICIES](https://github.com/sipcapture/homer/blob/homer11/docs/STORAGE_POLICIES.md)

### Build & Run

```bash
# Build
make

# Run as server (default mode)
./homer --config-path /etc/homer/homer.json

# With debug logging
./homer --config-path /etc/homer/homer.json --log-level debug
```

## Subcommands

Homer uses a subcommand-based CLI. Running `homer` without arguments starts the server.

```
homer Run the server (default)
homer search [flags] Search Homer data via coordinator API
homer cli [flags] Interactive DuckLake SQL shell
homer system [flags] System operations (compaction, extensions, reload)
homer wizard [flags] Interactive config generator wizard
homer mcp [flags] Start MCP stdio server
homer version Show version
homer help Show full help with all flags
```

### Server Mode (default)

```bash
homer --config-path /etc/homer/homer.json
homer --config-path /etc/homer/homer.json --log-level debug --syslog-disable
```

| Flag | Description |
|------|-------------|
| `--config-path ` | Path to config file or directory |
| `--log-level ` | Log level: debug, info, warn, error, trace |
| `--syslog-disable` | Disable syslog, use only stdout |
| `--pid-file ` | PID file path (default: /var/run/homer-core.pid) |

### Search (via coordinator API)

Search Homer data from the command line with table, vertical, CSV, JSON, chart, call flow, or **PCAP** output (SIP only; use `--format pcap` with `-o` / `--output`).

```bash
# Basic SIP search (last hour)
homer search --host 10.0.0.1:8081 --user admin --pass secret

# Search INVITE messages with call flow diagram
homer search --host 10.0.0.1:8081 --method INVITE --format callflow

# Export SIP messages to a pcap file (same framing as API /transactions/export/pcap)
homer search --host 10.0.0.1:8081 --call-id "abc123@host" --format pcap -o /tmp/call.pcap

# Search by Call-ID
homer search --host 10.0.0.1:8081 --call-id "abc123@host" --format vertical

# Post-filter: only INVITEs and BYEs, exclude provisional responses
homer search --host 10.0.0.1:8081 --grep "INVITE,BYE" --exclude "100,183"

# Interactive TUI mode
homer search --host 10.0.0.1:8081 --interactive
```

See [docs/SEARCH.md](docs/SEARCH.md) for full documentation and examples.

### CLI (DuckLake SQL Shell)

Interactive SQL shell for direct DuckLake queries:

```bash
# Start interactive CLI
homer cli --config-path /etc/homer/homer.json

# Execute single query and exit
homer cli --config-path /etc/homer/homer.json --query "SELECT COUNT(*) FROM homer_lake.main.hep_proto_1_call"
```

| Command | Description |
|---------|-------------|
| `help`, `\h`, `\?` | Show help |
| `tables`, `\dt` | List available tables |
| `clear`, `\c` | Clear screen |
| `exit`, `quit`, `\q` | Exit CLI |

### System Operations

```bash
# Run full compaction
homer system --config-path /etc/homer/homer.json --compaction-force

# Install DuckDB extensions
homer system --config-path /etc/homer/homer.json --install-extensions

# Show DuckDB version
homer system --config-path /etc/homer/homer.json --duckdb-version

# Generate example config
homer system --generate-example-config > homer.json

# Reload running process
homer system --reload
```

### Wizard (Config Generator)

Interactive wizard that generates a complete `homer.json` config:

```bash
# Interactive TUI wizard
homer wizard

# Non-interactive: generate config for a specific deployment profile
homer wizard --profile all-in-one --output homer.json
homer wizard --profile writer --output homer-writer.json
homer wizard --profile coordinator --output homer-coordinator.json
homer wizard --profile edge --output homer-edge.json
homer wizard --profile node --output homer-node.json
```

| Profile | Modules Enabled |
|---------|----------------|
| `all-in-one` | ingest + storage + node + coordinator |
| `writer` | ingest + storage |
| `edge` | ingest + storage + node |
| `coordinator` | coordinator only |
| `node` | node only |

## Configuration Examples

See the `examples/` directory:

| File | Description |
|------|-------------|
| `homer.json` | All-in-one deployment |
| `homer-writer.json` | Ingest + Storage + Node |
| `homer-node.json` | Node only (read-only) |
| `homer-coordinator.json` | Coordinator only |
| `homer-edge.json` | Edge deployment |

## Documentation

**Published docs (GitHub Pages):** [https://sipcapture.github.io/homer/](https://sipcapture.github.io/homer/) — built from `docs/` via MkDocs on push to `homer11`. Enable **Settings → Pages → Build and deployment: GitHub Actions** once after the first workflow run.

```bash
python3 -m venv .venv-docs && .venv-docs/bin/pip install -r docs-requirements.txt
.venv-docs/bin/mkdocs serve # http://127.0.0.1:8000
```

- [Search CLI](docs/SEARCH.md) - Search from the command line (examples, formats, call flow)
- [Config Wizard](docs/WIZARD.md) - Interactive config generator (TUI + presets)
- [Ingest performance tuning](docs/INGEST_PERFORMANCE.md) - multicore, DuckLake batch size, Prometheus flush interval; **`scripts/profile_ingest_load.sh`** / **`make profile-ingest`** for repeatable CPU profiles
- [Coordinator Module](docs/COORDINATOR.md) - REST API gateway
- [MCP UI Guide](docs/MCP_UI_GUIDE.md) - Natural-language query assistant (configuration + UI usage)
- [Node Module](docs/NODE.md) - FlightSQL data server
- [Storage Architecture](docs/STORAGE_ARCHITECTURE.md) - DuckLake storage
- [Storage Policies](docs/STORAGE_POLICIES.md) - Tiered storage (hot/cold)
- [Compaction Setup](docs/COMPACTION_SETUP.md) - File compaction

## License
Released under the [AGPL-3.0 License](LICENSE.md)

> Copyright (C) 2025 QXIP BV