Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/FrigadeHQ/trench

Trench — Open-Source Analytics Infrastructure
https://github.com/FrigadeHQ/trench

analytics clickhouse clickhouse-database clickhouse-server event-replay javascript kafka kafka-consumer kafka-engine kafka-producer nodejs open-source product-analytics self-host self-hosting tracking tracking-events

Last synced: about 1 month ago
JSON representation

Trench — Open-Source Analytics Infrastructure

Host: GitHub
URL: https://github.com/FrigadeHQ/trench
Owner: FrigadeHQ
License: mit
Created: 2024-09-30T19:19:01.000Z (2 months ago)
Default Branch: main
Last Pushed: 2024-10-25T14:16:06.000Z (about 2 months ago)
Last Synced: 2024-10-25T14:22:12.580Z (about 2 months ago)
Topics: analytics, clickhouse, clickhouse-database, clickhouse-server, event-replay, javascript, kafka, kafka-consumer, kafka-engine, kafka-producer, nodejs, open-source, product-analytics, self-host, self-hosting, tracking, tracking-events
Language: TypeScript
Homepage: https://trench.dev
Size: 4.32 MB
Stars: 21
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

awesome-clickhouse - FrigadeHQ/trench - Trench is an open-source analytics infrastructure designed for tracking events and providing real-time data analytics. (Integrations / Data Transfer and Synchronization)

README

Open-Source Analytics Infrastructure

Documentation ·
Website ·
Slack Community ·
Demo

## 🌊 What is Trench?

Trench is an event tracking system built on top of Apache Kafka and Clickhouse. It can handle large event volumes and provides real-time analytics. Trench is no-cookie, GDPR, and PECR compliant. Users have full control to access, rectify, or delete their data.

Our team built Trench to scale up the real-time event tracking pipeline at Frigade.

Trench Code Snippet

## ⭐ Features

- 🤝 Compliant with the Segment API (Track, Group, Identify)
- 🐳 Deploy quickly with a single production-ready Docker image
- 💻 Process thousands of events per second on a single node
- ⚡ Query data in real-time
- 🔗 Connect data to other destinations with webhooks
- 👥 Open-source and MIT Licensed

## 🖥️ Demo

**Live demo:**
[https://demo.trench.dev](https://demo.trench.dev)

**Video demo:**

Watch the following demo to see how you can build a basic version of Google Analytics using Trench and Grafana.

https://github.com/user-attachments/assets/e3f64590-6e7e-41b9-b425-7adb5a1e19b1

## 🚀 Quickstart

Trench has two methods of deployment:

1. **Trench Self-Hosted**: An open-source version to deploy and manage Trench on your own infrastructure.
2. **Trench Cloud**: A fully-managed serverless solution with zero ops, autoscaling, 99.99% SLAs.

### 1. Trench Self-Hosted 💻

Follow our self-hosting instructions below and in our [quickstart guide](https://docs.trench.dev/quickstart) to begin using Trench Self-Hosted.

If you have questions or need assistance, you can join our [Slack group](https://join.slack.com/t/trench-community/shared_invite/zt-2sjet5kh2-v31As3yC_zRIadk_AGn~3A) for support.

#### Quickstart

1. **Deploy Trench Dev Server**:
The only prerequisite for Trench is a system that has Docker and Docker Compose installed [see installation guide](https://docs.docker.com/compose/install/). We recommend having at least 4GB of RAM and 4 CPU cores for optimal performance if you're running a production environment.

After installing Docker, you can start the local development server by running the following commands:

```sh
git clone https://github.com/frigadehq/trench.git
cd trench/apps/trench
cp .env.example .env
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up --build --force-recreate --renew-anon-volumes
```

The above command will start the Trench server that includes a local Clickhouse and Kafka instance on `http://localhost:4000`. You can open this URL in your browser and you should see the message `Trench server is running`. You shouldupdate the `.env` file to change any of the configuration options.

2. **Send a sample event**:
You can find and update the default public and private API key in the `.env` file. Using your public API key, you can send a sample event to Trench as such:

```sh
curl -i -X POST \
-H "Authorization:Bearer public-d613be4e-di03-4b02-9058-70aa4j04ff28" \
-H "Content-Type:application/json" \
-d \
'{
"events": [
{
"userId": "550e8400-e29b-41d4-a716-446655440000",
"type": "track",
"event": "ConnectedAccount",
"properties": {
"totalAccounts": 4,
"country": "Denmark"
},
}]
}' \
'http://localhost:4000/events'
```

3. **Querying events**:
You can query events using the `/events` endpoint (see [API reference](https://docs.trench.dev/api-reference/events-get) for more details).

You can also query events directly from your local Trench server. For example, to query events of type `ConnectedAccount`, you can use the following URL:

```sh
curl -i -X GET \
-H "Authorization: Bearer private-d613be4e-di03-4b02-9058-70aa4j04ff28" \
'http://localhost:4000/events?event=ConnectedAccount'
```

This will return a JSON response with the event that was just sent:

```json
{
"results": [
{
"uuid": "25f7c712-dd86-4db0-89a8-d07d11b73e57",
"type": "track",
"event": "ConnectedAccount",
"userId": "550e8400-e29b-41d4-a716-446655440000",
"properties": {
"totalAccounts": 4,
"country": "Denmark"
},
"timestamp": "2024-10-22T19:34:56.000Z",
"parsedAt": "2024-10-22T19:34:59.530Z"
}
],
"limit": 1000,
"offset": 0,
"total": 1
}
```

4. **Execute raw SQL queries**:
Use the queries endpoint to analyze your data. Example:

```sh
curl -i -X POST \
-H "Authorization:Bearer public-d613be4e-di03-4b02-9058-70aa4j04ff28" \
-H "Content-Type:application/json" \
-d \
'{
"query": "SELECT COUNT(*) FROM events WHERE userId = '550e8400-e29b-41d4-a716-446655440000'"
}' \
'http://localhost:4000/queries'
```

Sample query result:

```json
{
"count": 5
}
```

### 2. Trench Cloud ☁️

If you don't want to selfhost, you can get started with Trench in a few minutes using our [Cloud Quickstart Guide](https://docs.trench.dev/cloud-quickstart).

## 🔗 Links

- [Website](https://trench.dev?utm_campaign=github-readme)
- [Documentation](https://docs.trench.dev/)
- [Slack community](https://join.slack.com/t/trench-community/shared_invite/zt-2sjet5kh2-v31As3yC_zRIadk_AGn~3A)

## 📚 Authors

Trench is a project built by [Frigade](https://frigade.com).

## 📄 License

MIT License