https://github.com/melosso/reef
A lightweight database export service, ready for your destination. Simple. Fast. Reliable.
https://github.com/melosso/reef
automation bpa european foss low-code windows-server workflow
Last synced: 5 months ago
JSON representation
A lightweight database export service, ready for your destination. Simple. Fast. Reliable.
- Host: GitHub
- URL: https://github.com/melosso/reef
- Owner: melosso
- License: agpl-3.0
- Created: 2025-10-30T14:57:22.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-12-26T09:01:59.000Z (6 months ago)
- Last Synced: 2025-12-27T02:48:20.263Z (6 months ago)
- Topics: automation, bpa, european, foss, low-code, windows-server, workflow
- Language: C#
- Homepage:
- Size: 5.64 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 🌟 Reef
[](LICENSE)
[](https://github.com/melosso/reef/commits/main)
[](https://github.com/melosso/reef/releases/latest)
This is **Reef**. It's a web-based integration platform that orchestrates your data workflows. It lets you run queries, transform results, and deliver them to multiple destinations, all from a single easy-to-use interface. Manage connections, profiles, and scheduled executions centrally. All low code, no scripts unless you need them.
## What is Reef?
Reef automates data exports for **reporting workflows**, **integration pipelines**, and **data synchronization** with customizable data structures. Keep it low-code, set-up your own custom templates if necessary, and export data effortlessly.
**Some of the key capabilities:**
- **Web-based interface**: Manage everything through the browser
- **Connection management**: PostgreSQL MySQL or SQL Server. Store and reuse database connections across profiles
- **Multiple formats**: JSON, XML, CSV, YAML with optional [Scriban](https://github.com/scriban/scriban) templates
- **Document generation**: Generate paginated PDF and DOCX documents (such as invoices, reports, picklists)
- **Flexible destinations**: Local, FTP/SFTP, AWS S3, Azure Blob, HTTP, SMB, SMTP
- **Job scheduling**: Cron expressions, intervals, or webhook triggers
- **Execution history**: Track all runs with detailed logging
- **Security first**: Encrypted credentials, record validation and JWT's
In other words, Reef can assist you in quickly getting data synchronisation going. Use your native database languages, or extend your results with (optional) advanced templating.
---
## Getting Started
We've prepared two methods to deploy Reef. It's up to you to choose your preferred method:
### Docker Compose (Recommended)
```yaml
services:
reef:
container_name: reef
image: ghcr.io/melosso/reef:latest
ports:
- "8085:8085"
volumes:
- reef_core:/app/.core
- reef_logs:/app/log
- reef_db:/app/data
- ./exports:/app/exports
environment:
- REEF_ENCRYPTION_KEY=YourKeyHere
- Reef__DatabasePath=/app/data/Reef.db
volumes:
reef_core:
reef_logs:
reef_db:
```
```bash
mkdir -p exports && docker compose up -d
```
Access at **http://localhost:8085**
Upon starting the first time, you can login with the default credentials `admin` / `admin123`. After you log in, follow the steps to change your password right away.
### Windows Installation
Download the latest release from Releases.
1. **Install .NET 9 Runtime:**
```powershell
winget install --id Microsoft.DotNet.Runtime.9 -e
```
2. **Set encryption key:**
```powershell
$bytes = New-Object byte[] 48; [Security.Cryptography.RandomNumberGenerator]::Create().GetBytes($bytes); [Environment]::SetEnvironmentVariable("REEF_ENCRYPTION_KEY", [Convert]::ToBase64String($bytes), "Machine")
```
3. **Install service:**
```powershell
.\Reef.bat install
.\Reef.bat start
```
4. Open browser → **http://localhost:8085**
As mentioned hereabove, upon starting the first time, you can login with the default credentials `admin` / `admin123`. You will be prompted to change them immediately.
---
## How It Works
You can centralize your database credentials easily; create one connection for many profiles. You can tag these profiles by assigning a `Group` to them. Then, create an export definition by creating a `Profile` that'll be assigned a destination:
#### Multiple Destinations
- **Local filesystem**: with date/profile variables
- **FTP/SFTP**: with SSL and passive mode
- **Cloud storage**: AWS S3, Azure Blob
- **HTTP endpoints**: POST to REST APIs
- **SMB shares**: Windows network drives
- **E-mail**: various SMTP providers supported
#### Custom Templates
Use Scriban for advanced transformations. If you're interested in more examples, make sure to checkout our `Examples/` folder.
```scriban
{
"export_date": "{{ date.now }}",
"records": [
{{~ for row in rows ~}}
{ "id": {{ row.ID }}, "name": "{{ row.Name }}" }
{{~ end ~}}
]
}
```
#### Document Generation (PDF, DOCX)
Reef includes built-in document generation capabilities for creating professional PDF and DOCX documents directly from your query results:
**Features:**
- **PDF & DOCX Support**: Generate paginated documents with automatic page numbering
- **Multi-page Documents**: Headers and footers repeat on every page automatically
- **Scriban Data Binding**: Full template syntax support within documents
- **Flexible Layouts**: Control page size (A4/Letter/Legal), orientation, margins
- **Document Options**: Watermarks, custom page numbering formats
**Example Invoice Template:**
```liquid
{{! format: pdf }}
{{! pageSize: A4 }}
{{! orientation: Portrait }}
{{# header }}
{{ .[0].company_name }}
{{/ header }}
{{# content }}
Invoice {{ .[0].invoice_number }}
Date: {{ .[0].invoice_date }}
Customer: {{ .[0].customer_name }}
DescriptionQtyPriceTotal
{{~ for line in . ~}}
{{ line.item_description }}
{{ line.quantity }}
{{ line.unit_price }}
{{ line.line_total }}
{{~ end ~}}
Total: ${{ .[0].total_amount }}
{{/ content }}
{{# footer }}
Thank you for your business
{{/ footer }}
```
#### Job Scheduling
After creating a job, you can schedule it using various methods such as:
- **Cron:** `0 2 * * *` (daily at 02:00)
- **Interval:** Every 15 minutes
- **Webhooks:** Trigger via HTTP POST
## Security
We've made sure to keep your data safe. Here's a glimpse on how we do this:
- **Encrypted credentials**: Sensitive data is stored using RSA+AES hybrid encryption
- **Transient execution**: Exchanged data exists only in memory while being processed, nothing persists *
- **Hash validation**: Detect configuration tampering
- **Audit logging**: Every change is tracked for accountability
Remark: * Outgoing mail temporarily persist recipient information, until approved by the (when using the email approval option).
> [!IMPORTANT]
> Since this application is built for local data orchistration, make sure to **never expose** Reef outside of your network. If you need to reach it from outside, expose only the `/api` routes and run them through Nginx or whatever reverse proxy you use. There's an IIS example in [web.config.md](web.config.md) if you need a starting point, though you'l likely have to tweak it for your setup.
## Lore
> Reef comes from the idea of a coral reef, something that grows slowly from countless small bits until it becomes its own world. Database exports feel the same way. Each customer, each dump, each migration, another layer added to the structure. It's alive in its own way, even if it's just data.
## License
Free for open source projects and personal use under the **AGPL 3.0** license. For more information, please see the [license](LICENSE) file.
## Contributing
Contributions welcome! Please submit issues and pull requests, using the templates we provided.