{"id":13494539,"url":"https://github.com/daptin/daptin","last_synced_at":"2026-05-12T07:01:57.476Z","repository":{"id":25932138,"uuid":"92829622","full_name":"daptin/daptin","owner":"daptin","description":"Daptin - Backend As A Service - GraphQL/JSON-API Headless CMS","archived":false,"fork":false,"pushed_at":"2025-03-31T13:33:16.000Z","size":143291,"stargazers_count":1857,"open_issues_count":20,"forks_count":112,"subscribers_count":46,"default_branch":"master","last_synced_at":"2025-04-03T18:59:40.925Z","etag":null,"topics":["cloud-storage","content-management","crud-apis","graphql","graphql-api","graphql-server","headless-cms","json-api","static-site"],"latest_commit_sha":null,"homepage":"https://daptin.github.io/daptin/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daptin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"COPYING.LESSER","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2017-05-30T12:16:00.000Z","updated_at":"2025-04-01T13:26:25.000Z","dependencies_parsed_at":"2023-07-13T04:21:07.886Z","dependency_job_id":"88512fe8-2372-4813-82fa-ea6c899e7b0c","html_url":"https://github.com/daptin/daptin","commit_stats":{"total_commits":2699,"total_committers":17,"mean_commits":"158.76470588235293","dds":0.1689514635050019,"last_synced_commit":"be7edc83ebf8dc1f2fd36e8092b2c8f5a8cf3eee"},"previous_names":["artpar/goms"],"tags_count":55,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daptin%2Fdaptin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daptin%2Fdaptin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daptin%2Fdaptin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daptin%2Fdaptin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daptin","download_url":"https://codeload.github.com/daptin/daptin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248334950,"owners_count":21086477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-storage","content-management","crud-apis","graphql","graphql-api","graphql-server","headless-cms","json-api","static-site"],"created_at":"2024-07-31T19:01:25.877Z","updated_at":"2026-05-12T07:01:57.469Z","avatar_url":"https://github.com/daptin.png","language":"Go","funding_links":[],"categories":["Go","graphql","headless-cms"],"sub_categories":[],"readme":"\n\u003ch1 align=\"left\"\u003e\n  daptin\n  \u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://travis-ci.org/daptin/daptin\"\u003e\u003cimg alt=\"Travis\" src=\"https://img.shields.io/travis/daptin/daptin.svg?style=flat-square\"\u003e\u003c/a\u003e\n    \u003ca href='https://semaphoreci.com/artpar/daptin'\u003e \u003cimg src='https://semaphoreci.com/api/v1/artpar/daptin/branches/master/badge.svg' alt='Build Status'\u003e\u003c/a\u003e\n    \u003ca href='https://circleci.com/gh/daptin/daptin'\u003e \u003cimg src='https://circleci.com/gh/daptin/daptin.svg?style=svg' alt='Build Status'\u003e\u003c/a\u003e\t\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"/LICENSE\"\u003e\u003cimg alt=\"Software License\" src=\"https://img.shields.io/badge/LICENSE-LGPL%20v3-brightgreen.svg?style=flat-square\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://goreportcard.com/report/github.com/daptin/daptin\"\u003e\u003cimg alt=\"Go Report Card\" src=\"https://goreportcard.com/badge/github.com/daptin/daptin?style=flat-square\"\u003e\u003c/a\u003e\n    \u003ca href=\"http://godoc.org/github.com/daptin/daptin\"\u003e\u003cimg alt=\"Go Doc\" src=\"https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://codecov.io/gh/daptin/daptin\"\u003e\u003cimg alt=\"Codecov branch\" src=\"https://img.shields.io/codecov/c/github/daptin/daptin/master.svg?style=flat-square\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/daptin/daptin/releases/latest\"\u003e\u003cimg alt=\"Release\" src=\"https://img.shields.io/github/release/daptin/daptin.svg?style=flat-square\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n     \u003ca href=\"https://discord.gg/t564q8SQVk\"\u003e\u003cimg src=\"https://img.shields.io/badge/JOIN-ON%20DISCORD-blue\u0026?style=for-the-badge\u0026logo=discord\"\u003e\u003c/a\u003e \n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n\t\u003ca href=\"https://github.com/daptin/daptin/releases\"\u003eDownload\u003c/a\u003e •\n\t\u003ca href=\"https://daptin.github.io/daptin/\"\u003eDocumentation\u003c/a\u003e •\n\t\u003ca href=\"https://join.slack.com/t/daptin/shared_invite/enQtMzM1NTM1NTkyMDgzLTVlYzBlMmM4YjMyOTk0MDc5MWJmMzFlMTliNzQwYjcxMzc5Mjk0YzEyZDIwYTljZmE5NDU3Yjk3YzQ3MzhkMzI\"\u003eCommunity\u003c/a\u003e\n\u003c/p\u003e\n\n\n```bash\n./daptin\n.\n. // logs truncated for brevity\n.\nINFO[2024-10-16 11:08:58] Listening websocket server at ... /live\nINFO[2024-10-16 11:08:58] Our admin is [artpar@gmail.com]\nINFO[2024-10-16 11:08:58] [ProcessId=86403] Listening at port: :6336\nINFO[2024-10-16 11:08:58] Get certificate for [Parths-MacBook-Pro.local]: true\nINFO[2024-10-16 11:08:58] Listening at: [:6336]\nINFO[2024-10-16 11:08:58] TLS server listening on port :6443\nINFO[2024-10-16 11:09:03] Member says: Message\u003cmembers: Joining from 192.168.0.125:5336\u003e\n```\n\nServer is up, sqlite database is used since we did not specify mysql or postgres.\n\n\n### signup, signin, user_account and usergroup\n\n## signup \n\ncall the signup \"action\" api to create a new user_account\n\n```bash\ncurl 'http://localhost:6333/action/user_account/signup' -X POST \\\n--data-raw '{\"attributes\":{\"email\":\"artpar@gmail.com\",\"password\":\"artpar@gmail.com\",\"name\":\"artpar@gmail.com\",\"passwordConfirm\":\"artpar@gmail.com\"}}'\n```\n\nOn a fresh instance all actions are allowed to be executed by guests, so you shouldn't see this\n\n```json\n[\n    {\n        \"Attributes\": {\n            \"message\": \"http error (403) forbidden and 0 more errors, forbidden\",\n            \"title\": \"failed\",\n            \"type\": \"error\"\n        },\n        \"ResponseType\": \"client.notify\"\n    }\n]\n```\n\nYou should see this\n\n```json\n[\n  {\n    \"ResponseType\": \"client.notify\",\n    \"Attributes\": {\n      \"__type\": \"client.notify\",\n      \"message\": \"Sign-up successful. Redirecting to sign in\",\n      \"title\": \"Success\",\n      \"type\": \"success\"\n    }\n  },\n  {\n    \"ResponseType\": \"client.redirect\",\n    \"Attributes\": {\n      \"__type\": \"client.redirect\",\n      \"delay\": 2000,\n      \"location\": \"/auth/signin\",\n      \"window\": \"self\"\n    }\n  }\n]\n```\n\n#### Sign in to get a JWT Bearer token\n\n```bash\ncurl 'http://localhost:6336/action/user_account/signin' \\\n--data-raw '{\"attributes\":{\"email\":\"artpar@gmail.com\",\"password\":\"artpar@gmail.com\"}}'\n\n[\n    {\n        \"Attributes\": {\n            \"key\": \"token\",\n            \"value\": \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFydHBhckBnbWFpbC5jb20iLCJleHAiOjE3MjkzMjExMjIsImlhdCI6MTcyOTA2MTkyMiwiaXNzIjoiZGFwdGluLTAxOTIyOCIsImp0aSI6IjAxOTI5NDFmLTI2MGUtN2I0Ni1hMWFlLWYxMGZhZTcwMDE3OSIsIm5hbWUiOiJhcnRwYXJAZ21haWwuY29tIiwibmJmIjoxNzI5MDYxOTIyLCJzdWIiOiIwMTkyMmUxYS1kNWVhLTcxYzktYmQzZS02MTZkMjM3ODBmOTMifQ.H-GLmXCT-o7RxXrjo5Of0K8Nw5mpOOw6jgoXnd5KUxo\"\n        },\n        \"ResponseType\": \"client.store.set\"\n    },\n    {\n        \"Attributes\": {\n            \"key\": \"token\",\n            \"value\": \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFydHBhckBnbWFpbC5jb20iLCJleHAiOjE3MjkzMjExMjIsImlhdCI6MTcyOTA2MTkyMiwiaXNzIjoiZGFwdGluLTAxOTIyOCIsImp0aSI6IjAxOTI5NDFmLTI2MGUtN2I0Ni1hMWFlLWYxMGZhZTcwMDE3OSIsIm5hbWUiOiJhcnRwYXJAZ21haWwuY29tIiwibmJmIjoxNzI5MDYxOTIyLCJzdWIiOiIwMTkyMmUxYS1kNWVhLTcxYzktYmQzZS02MTZkMjM3ODBmOTMifQ.H-GLmXCT-o7RxXrjo5Of0K8Nw5mpOOw6jgoXnd5KUxo; SameSite=Strict\"\n        },\n        \"ResponseType\": \"client.cookie.set\"\n    },\n    {\n        \"Attributes\": {\n            \"message\": \"Logged in\",\n            \"title\": \"Success\",\n            \"type\": \"success\"\n        },\n        \"ResponseType\": \"client.notify\"\n    },\n    {\n        \"Attributes\": {\n            \"delay\": 2000,\n            \"location\": \"/\",\n            \"window\": \"self\"\n        },\n        \"ResponseType\": \"client.redirect\"\n    }\n]\n\n```\n\nWe will use\n\n```bash\nexport TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFydHBhckBnbWFpbC5jb20iLCJleHAiOjE3MjkzMjExMjIsImlhdCI6MTcyOTA2MTkyMiwiaXNzIjoiZGFwdGluLTAxOTIyOCIsImp0aSI6IjAxOTI5NDFmLTI2MGUtN2I0Ni1hMWFlLWYxMGZhZTcwMDE3OSIsIm5hbWUiOiJhcnRwYXJAZ21haWwuY29tIiwibmJmIjoxNzI5MDYxOTIyLCJzdWIiOiIwMTkyMmUxYS1kNWVhLTcxYzktYmQzZS02MTZkMjM3ODBmOTMifQ.H-GLmXCT-o7RxXrjo5Of0K8Nw5mpOOw6jgoXnd5KUxo \n```\n\nfor the rest of the api calls. This is a JWT token with following data\n\n```json\n{\n  \"email\": \"artpar@gmail.com\",                    // user email\n  \"exp\": 1729321122,                              // token expiry\n  \"iat\": 1729061922,                              // token issued at time\n  \"iss\": \"daptin-019228\",                         // token issuer (your daptin instance)\n  \"jti\": \"0192941f-260e-7b46-a1ae-f10fae700179\",  // unique identifier for this token\n  \"name\": \"artpar@gmail.com\",                     // user name\n  \"nbf\": 1729061922,                              // token valid not before timestamp\n  \"sub\": \"01922e1a-d5ea-71c9-bd3e-616d23780f93\"   // user reference id\n}\n```\n\n\n---\n\nSo you have an account and a token to authenticate as that account. But do you need it? No. \nCall to fetch all user accounts works without any authorization\n\n```bash\ncurl http://localhost:6333/api/user_account\n```\n\n```json\n{\n  \"links\": {\n    \"current_page\": 1,\n    \"from\": 0,\n    \"last_page\": 1,\n    \"per_page\": 10,\n    \"to\": 10,\n    \"total\": 1\n  },\n  \"data\": [\n    {\n      \"type\": \"user_account\",\n      \"id\": \"01929429-3d8f-7e53-8f15-a663e05fb01b\",\n      \"attributes\": {\n        \"__type\": \"user_account\",\n        \"confirmed\": 0,\n        \"created_at\": \"2024-10-16T07:09:43.86360642Z\",\n        \"email\": \"artpar1@gmail.com\",\n        \"name\": \"artpar1@gmail.com\",\n        \"password\": \"\",\n        \"permission\": 2097151,\n        \"reference_id\": \"01929429-3d8f-7e53-8f15-a663e05fb01b\",\n        \"updated_at\": \"2024-10-16T07:09:43.863622045Z\",\n        \"user_account_id\": \"01929429-3d8f-7e53-8f15-a663e05fb01b\"\n      },\n      \"relationships\": { /// ...}\n    }\n  ]\n}\n```\n\nAnd so does all the data in all other tables (eg site, cloud_store, document, usergroup). \nAnd you can call update and delete APIs as well \n(not demonstrated here, but you can try, delete the sqlite database file after you are done playing to reset it all)\n\n\nAs the first user, it is an option for you to leave it open or enable the multi-tier permission and becoming the Administrator\n\n```bash\ncurl 'http://localhost:6336/action/world/become_an_administrator' --compressed -X POST \\\n-H \"Authorization:  Bearer $TOKEN\" --data-raw '{}'\n```\n\nAt this point, all other apis are locked-down and only accessible by administrator, that is you. \nYou want to open up few or many of actions to guests or users.\n\n\n... Will be updated soon\n\n## 📊 Self-Documentation Progress (Multi-Session Project)\n\n**Overall Progress: 71% Complete (37/52 features documented)**\n\n### Session Tracking:\n- **Session 1**: Foundation (7 features) - Configuration, Statistics, Meta, Health, JS Models, Aggregation ✅\n- **Session 2**: Real-time \u0026 Communication (12 features) - WebSocket, YJS, SMTP, CalDAV, FTP, Feeds ✅\n- **Session 3**: Advanced Data \u0026 Analytics (8 features) - Aggregation, GraphQL, Import/Export, Relationships ✅\n- **Session 4**: Infrastructure \u0026 Configuration (10 features) - Config API, Rate Limiting, GZIP, Caching, CORS, TLS ✅\n- **Session 5**: Workflow \u0026 Automation (Planned)\n- **Session 6**: Client Integration \u0026 Developer Experience (Planned)\n- **Session 7**: Final Documentation \u0026 Polish (Planned)\n\n**Documentation Artifacts:**\n- `/openapi.yaml` - Self-updating API documentation\n- `SELF_DOCUMENTATION_MASTER_PLAN.md` - Complete roadmap\n- `SESSION_HANDOFF.md` - Progress tracking\n- `NEXT_SESSION_PROMPT.md` - Next session guide\n\n## Overview\n\n\n- [Database backed](https://daptin.github.io/daptin/setting-up/installation/#database-configuration) persistence, 3NF normalized tables\n- [JSON API](https://daptin.github.io/daptin/apis/overview/)/[GraphQL](https://daptin.github.io/daptin/features/enable-graphql/) for CRUD apis\n- [User](https://daptin.github.io/daptin/setting-up/access/) and [group management](https://daptin.github.io/daptin/setting-up/access/) and access control\n- Social login with [OAuth](https://daptin.github.io/daptin/extend/oauth_connection/): tested with google, github, linkedin\n- [Actions](https://daptin.github.io/daptin/actions/actions/) for abstracting out business flows\n- Extensive [state tracking APIs](https://daptin.github.io/daptin/state/machines/)\n- Enable [Data Auditing](https://daptin.github.io/daptin/features/enable-data-auditing/) from a single toggle\n- [Synchronous Data Exchange](https://daptin.github.io/daptin/extend/data_exchange/) with 3rd party APIs\n- [Multilingual tables](https://daptin.github.io/daptin/features/enable-multilingual-table/) support, supports Accept-Language header \n- [Cloud storage sync](https://daptin.github.io/daptin/cloudstore/cloudstore/) like gdrive, dropbox, b2, s3 and more\n- [Asset column](https://daptin.github.io/daptin/cloudstore/assetcolumns/) to hold file and blob data, backed by storage\n- [Multiple websites](https://daptin.github.io/daptin/subsite/subsite/) under separate sub-domain/sub-paths\n- [Connect with external APIs](https://daptin.github.io/daptin/integrations/overview/) by using extension points\n- [Data View Streams](https://daptin.github.io/daptin/streams/streams/)\n- Flexible [data import](https://daptin.github.io/daptin/setting-up/data_import/) (auto create new tables and automated schema generation)\n    - XLSX \n    - JSON\n    - CSV\n\nJavascript/Typescript Client\n===\n\nhttps://github.com/daptin/daptin-js-client\n\nStarter kit: https://github.com/daptin/vue_typescript_starter_kit\n\n\nDefine Schema\n\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/api.jpg\"\u003e\n\nFind\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/apigetall.png\"\u003e\n\nGet By Id\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/apigetbyid.png\"\u003e\n\nCreate\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/apicreate.png\"\u003e\n\nDelete\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/apidelete.png\"\u003e\n\nDelete relations\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/apideleterelated.png\"\u003e\n\nList relations\n\u003cimg src=\"https://github.com/daptin/daptin/raw/master/images/apifetchrelated.png\"\u003e\n\n\n\n* Versioning of the data\n* Authentication and authorization\n* JSON API endpoint\n* GraphQL endpoint\n* Actions and integrations with external services\n\n\nGet Started\n---\n\n* [Native binary](https://daptin.github.io/daptin/setting-up/installation/#native-binary)\n* [Heroku](https://daptin.github.io/daptin/setting-up/installation/#heroku-deployment)\n* [Docker image](https://daptin.github.io/daptin/setting-up/installation/#docker-image)\n* [Kubernetes YAML](https://daptin.github.io/daptin/setting-up/installation/#kubernetes-deployment)\n\n\n### APIs\n\n- CRUD APIs\n    - [Read, search, filter](https://daptin.github.io/daptin/apis/read)\n    - [Create](https://daptin.github.io/daptin/apis/create)\n    - [Update](https://daptin.github.io/daptin/apis/update)\n    - [Delete](https://daptin.github.io/daptin/apis/delete)\n    - [Relations](https://daptin.github.io/daptin/apis/relation)\n    - [Execute](https://daptin.github.io/daptin/apis/execute)\n- Action APIs\n    - [Using actions](https://daptin.github.io/daptin/actions/actions)\n    - [Actions list](https://daptin.github.io/daptin/actions/default_actions)\n- User APIs\n    - [User registration/signup](https://daptin.github.io/daptin/actions/signup)\n    - [User login/signin](https://daptin.github.io/daptin/actions/signin)\n- State tracking APIs\n    - [State machines](https://daptin.github.io/daptin/state/machines)\n\n### Users\n\n- [Guests](https://daptin.github.io/daptin/setting-up/access/#guests)\n- [Adding users](https://daptin.github.io/daptin/setting-up/access/#signup-api)\n- [User groups](https://daptin.github.io/daptin/setting-up/access/#user-groups)\n- [Data access permission](https://daptin.github.io/daptin/setting-up/access/#authorization)\n- [Social login](https://daptin.github.io/daptin/setting-up/access/#social-login)\n\n\n### Asset and file storage\n\n- [Cloud storage](https://daptin.github.io/daptin/cloudstore/cloudstore)\n\n### Sub-sites\n\n- [Create a subsite](https://daptin.github.io/daptin/subsite/subsite)\n\n\n## CLI\n\n[daptin-cli](https://github.com/daptin/daptin-cli) is the official command-line client for Daptin.\n\n```bash\n# Install\nbrew install daptin/tap/daptin-cli\n\n# Connect\ndaptin-cli context add local http://localhost:6336\n\n# CRUD\ndaptin-cli list --columns table_name,is_top_level world\ndaptin-cli get world \u003creference_id\u003e\ndaptin-cli create document document_name=report.pdf\n\n# All actions (signin, signup, upload, export, etc.) via a single command\ndaptin-cli execute user_account signin email=admin@example.com password=secret\ndaptin-cli execute world export_data table_name=document format=json\n\n# Describe schemas\ndaptin-cli describe table document\ndaptin-cli describe action document createDocument\n```\n\nAvailable via [Homebrew](https://github.com/daptin/homebrew-tap), [Scoop](https://github.com/daptin/scoop-bucket), deb, rpm, and [binary downloads](https://github.com/daptin/daptin-cli/releases/latest).\n\n## Client library\n\n|                                                                                |                                                                        |                                                                                |\n| ------------------------------------------------------------------------------ | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------ |\n| [Ruby](http://jsonapi.org/implementations/#client-libraries-ruby)              | [Python](http://jsonapi.org/implementations/#client-libraries-python)  | [Javascript](http://jsonapi.org/implementations/#client-libraries-javascript)  |\n| [Typescript](http://jsonapi.org/implementations/#client-libraries-typescript)  | [PHP](http://jsonapi.org/implementations/#client-libraries-php)        | [Dart](http://jsonapi.org/implementations/#client-libraries-dart)              |\n| [.NET](http://jsonapi.org/implementations/#client-libraries-net)               | [Java](http://jsonapi.org/implementations/#client-libraries-java)      | [iOS](http://jsonapi.org/implementations/#client-libraries-ios)                |\n| [Elixir](http://jsonapi.org/implementations/#client-libraries-elixir)          | [R](http://jsonapi.org/implementations/#client-libraries-r)             | [Perl](http://jsonapi.org/implementations/#client-libraries-perl)               |\n\n## API spec RAML\n\nOpenAPI V3 spec is auto generated for each endpoint exposed. This can be use to generate further documentation and clients.\n\n![YAML API documentation](docs_markdown/docs/images/api-documentation.png)\n\n```curl http://localhost/apispec.yaml```\n\n\n## Road Map\n\n\n* [x] Normalised Db Design from JSON schema upload\n* [x] Json Api, with CRUD and Relationships\n* [x] OAuth Authentication, inbuilt jwt token generator (setups up secret itself)\n* [x] Authorization based on a slightly modified linux FS permission model\n* [x] Objects and action chains\n* [x] State tracking using state machine\n* [x] Data connectors -\u003e Incoming/Outgoing data\n* [x] Plugin system -\u003e Grow the system according to your needs\n* [x] Native support for different data types (geo location/time/colors/measurements)\n* [x] Configurable intelligent Validation for data in the APIs\n* [x] Pages/Sub-sites -\u003e Create a sub-site for a target audience\n* [x] Define events all around the system\n* [x] Data conversion/exchange/transformations\n* [x] Store connectors for storing big files/subsites - rclone\n* [x] Market place to allow plugins/extensions to be installed\n* [x] Online entity designer\n* [x] Excel to entity identification\n* [x] CSV to entity identification\n\n## Self-Discoverability and Self-Management Analysis\n\nBased on comprehensive testing of a fresh Daptin instance, here are the key findings:\n\n### Self-Discoverability Score: 9/10\n\nDaptin excels at self-discoverability through:\n\n- **Comprehensive OpenAPI Documentation** at `/openapi.yaml` with detailed endpoint descriptions, parameters, and examples\n- **Meta-Endpoints** for runtime discovery:\n  - `/api/world` - Lists all 56 available entities\n  - `/api/action` - Shows available actions per entity\n  - `/action/world/download_system_schema` - Exports complete system configuration\n- **JSON:API Compliance** with consistent CRUD patterns\n- **Clear Authentication Flow** with public signup/signin endpoints\n\n### Self-Management Score: 7/10\n\nDaptin provides good self-management capabilities:\n\n**Strengths:**\n- ✅ Dynamic entity creation via API\n- ✅ Programmatic server restart (`/action/world/restart_daptin`)\n- ✅ Multi-admin support via usergroups\n- ✅ Schema export/import functionality\n- ✅ Multiple data format exports (JSON, CSV, XML, PDF)\n\n**Limitations:**\n- ❌ Some actions restricted even for admins (generate_random_data, get_action_schema)\n- ❌ Schema changes require server restart\n- ❌ No built-in admin UI\n\n### Quick Reference for New Users\n\n#### Authentication Flow\n```bash\n# 1. Create user (8+ character password required)\ncurl -X POST http://localhost:6336/action/user_account/signup \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"attributes\": {\"email\": \"admin@test.com\", \"password\": \"testpass123\"}}'\n\n# 2. Get JWT token\nTOKEN=$(curl -X POST http://localhost:6336/action/user_account/signin \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"attributes\": {\"email\": \"admin@test.com\", \"password\": \"testpass123\"}}' \\\n  | jq -r '.[0].Attributes.value')\n\n# 3. Become admin (ONE-TIME ONLY!)\ncurl -X POST http://localhost:6336/action/world/become_an_administrator \\\n  -H \"Authorization: Bearer $TOKEN\"\n```\n\n#### Common Pitfalls\n- Empty API responses? Check Authorization header\n- Password errors? Use 8+ characters\n- Schema not updated? Restart server after changes\n- 403 errors? Verify token is valid and included\n\n### Key Insights\n\n1. **Unique Security Model**: Before admin setup, ALL users have full access (permission: 2097151)\n2. **Multi-Admin Support**: Add users to \"administrators\" usergroup for admin access\n3. **Token Management**: JWT tokens valid for 3 days, always include `Authorization: Bearer $TOKEN`\n4. **Column Types**: Extensive type system with validations (see `/server/resource/column_types.go`)\n\n## 🔄 Real-time \u0026 Communication Features (Session 2 Deep Dive - 37% Complete)\n\n### ✅ WebSocket Real-time (SOLUTION FOUND)\nWebSocket authentication works via **query parameter**, not headers:\n\n```bash\n# WORKING WebSocket Connection\ncurl --include \\\n  --no-buffer \\\n  --header \"Connection: Upgrade\" \\\n  --header \"Upgrade: websocket\" \\\n  --header \"Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==\" \\\n  --header \"Sec-WebSocket-Version: 13\" \\\n  \"ws://localhost:6336/live?token=$TOKEN\"\n```\n\n**WebSocket Features Discovered:**\n- **Pub/Sub Messaging**: Subscribe to database events and custom topics\n- **Permission-Aware**: Events filtered based on user permissions\n- **Distributed**: Uses Olric for cluster-wide messaging\n- **Auto Topics**: One system topic per database table\n\n**WebSocket Message Methods:**\n```javascript\n// Subscribe to table events\n{\"method\": \"subscribe\", \"attributes\": {\"topicName\": \"user_account,document\"}}\n\n// Create custom topic\n{\"method\": \"create-topicName\", \"attributes\": {\"name\": \"chat-room-1\"}}\n\n// Publish message\n{\"method\": \"new-message\", \"attributes\": {\"topicName\": \"chat-room-1\", \"message\": \"Hello!\"}}\n\n// List all topics\n{\"method\": \"list-topicName\", \"attributes\": {}}\n```\n\n### ✅ YJS Collaborative Editing (Fully Mapped)\nReal-time document collaboration with conflict resolution:\n\n**YJS Endpoints Pattern:**\n- `/live/{typename}/{referenceId}/{columnName}/yjs` - WebSocket collaboration\n- `/yjs/{documentName}` - Direct YJS access\n- Any `file.*` column type gets automatic YJS endpoints\n\n**Working Example from dadadash:**\n```javascript\nconst yjsProvider = new WebsocketProvider(\n  `ws://localhost:6336/live/document/${referenceId}/content/yjs?token=${token}`,\n  'document-room',\n  ydoc,\n  {\n    awareness: {\n      user: { name: 'User Name', color: '#ff0000' }\n    }\n  }\n);\n```\n\n### ✅ SMTP Email Server (Complete Implementation)\nBuilt on go-guerrilla with enterprise features:\n\n**Email Infrastructure:**\n- **Entities**: mail_server, mail_account, mail_box, mail\n- **Security**: TLS/SSL, DKIM signing, SPF verification\n- **Actions**: mail.send, aws.mail.send\n- **IMAP**: Full email retrieval support\n\n**Complete Email Setup:**\n```bash\n# 1. Enable SMTP\ncurl -X PUT http://localhost:6336/_config/backend/smtp.enable \\\n  -H \"Authorization: Bearer $TOKEN\" -d '\"true\"'\n\n# 2. Create mail server\ncurl -X POST http://localhost:6336/api/mail_server \\\n  -H \"Authorization: Bearer $TOKEN\" \\\n  -H \"Content-Type: application/vnd.api+json\" \\\n  -d '{\n    \"data\": {\n      \"type\": \"mail_server\",\n      \"attributes\": {\n        \"hostname\": \"smtp.yourdomain.com\",\n        \"is_enabled\": true,\n        \"listen_interface\": \"0.0.0.0:465\",\n        \"always_on_tls\": true\n      }\n    }\n  }'\n\n# 3. Send email\ncurl -X POST http://localhost:6336/action/world/mail.send \\\n  -H \"Authorization: Bearer $TOKEN\" \\\n  -d '{\"attributes\": {\"from\": \"noreply@yourdomain.com\", \"to\": [\"user@example.com\"], \"subject\": \"Test\", \"body\": \"Hello!\"}}'\n```\n\n### ✅ Communication Protocols (All Verified)\n- **CalDAV/CardDAV**: `/caldav/*` and `/carddav/*` endpoints\n- **FTP Server**: Multi-site support with cloud storage backends\n- **RSS/Atom Feeds**: Auto-generation from any entity\n\n### 📊 Session 2 Feature Count: 19/52 (37%)\n\n**Documented Features:**\n1. WebSocket server architecture\n2. Pub/Sub messaging patterns  \n3. Permission-aware event filtering\n4. Custom topic management\n5. YJS collaborative editing\n6. YJS document persistence\n7. Multi-editor support (Quill, CodeMirror)\n8. User presence/awareness\n9. SMTP server implementation\n10. Email entity schemas (4 tables)\n11. DKIM/SPF security\n12. IMAP email retrieval\n13. CalDAV calendar sync\n14. CardDAV contact sync\n15. FTP file transfer\n16. RSS feed generation\n17. Atom feed support\n18. JSON feed format\n19. Feed configuration patterns\n\n### 🔧 Key Learnings for Future Sessions\n\n**WebSocket Authentication:**\n- Use query parameter `?token=JWT_TOKEN` not headers\n- Token validation happens during WebSocket upgrade\n- Same JWT tokens from signin work perfectly\n\n**YJS Integration:**\n- Requires file-type columns (file.document, file.spreadsheet, etc.)\n- Documents stored as ZIP with YJS state + plain text\n- Supports real-time presence and conflict resolution\n\n**Server Configuration:**\n- Most features toggle via `/_config` API\n- Some changes need server restart (actions, world schema)\n- Configuration stored in database, persists across restarts\n\n**Testing Approach:**\n- Always verify configuration changes took effect\n- Check multiple related endpoints for full feature validation\n- Use real-world examples (dadadash) for integration patterns\n\n## 📊 Advanced Data \u0026 Analytics Features (Session 3 Deep Dive - 52% Complete)\n\n### ✅ Aggregation API (Fully Tested)\nPowerful data aggregation with SQL-like capabilities via REST:\n\n**Endpoint Pattern:** `/aggregate/{entityName}`\n\n**Working Examples:**\n```bash\n# Group by with count\ncurl -X GET \"http://localhost:6336/aggregate/world?group=is_hidden\u0026column=is_hidden,count\" \\\n  -H \"Authorization: Bearer $TOKEN\"\n\n# Response:\n{\n  \"data\": [\n    {\"type\": \"aggregate_world\", \"attributes\": {\"is_hidden\": 0, \"count\": 60}},\n    {\"type\": \"aggregate_world\", \"attributes\": {\"is_hidden\": 1, \"count\": 1}}\n  ]\n}\n\n# Sum with filter\ncurl -X GET \"http://localhost:6336/aggregate/world?filter=eq(is_top_level,1)\u0026column=count\" \\\n  -H \"Authorization: Bearer $TOKEN\"\n```\n\n**Aggregation Features:**\n- **Functions**: count, sum(col), avg(col), min(col), max(col), first(col), last(col)\n- **Filters**: eq(), not(), lt(), lte(), gt(), gte(), in(), notin(), is(), not()\n- **Advanced**: group by multiple columns, having clauses, joins, time sampling\n- **Methods**: Both GET (query params) and POST (JSON body) supported\n\n### ✅ GraphQL API (Configuration Mapped)\nAuto-generated GraphQL schema from database:\n\n**Enable GraphQL (Requires Restart):**\n```bash\n# Method 1: Configuration API\ncurl -X POST http://localhost:6336/_config/backend/graphql.enable \\\n  -H \"Authorization: Bearer $TOKEN\" -d 'true'\n\n# Method 2: System Action (if available)\ncurl -X POST http://localhost:6336/action/world/__enable_graphql \\\n  -H \"Authorization: Bearer $TOKEN\" -d '{\"attributes\":{}}'\n\n# Restart server\ncurl -X POST http://localhost:6336/action/world/restart_daptin \\\n  -H \"Authorization: Bearer $TOKEN\" -d '{\"attributes\":{}}'\n```\n\n**GraphQL Features:**\n- **Auto Schema**: Generated from all tables and relationships\n- **Operations**: Queries, mutations, subscriptions\n- **Relationships**: Automatic traversal\n- **Actions**: Execute via mutations\n- **Security**: Disabled by default, admin-only enable\n\n### ✅ Import/Export System (Architecture Documented)\nEnterprise-grade data migration with streaming:\n\n**Export Action:**\n```bash\n# Export to CSV\ncurl -X POST \"http://localhost:6336/api/{entity}/action/__data_export\" \\\n  -H \"Authorization: Bearer $TOKEN\" \\\n  -d '{\n    \"attributes\": {\n      \"format\": \"csv\",        # json, csv, xlsx, pdf, html\n      \"table_name\": \"books\",\n      \"include_headers\": true,\n      \"columns\": [\"title\", \"created_at\"],\n      \"page_size\": 1000      # For streaming large datasets\n    }\n  }'\n```\n\n**Import Action:**\n```bash\n# Import from CSV\ncurl -X POST \"http://localhost:6336/api/{entity}/action/__data_import\" \\\n  -H \"Authorization: Bearer $TOKEN\" \\\n  -d '{\n    \"attributes\": {\n      \"table_name\": \"books\",\n      \"batch_size\": 500,\n      \"truncate_before_insert\": false,\n      \"dump_file\": [{\n        \"name\": \"books.csv\",\n        \"file\": \"data:text/csv;base64,...\"\n      }]\n    }\n  }'\n```\n\n**Import/Export Features:**\n- **Formats**: JSON, CSV, XLSX, PDF (export), HTML (export)\n- **Streaming**: Memory-efficient for large datasets\n- **Batch Processing**: Configurable batch sizes\n- **Schema Creation**: CSV/XLSX can create new tables\n- **Base64 Response**: Browser-friendly downloads\n\n### ✅ Relationship Management (Query Patterns Verified)\nJSON:API compliant relationship handling:\n\n**Include Related Data:**\n```bash\n# Get world with related user_account\ncurl -X GET \"http://localhost:6336/api/world/{id}?include=user_account_id\" \\\n  -H \"Authorization: Bearer $TOKEN\"\n```\n\n**Relationship Types:**\n- **belongs_to**: Many-to-one (foreign key on subject)\n- **has_one**: One-to-one relationship\n- **has_many**: One-to-many relationship\n- **many_to_many**: Via join tables (auto-created)\n\n**Features:**\n- Automatic foreign key tracking\n- Cascade operations support\n- Lazy/eager loading via include parameter\n- Join table auto-management\n\n### 📊 Session 3 Feature Count: 27/52 (52%)\n\n**Documented Features:**\n1. Aggregation endpoint patterns\n2. Aggregate function syntax (7 functions)\n3. Filter function syntax (10+ operators)\n4. Group by and having clauses\n5. GraphQL enable process\n6. GraphQL auto-schema generation\n7. Import/Export action system\n8. Streaming architecture patterns\n\n### 🔧 Key Learnings for Future Sessions\n\n**Authentication Requirements:**\n- Aggregation endpoints require valid JWT tokens\n- Admin privileges needed for some features\n- Token in Authorization header: `Bearer $TOKEN`\n\n**Configuration Patterns:**\n- GraphQL disabled by default (security)\n- Enable via `/_config/backend/` namespace\n- Some changes require restart (GraphQL, world schema)\n\n**Data Operations:**\n- Import/Export via actions, not REST endpoints\n- Base64 encoding for file transfers\n- Streaming support for large datasets\n\n**API Consistency:**\n- JSON:API spec for relationships\n- Consistent error responses\n- Pagination on all list endpoints\n\n## 🏗️ Infrastructure \u0026 Configuration Features (Session 4 Deep Dive - 71% Complete)\n\n### ✅ Configuration Management System (Fully Tested)\nDatabase-backed configuration with runtime updates:\n\n**Configuration API Pattern:** `/_config/{configType}/{key}`\n\n**18 Backend Configuration Parameters:**\n```bash\n# Set configuration value\ncurl -X POST http://localhost:6336/_config/backend/graphql.enable \\\n  -H \"Authorization: Bearer $TOKEN\" -d 'false'\n\n# Get configuration value (if set)\ncurl -H \"Authorization: Bearer $TOKEN\" \\\n  http://localhost:6336/_config/backend/graphql.enable\n```\n\n**Documented Parameters:**\n1. **graphql.enable** - Enable/disable GraphQL endpoint\n2. **gzip.enable** - Enable/disable GZIP compression\n3. **limit.rate** - API rate limiting per second\n4. **yjs.enabled** - Enable YJS collaborative editing\n5. **caldav.enable** - Enable CalDAV calendar sync\n6. **ftp.enable** - Enable FTP server\n7. **ftp.listen_interface** - FTP server interface\n8. **imap.enabled** - Enable IMAP email server\n9. **imap.listen_interface** - IMAP server interface\n10. **jwt.secret** - JWT signing secret\n11. **jwt.token.issuer** - JWT issuer name\n12. **language.default** - Default language\n13. **hostname** - Server hostname\n14. **encryption.secret** - Data encryption secret\n15. **totp.secret** - TOTP 2FA secret\n16. **password.reset.email.from** - Password reset sender\n17. **yjs.storage.path** - YJS document storage path\n18. **caldav.enable** - CalDAV server enable\n\n**Configuration Features:**\n- Stored in `_config` table in database\n- Environment-aware (debug/test/release)\n- Versioning with previous value tracking\n- Admin-only access required\n- Changes persist across restarts\n\n### ✅ Performance Features (Verified)\n\n**Rate Limiting:**\n- Per-route rate limiting\n- IP + path based limiting\n- Default 500 requests/second\n- Returns 429 on limit exceeded\n- Configurable via `limit.rate`\n\n**GZIP Compression:**\n- Automatic compression for responses\n- Enabled via `gzip.enable` config\n- Content-Encoding: gzip header\n- Works with all API endpoints\n\n**Caching Architecture:**\n- **Olric Distributed Cache** for cluster-wide caching\n- **File Cache** for static assets\n- Cache namespaces: `assets-cache`\n- Size limits: 2MB max file size\n- Compression threshold: 5KB\n- Expiry times:\n  - Default: 24 hours\n  - Images: 7 days\n  - Videos: 14 days\n  - Text files: 1 day\n\n### ✅ Security Infrastructure (Tested)\n\n**CORS Configuration:**\n- Fully configurable CORS headers\n- Credentials support enabled\n- Wildcard methods allowed\n- Per-origin configuration\n- Preflight handling\n\n**Certificate Management:**\n- Self-signed certificate generation\n- RSA 2048-bit keys\n- 365-day validity\n- Automatic TLS on port 6443\n- Certificate storage encrypted\n\n**Security Headers:**\n- CORS headers on all responses\n- Authentication via JWT Bearer tokens\n- Admin-only configuration access\n\n### ✅ Multi-Site Architecture (Mapped)\n\n**Subsite Features:**\n- Multiple sites on single instance\n- Host-based routing\n- Path-based routing\n- Static file serving\n- Cloud storage integration\n\n**Site Configuration:**\n- Entity: `site` table\n- Admin permission required\n- Dynamic site loading\n- Template engine support\n\n### 📊 Session 4 Feature Count: 37/52 (71%)\n\n**Documented Features:**\n1. Configuration API pattern\n2. 18 configuration parameters\n3. Runtime configuration updates\n4. Database-backed config storage\n5. Rate limiting implementation\n6. Per-route rate configuration\n7. GZIP compression support\n8. Olric distributed cache\n9. File cache with size limits\n10. CORS configuration\n\n### 🔧 Key Infrastructure Insights\n\n**Configuration Best Practices:**\n- Use `/_config/backend/` for server settings\n- Changes take effect immediately (except GraphQL)\n- Store secrets encrypted\n- Environment-specific values\n\n**Performance Optimization:**\n- Rate limiting prevents abuse\n- GZIP reduces bandwidth\n- Caching improves response times\n- Distributed cache for scaling\n\n**Security Hardening:**\n- CORS properly configured\n- TLS auto-enabled on 6443\n- JWT tokens for all admin operations\n- Configuration requires admin role\n\nFor detailed documentation and examples, see `todo.md` in this repository and the comprehensive OpenAPI documentation at `/openapi.yaml`.\n\n![Alt](https://repobeats.axiom.co/api/embed/f833f4480ea5c9966619d330b90e49f882831f03.svg \"Repobeats analytics image\")\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaptin%2Fdaptin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaptin%2Fdaptin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaptin%2Fdaptin/lists"}