Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aetherinox/obi-sync-docs
Documentation for Obsidian's obi-sync plugin located at https://github.com/acheong08/obi-sync
https://github.com/aetherinox/obi-sync-docs
obi-sync obsidian-md obsidian-plugin obsidian-plugins
Last synced: 26 days ago
JSON representation
Documentation for Obsidian's obi-sync plugin located at https://github.com/acheong08/obi-sync
- Host: GitHub
- URL: https://github.com/aetherinox/obi-sync-docs
- Owner: Aetherinox
- Created: 2023-09-09T09:06:19.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-09-10T02:29:07.000Z (over 1 year ago)
- Last Synced: 2024-05-21T02:20:44.307Z (7 months ago)
- Topics: obi-sync, obsidian-md, obsidian-plugin, obsidian-plugins
- Homepage: https://github.com/acheong08/obi-sync
- Size: 161 KB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Reversed Obsidian Sync (Unofficial)
 
This is an unofficial Obsidian Sync library which allows you to host your own server for [syncing](https://obsidian.md/sync) and [publishing](https://obsidian.md/publish) obsidian notes. Behind the scenes, it re-routes Sync and Publish tasks from the official Obsidian servers to your own self-hosted server.
This system consists of two parts:
1. Self-hosted Server | [Repo](https://github.com/acheong08/obi-sync)
2. Obsidian Plugin | [Repo](https://github.com/acheong08/rev-obsidian-sync-plugin)
You will first need to configure the self-hosted server which will be responsible for storing all of your Obsidian.md vault plugins and files. Then you will need to install the associated plugin in your Obsidian.md software which will be responsible for telling Obsidian to use your servers instead of the official Obsidian servers.
> [!WARNING]
> The main branch is the development branch. For stable usage, use the latest release.
> [!NOTE]
> If you have the time and energy, feel free to help out with PRs or suggestions.
# Index
- [Features](#Features)
- [Server Installation](#Server-Installation)
- [Create DNS Records / SSL](#Create-DNS-Records--SSL)
- [Install with Docker-compose](#Docker-Compose-Option-1)
- [Install with Docker](#Docker-Option-2)
- [Nginx Configuration](#Nginx-Configuration)
- [Creating New User](#Creating-New-User)
- [Windows (Powershell)](#%EF%B8%8F-Windows-Powershell)
- [Linux (Terminal)](#%EF%B8%8F-Linux-Terminal)
- [Plugin Installation](#Plugin-Installation)
- [Install -> WGET](#install-with-wget)
- [Install -> Manually](#install-manually)
- [Enable Plugin](#Enable-Plugin)
- [Configure Sync](#Configure-Sync)
- [Configure Publish](#Configure-Publish)
- [Build & Run](#Build--Run)
- [FAQ](#FAQ)
- [Where Are Files Stored](#Where-Are-Files-Stored)
- [What Files Are Created On Server?](#what-files-are-created-on-server)
- [Error: User Not Signed Up](#error-user-not-signed-up)
- [Network Error Occured. Network Unavailable](#network-error-occured-network-unavailable)
- [Error: No Matching Manifest linux/amd64](#error-no-matching-manifest-linuxamd64)
- [Docker-compose.yml vs .Env File](#docker-composeyml-vs-env-file)
- [Can't Find .obsidian Folder or .Env File](#cant-find-obsidian-folder-or-env-file)
- [Privacy & Security](#privacy--security)
---
## Features
- End to end encryption
- Live sync (across devices)
- File history/recovery/snapshots
- Works natively on IOS/Android/Linux/MacOS/Windows... (via the plugin)
- Vault sharing
- [Publish (markdown only. no rendering yet)](https://github.com/acheong08/obi-sync/wiki/dian-Publish)
[🔝 Top](#Index)
## Server Installation
Follow the instructions below to set up obsidian sync with your [Obsidian.md](https://obsidian.md/) program.
### Create DNS Records / SSL
If you are wanting to host your sync server outside of `localhost`, you can use [Cloudflare](https://cloudflare.com/) to take care of your SSL certificate needs. If you do not want to use Cloudflare but still require an SSL certificate, you can skip the instructions below and utilize [Certbot](https://certbot.eff.org/).
Access your Domain or Cloudflare control panel and create new records for two new subdomains.
The `Content` value should be the `IP Address` your self-hosted sync server will be hosted on.
Each subdomain plays the following roles:
| Subdomain | Purpose |
| --- | --- |
| `api.domain.com` |
Used for `Obsidian Sync`.
This URL needs to be plugged into your `docker-compose.yml`, and set up in the settings of the [Unofficial obi-sync plugin](https://github.com/acheong08/rev-obsidian-sync-plugin) as the `Obsidian Sync URL`
|
| `publish.domain.com` |
Used for `Obsidian Publish`.
Configure your self-hosted server, enable the `Obsidian Plugin`; then upload your vault files using `Obsidian Publish`.
View your uploaded files at:
`https://publish.domain.com/vaultname/Path/To/File.md`
|
If you want Cloudflare to handle the SSL certificate, set each record's `Proxy Status` to  **Proxied**
### Docker-Compose (Option 1)
If using this option, you must decide which setup you would like to use.
- One File: `docker-compose.yml`
- Two Fles: `docker-compose.yml` and `.env`
Using the `two file` option is slightly more secure in regards to your `SIGNUP_KEY` being exposed to logs.
- If you are not concerned about security, use [DOCKER-COMPOSE.YML ONLY](#-docker-composeyml-only) option.
- If you want the extra security, use [DOCKER-COMPOSE.YML + .ENV](#-docker-composeyml--env) option.
---
##### 🟢 DOCKER-COMPOSE.YML ONLY
Create a new file named `docker-compose.yml`, and then paste the corresponding code provided below:
```yml
version: '3.8'
services:
obsidian_sync:
image: ghcr.io/acheong08/obi-sync:latest
container_name: obsidian_sync
restart: always
ports:
- 3000:3000
environment:
- DOMAIN_NAME=api.domain.com
- ADDR_HTTP=0.0.0.0:3000
- SIGNUP_KEY=YOUR_PRIVATE_STRING_HERE
- DATA_DIR=/obi-sync/
- MAX_STORAGE_GB=10
- MAX_SITES_PER_USER=5
volumes:
- ./obi-sync:/obi-sync
volumes:
obi-sync:
```
[🔹Continue With Installation](#Variable-List)
---
##### 🟢 DOCKER-COMPOSE.YML + .ENV
Create the following two files in the same folder, and then paste the corresponding code provided below:
- [docker-compose.yml](#-docker-composeyml)
- [.env](#-env)
#### 📄 docker-compose.yml
```yml
version: '3.8'
services:
obsidian_sync:
image: ghcr.io/acheong08/obi-sync:latest
container_name: obsidian_sync
restart: always
ports:
- 3000:3000
environment:
DOMAIN_NAME: ${DOMAIN}
ADDR_HTTP: ${ADDR_HTTP}
SIGNUP_KEY: ${USER_SIGNUP_KEY}
DATA_DIR: ${DIR_DATA}
MAX_STORAGE_GB: ${USER_MAX_STORAGE}
MAX_SITES_PER_USER: ${USER_MAX_SITES}
volumes:
- ./obi-sync:/obi-sync
volumes:
obi-sync:
```
#### 📄 .env
```env
DOMAIN='api.domain.com'
ADDR_HTTP='0.0.0.0:3000'
DIR_DATA='/obi-sync/'
USER_SIGNUP_KEY='YOUR_PRIVATE_STRING_HERE'
USER_MAX_STORAGE=10
USER_MAX_SITES=5
```
[🔹Continue With Installation](#Variable-List)
---
#### Variable List
After creating `docker-compose.yml` / `.env` file(s), edit the variables to match your domain `DOMAIN_NAME`, registration signup key `SIGNUP_KEY`, etc.
A description of each variable is provided below:
| Variable | Description | Required | Default |
| --- | --- | --- | --- |
| `DOMAIN_NAME` | This is the URL to your API subdomain | Yes | `localhost:3000` |
| `ADDR_HTTP` | The address to run Obi-Sync on | No | `127.0.0.1:3000` |
| `SIGNUP_KEY` | Required later when creating users who will be able to access your self-hosted server | No | None |
| `DATA_DIR` | Where encrypted `vault.db` and other files will be stored | No | `./` |
| `MAX_STORAGE_GB` | The maximum storage per user in GB. | No | `10` |
| `MAX_SITES_PER_USER` | The maximum number of sites per user. | No | `5` |
#### Start/Stop Docker
After you finish configuring your docker's variables, you need to `cd` to the folder where you created the file(s) and execute:
```shell
docker compose up -d
```
To shut down the container:
```shell
docker compose down
```
To confirm container is running:
```shell
docker ps
```
---
### Docker (Option 2)
To install obi-sync using docker `pull`, you can run the default command below:
```shell
docker pull ghcr.io/acheong08/obi-sync:latest
```
For a full list of pull commands based on your operating system, view [Package Release](https://github.com/acheong08/obi-sync/pkgs/container/obi-sync).
---
### Nginx Configuration
You must configure nginx to handle your new subdomains by adding a few entries to your nginx config file. If you wish to add this to your current running nginx webserver, you can paste it somewhere in `\etc\nginx\sites-enabled\default`.
> [!NOTE]
> Do not copy/paste the server blocks below without looking them over. If you have changed port `3000` in your docker container, you must change it below to that same port. Also change `domain.com` to your correct domain.
```nginx
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
#
# Obsidian Sync
#
# defined within the obi-sync plugin for Obsidian so that Obsidian and
# the self-hosted server can sync files.
#
server
{
listen 443 ssl http2;
listen [::]:443 ssl http2;
listen 80;
listen [::]:80;
server_name www.api.domain.com api.domain.com;
location /
{
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:3000/;
}
}
#
# Obsidian Publish
#
# used for viewing published Obsidian .md files on a webserver.
#
server
{
listen 443 ssl http2;
listen [::]:443 ssl http2;
listen 80;
listen [::]:80;
server_name www.publish.domain.com publish.domain.com;
location /
{
proxy_pass http://127.0.0.1:3000/published/;
}
}
```
After modifying your nginx service, **restart** it to load up the new configs.
```shell
sudo systemctl restart nginx
```
Or restart nginx via docker if you run it through a container.
---
### Creating New User
You must create a user that will be allowed to access your self-hosted server from within Obsidian.md.
This can be done by opening Powershell in Windows, or Terminal in Linux and executing the following:
> [!NOTE]
> `email`: This is the email address you will use in Obsidian's `About` tab to sign into your self-hosted server.
>
> `password`: Pick any password you wish to use. This is the password you will use in the Obsidian.md program once you configure the plugin to connect to your self-hosted server.
>
> `name`: Can be anything, not super important.
>
> `signup_key`: Variable you provided in `docker-compose.yml` or `.env` file.
> If you removed `signup_key` from your docker container's variables and don't want to require a signup key for registration, remove that line from the command below.
#### ⚙️ Windows `Powershell`
```powershell
curl --request POST `
--url https://api.domain.com/user/signup `
--header 'Content-Type: application/json' `
--data '{
"email": "[email protected]",
"password": "example_password",
"name": "Example User",
"signup_key": ""
}'
```
#### ⚙️ Linux `Terminal`
```bash
curl --request POST \
--url https://api.domain.com/user/signup \
--header 'Content-Type: application/json' \
--data '{
"email": "[email protected]",
"password": "example_password",
"name": "Example User",
"signup_key": ""
}'
```
A **successful** registration will return the following response:
```json
{"email":"[email protected]","name":"Example User"}
```
A **failed** registration will return the following response:
> [!WARNING]
> If you receive a **failure** from registration, make sure you're not trying to sign up the same user multiple times.
```json
{"error":"not sign up!"}
```
[🔝 Top](#Index)
---
## Plugin Installation
In order for your new self-hosted Publish and Sync server to function properly, you must install a plugin to your copy of Obsidian.md
### Install with WGET
Navigate to your vault's `.obsidian` folder:
```shell
cd /path/to/vault/.obsidian
```
Create a new folder for the plugin and enter the folder:
```shell
mkdir -p plugins/custom-sync-plugin && cd plugins/custom-sync-plugin
```
Use `wget` to download the plugin files:
```shell
wget https://github.com/acheong08/rev-obsidian-sync-plugin/raw/master/main.js https://github.com/acheong08/rev-obsidian-sync-plugin/raw/master/manifest.json
```
### Install Manually
To manually get the latest copy of the unofficial Obi-Sync Plugin, [download here](https://github.com/acheong08/rev-obsidian-sync-plugin/releases).
- Navigate to the folder where your vault is on your local machine, and enter the `.obsidian\plugins\` folder.
- Create a new folder: `custom-sync-plugin`
- Inside the `custom-sync-plugin` folder, install the plugin files:
- 📄 `main.js`
- 📄 `manifest.json`
> [!NOTE]
> Alternatively, you can use https://github.com/TfTHacker/obsidian42-brat which can be found in the official community plugins list.
### Enable Plugin
Once the plugin is installed, activate it by launching Obsidian.md.
- Open `Obsidian Settings` 
- On left menu, click `Community Plugins`
- Scroll down to `Custom Native Sync` and enable 
- To the left of the enable button for `Custom Native Sync`, press the options  button.
- Configure the `Obsidian Sync URL` in the Plugin settings after setting up your sync server
- By default, it is `https://api.domain.com`
- Go to the `Core Plugins` section of Obsidian
- Locate the core plugin `Sync` and enable 
### Configure Sync
Before doing these steps, ensure that your obi-sync self-hosted server is running.
Open your Obsidian.md Settings  and select `About`.
You need to sign in and connect Obsidian.md to your self-hosted server by clicking `Log In`:
The login dialog will appear:
Fill in your email address and password that you used to register your account with the `cURL` command in the section [Creating New User](#Creating-New-User) above.
```powershell
curl --request POST `
--url https://api.domain.com/user/signup `
--header 'Content-Type: application/json' `
--data '{
"email": "[email protected]", <----- The email you will use
"password": "example_password", <----- The password you will use
"name": "Example User",
"signup_key": ""
}'
```
> [!NOTE]
> After signing in successfully, the `Log In` button will change to `Manage settings`. Keep in mind though that clicking the `Manage Settings` button will take you to Obsidian's official website. It has nothing to do with your own self-hosted server.
Next, on the left side under `Core Plugins`, select `Sync`.
On the right side, click the `Choose` button:
A new dialog will then appear. Select `Create New Vault`
Then fill in the information for your new vault. The `Encryption Password` can be anything you want. Do **not** lose this password. You cannot unlock / decrypt your vault if you can't remember it.
Your new vault will be created, along with a few options you can select, and a `Connect` button.
Click the `Connect` button to start Syncing between your local vault and your self-hosted sync server. If you get a warning message labeled `Confirm Merge Vault`, click `Continue`.
If you provided an `Encryption Password` a few steps back, you will now be asked to enter that password, and then click `Unlock Vault`.
You will then get one more confirmation that your vault is now synced.
From this point on, you can adjust any of the Sync settings you wish to modify, which includes syncing things like images, videos, plugins, settings, and more. Whatever options you enable, will be included in the Sync job.
Ensure `Sync Status` is set to `running`. You can enable / disable the sync's state by clicking the button to the right.
You can confirm that your files are successfully syncing by clicking the `View` button to the right of `Sync Activity`.
Once pressing the `View` button, a new dialog will appear and show you the status of your Sync job.
```console
2023-09-09 01:38 - Uploading file Test Folder/Subfolder/My File 1.md
2023-09-09 01:38 - Upload complete Test Folder/Subfolder/My File 1.md
2023-09-09 01:38 - Uploading file Test Folder/Subfolder/My File 2.md
2023-09-09 01:38 - Upload complete Test Folder/Subfolder/My File 2.md
2023-09-09 01:40 - Fully synced
```
### Configure Publish
These instructions help guide you through setting up `Obsidian Publish`. This feature will allow you to view your Vault .md notes on your webserver.
Open your Obsidian.md settings, and under `Options` on the left, select `Core Plugins`. In the middle of the screen, enable  `Publish`.
Back on the main Obsidian interface, select the `Publish Changes`  button in your Obsidian side / toolbar.
In the new dialog, enter a `Site ID` in the textfield provided. This will become the `slug` name for your new vault.
Once you have a name, click `Create`.
You will then see a list of all your vault's associated files. Select the checkbox to the left of each folder you wish to upload to your self-hosted Publish server.
Once you've selected the desired folders, click `Publish` in the bottom right.
Yet another dialog will appear which confirms your uploaded vault documents.
> [!NOTE]
> Because we are hosting our own server, the link provided at the bottom of the dialog **will not work** since it goes to Obsidian's official publish server.
Open your web browser and go to the URL for your self-hosted publish server. In our example, we would view our `Testing Folder/Tags.md` page by entering the following URL:
```
https://publish.domain.com/myvault/Testing Folder/Tags.md
```
- `myvault`: Name given to example vault
- `Testing Folder`: Folder name the note resides in
- `Tags.md`: Note filename
To see an overview of files you have uploaded to your publish server, go to your publish subdomain, and add your vault name at the end. Nothing else needs added.
```
https://publish.domain.com/myvault
```
In our example vault, visiting the URL above displays `JSON` and includes a list of all files uploaded from the test vault:
From this point, you can upload whatever files you wish to have published and play around with the settings.
[🔝 Top](#Index)
---
## Build & Run
> [!NOTE]
> This requires you to install the Go interrupter on your machine.
> You may download it [here](https://go.dev/doc/install).
To build and run Obi-sync directly from the git repo, execute the following:
- `git clone https://github.com/acheong08/obi-sync`
- `cd obsidian-sync`
- `go run cmd/obsidian-sync/main.go`
[🔝 Top](#Index)
---
## FAQ
### Where Are Files Stored?
This depends on how you've configured obsidian sync server to run.
If you installed this project using [Install with Docker-compose](#Docker-Compose-Option-1), and specified the environment variable `DATA_DIR`, then the files should exist in the same folder you specified the variable to use. By default, vault files are placed in the folder `obi-sync`.
If you installed this project using [Install with Docker](#Docker-Option-2) or did not specify a variable for `DATA_DIR`, then the files are typically stored in `/var/lib/docker/*`
Some users may be running [Portainer](https://www.portainer.io/), which allows you to view your docker containers and volumes within a graphical user interface. Login to the portainer web admin panel and under `Volumes`, find out which volume is assigned to `obsidian_sync`, and copy the value `Mount path`. Open your File Explorer and go to that location to view your vault files.
---
### What Files Are Created On Server?
This project will store your vault data in the following files:
```
📁 obi-sync
📄 publish.db
📄 secret.gob
📄 vault.db
📄 vaultfiles.db
```
---
### Error: User Not Signed Up
This error typically occurs when you're creating a user with `cURL` and shows the following:
```json
{"error":"not sign up!"}
```
If you receive this error when creating your first API user, ensure the user did not previously exist.
---
### Network Error Occured. Network Unavailable
If you are attempting to use the `Obsidian Publish` feature and receive the error:
Ensure you have provided the correct environment variable `DOMAIN_NAME=api.domain.com`.
If you provided the wrong domain name and need to change it, you must do the following:
- Edit `docker-compose.yml`
- Change `DOMAIN_NAME` variable to correct domain.
- Locate the project file `publish.db` and DELETE it completely.
- Restart the docker container and Obsidian.md program
Once back in Obsidian.md program, click `Publish Changes`  button again.
You can also attempt to locate the root cause of the issue by pressing `CTRL + SHIFT + I` inside of Obsidian.md. Then on the right side, click `Network` and then re-open the `Publish Changes` interface again.
Anything listed in `red` is an error and needs to be corrected. Ensure that it is trying to fetch the correct domain from the `Request URL`. Anything listed in `white` and with a `Status Code` of `200` is properly connecting.
---
### Error: No Matching Manifest linux/amd64
Make sure you are pulling the correct docker image.
You can also visit the [Package Release](https://github.com/acheong08/obi-sync/pkgs/container/obi-sync) page, select `OS / Arch` tab, and copy the pull command listed below `linux/amd64`
---
### Docker-compose.yml vs .Env File
In the section above titled [Install with Docker-compose](#Docker-Compose-Option-1), there are two ways to install obi-sync using `docker-compose`.
1. Single `docker-compose.yml` file
- See [DOCKER-COMPOSE.YML ONLY](#-docker-composeyml-only)
3. `docker-compose.yml` and `.env` file
- See [DOCKER-COMPOSE.YML + .ENV](#-docker-composeyml--env)
#### 🟢 Single `docker-compose.yml` File
This method involves creating a single `docker-compose.yml` file which will hold all of your settings for this project.
#### 🟢 `docker-compose.yml` and `.env` File
This method requires you to create two files which both exist in the same folder. The benefit of this method is that your `SIGNUP_KEY` environment variable will not be leaked in your docker logs and is slightly better if you are worried about security.
You can decide to use either one of the two options.
If using `Portainer` web manager for Docker, you can access the environment variables clicking `Containers` on left side menu, then click `obsidian-sync` in your list of containers. Then scroll down to the `ENV` section.
---
### Can't Find `.obsidian` Folder or `.Env` File
The `.obsidian` plugin folder and docker `.env` file may be hidden. You can configure your operating system's File Explorer show hidden files or use a terminal.
`Linux`: By default, this OS hides all folders and files that start with a period. To view these files in your File Explorer, press the key combination `CTRL + H`. You may be asked for your account or sudo password.
`iOS`: You may need to plug the device into a computer, then to access your `.obsidian` folder, go to `Documents on iPhone`, then go to `Obsidian//.obsidian`.
---
### Privacy & Security
The plugin and self-hosted server are dead simple. No data is collected. You can even run your sync server on the local network without access to the internet. All vault data is stored behind an encrypted password and inside a database file.
---
[🔝 Top](#Index)