Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Winbee/SynoCI

A guide to install a continuous integration infrastructure on a NAS Synology
https://github.com/Winbee/SynoCI

Last synced: 7 days ago
JSON representation

A guide to install a continuous integration infrastructure on a NAS Synology

Host: GitHub
URL: https://github.com/Winbee/SynoCI
Owner: Winbee
License: mit
Created: 2017-05-01T17:43:08.000Z (over 7 years ago)
Default Branch: master
Last Pushed: 2017-05-12T16:33:48.000Z (over 7 years ago)
Last Synced: 2024-08-02T15:23:27.836Z (3 months ago)
Language: Java
Size: 268 KB
Stars: 21
Watchers: 4
Forks: 5
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# Goal

This repo is made to provide a guide to install a basic infrastructure for Continuous integration on a NAS Synology.

# What does it contains

Here is a schema of the architecture:

![architecture_schema](https://rawgit.com/Winbee/SynoCI/master/extra/architecture_schema.svg)

Everything run on the NAS itself. The Firewall, DNS Server and Reverse Proxy are available in DSM 6. The other elements come from the official docker hub.

A brief introduction:
- [Gogs](https://github.com/gogits/gogs): a self-hosted Git service.
- [Nexus](https://books.sonatype.com/nexus-book/reference3/): an artifact repository for java, javascript, docker, C#, Python or Ruby packages
- [Jenkins](https://jenkins.io/): an automation server used to automate the CI jobs.
- [Portainer](http://portainer.io/): an UI to manage the docker environnment

# Limitation

This has only been tested on a Synology DS716+ upgraded with 8Gb of ram (If you are interested to upgrade yours as well, this one Kingston 8GB DDR3L 1600MHz KVR16LS11/8 works). In theory, it should work with any other x86 NAS Synology with at least 4Gb of ram.

# How to proceed
## Warm up
- First go check this [page](https://www.synology.com/en-global/knowledgebase/DSM/tutorial/General/How_to_add_extra_security_to_your_Synology_NAS) to add extra security on your NAS.
- Activate the ssh service of the NAS by following this [page](https://www.synology.com/en-global/knowledgebase/DSM/help/DSM/AdminCenter/system_terminal).
- Install Git server and Docker in the package center of the DSM
- Optional: Copy the ```.bashrc``` located in ```synologyMachine/config``` of the project and paste it in your ```/volume1/homes/``` and ```/root``` of your NAS.

## Docker
When Docker is installed, two directories are created on the NAS:
- ```/volume1/docker```
- ```/volume1/@docker```

From what I understood, ```/volume1/@docker``` contains all the data generated by docker i.e. the images, the containers, the volumes and other things. The other directory ```/volume1/docker``` contains a directory called "docker_registry" but we won't use it. The interesting part is that ```/volume1/docker``` is a shared folder and therefore can be easily encrypted directly in DSM UI.

All the docker containers that we will run will have a [data volume](https://docs.docker.com/engine/tutorials/dockervolumes/#data-volumes) attached to them. This data volume won't be lost if the container is stopped or removed. By default, docker volumes are created here ```/volume1/@docker/volumes```. We have to change that and put the volumes directory in the ```/volume1/docker``` folder. Connect yourself to your NAS with SSH and type the following commands:
``` bash
# Swith to root user
sudo su
# Move the old folder to the new place
mv /volume1/@docker/volumes /volume1/docker/volumes
# Create a symbolic link to keep the same structure as before.
ln -s /volume1/docker/volumes /volume1/@docker/volumes
```

- In your DSM => Control Panel => Shared Folder, click on the "docker" folder and Edit => "Encrypt this shared folder"
- Do the same with the "homes" folder.
Your valuable data won't be accessible if the NAS is shut down.
Limitation: When you relaunch the NAS, you need first to mount the docker folder manually in the Control Panel before launching the docker
service in the package center.

Connect yourself to your NAS with SSH and type the following commands:
``` bash
# In your home folder clone this project
git clone [email protected]:Winbee/SynoCI.git
# Swith to root user
sudo su
# Go inside the project
cd /volume1/homes//SynoCI/synologyMachine/CI
# Launch the docker container
docker-compose up -d
# Watch the log to check that everything is launching properly
docker-compose logs -f
```

- On your local machine try to connect to the Portainer on this adress: http://:18050
- Choose to connect to the local docker
- You should be able to see all your containers

## DNS configuration
- Install the DNS server in the package center
- Open the DNS server in the DSM
- In the resolution panel:
- Check "Enable resolution service".
- Check "Enable forwarders"
- Put the DNS server IP you want in "Forwarder 1" and "Forwarder 2". Check [openNIC](https://www.opennic.org/) or [openDNS](https://www.opendns.com/setupguide/) or your internet service provider.
- Configure your router to select the IP of the NAS as your DNS
- You should be able to access internet normaly
- Open again the DNS server in the DSM
- Create a master zone :
- Domain type: Forward Zone
- Domain name: mydomain.com
- Master DNS server: IP of your NAS
- Double click on this newly created zone and create a "A type" resource record for every adress you need. You should have a table resembling to that:

Name | TTL | Type | Information
--------------------------- | ----- | ---- | -------------
gogs.mydomain.com. | 86400 | A |
docker-hosted.mydomain.com. | 86400 | A |
docker-all.mydomain.com. | 86400 | A |
jenkins.mydomain.com. | 86400 | A |
nexus.mydomain.com. | 86400 | A |
mydomain.com. | 86400 | NS | ns.mydomain.com.
ns.mydomain.com. | 86400 | A |

Limitation: For some reason, I couldn1t make Portainer to work with a specific name like portainer.mydomain.com. You have to access it directly with the IP.

## Reverse proxy
- Go in your DSM => Control Panel => Application Portal => Reverse Proxy
- Create entry for each adress you need. When http, the port should be 80, When https, the port should be 443. At the end, you should have a table resembling to that:

Description | Source | Destination
-------------------- | ---------------------------------- | ---------------------
gogs | http://gogs.mydomain.com | http://localhost:18061
gogs https | https://gogs.mydomain.com | http://localhost:18061
jenkins | https://jenkins.mydomain.com | http://localhost:8082
jenkins https | https://jenkins.mydomain.com | http://localhost:8082
nexus | https://nexus.mydomain.com | http://localhost:8081
nexus https | https://nexus.mydomain.com | http://localhost:8081
nexus docker group | https://docker-hosted.mydomain.com | http://localhost:18044
nexus docker hosted | https://docker-all.mydomain.com | http://localhost:18045

## Certificate
- Go in your DSM => Control Panel => Security => Certificate
- Click on Add => Add a new certificate => Create a self-signed Certificate
- Fill up all the inputs. Do not forget to fill up the "Subject Alternative Name" field. This one should contain all the domain you want to access so: gogs.mydomain.com;docker-hosted.mydomain.com;docker-all.mydomain.com;jenkins.mydomain.com;nexus.mydomain.com
- As it is a self-signed certificate the root certificate of the certificate authority is unknown. You will have to add this authority in your local computer to be able to have https access on your website.
- Export the certificate on your local computer
- In the zip file, you should find a file named "syno-ca-cert.pem". This is the one you need to add to all your tools on your local machine in order to work.

## Nexus configuration

To create a docker registry in Nexus, you have to have a specific port for each one you want to reach.
I've created 3 docker repositories:
- docker-hosted: responsible for hosting all my private images
- docker-proxy: responsible for proxying docker hub (This one doesn't need a specific port as it will be distributed by docker-all)
- docker-all: responsible for distributing artifacts stored in docker-hosted and docker-proxy repos.

The official documentation should be enough for the rest.

### Acces the private docker registry
Nexus is able to provide a docker registry. Once you've configured it, it needs extra steps to be able to be used.
Docker command always use https and ssl. If you created a self-signed certificate like explained in the Certificate part of this README,
you have to configure your client computer to tell docker that this is an insecure-registry.

#### From the NAS
If you want to access the registry from the NAS, you have to modify this file ```/var/packages/Docker/etc/dockerd.json``` and add the
insecure-registries property.
``` bash
vim /var/packages/Docker/etc/dockerd.json
{
"ipv6": true,
"insecure-registries": ["docker-all.mydomain.com", "docker-hosted.mydomain.com"]
}
```
Once it is saved, go back to your DSM => Package Center => Docker and restart the service.
You should now be able to pull and push from the nexus registry.

#### From other computer
Depending of the plateform you're using, there are different ways to solve the problem. This [stackoverflow answer](http://stackoverflow
.com/questions/26710153/remote-access-to-a-private-docker-registry) can help you.

On linux mint, I could make it work by changing this file:
``` bash
sudo vim /etc/docker/daemon.json
{
"hosts": ["fd://"],
"insecure-registries": ["docker-all.mydomain.com", "docker-hosted.mydomain.com"]
}
```
and this file:
``` bash
sudo vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd
#ExecStart=/usr/bin/dockerd -H fd://
```
and by restarting the service:
``` bash
sudo systemctl stop docker
sudo systemctl daemon-reload
systemctl start docker
```

## Gogs configuration

I didn't encounter any particular problem. The official doc should be enough to start.

### Duplicating your private git repo
If you are paranoid about losing your git repo on your NAS, you can easily duplicate your repo by adding multiple remote location in git
config. See this [stackoverflow answer](http://stackoverflow.com/a/3195446) for more details.

## Jenkins configuration

I didn't encounter any particular problem. The official doc should be enough to start.

## Portainer configuration

I coudn't not access Portainer through a domain name. I had to type the IP of my NAS.
Another important thing, when Portainer start for the first time, select the local Docker option.

## Sidenote

Don't forget to backup everything by following the recommendation of each products.