Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/whatwedo/dde

Local development environment toolset based on Docker
https://github.com/whatwedo/dde

docker hacktoberfest

Last synced: 10 days ago
JSON representation

Local development environment toolset based on Docker

Awesome Lists containing this project

README 

        
![Logo](assets/img/logo.png)



# dde (Docker based Development Environment)



Local development environment toolset based on Docker supporting multiple projects.



Features include:



* Simplified Docker web application development

* Installation of system wide services:

    * `*.test` domain lookup based on [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html)

    * Reverse Proxy based on [jwilder/nginx-proxy](https://github.com/jwilder/nginx-proxy) to run multiple projects on same port (80/443) with autoconfigured SSL certificates

    * [MariaDB](https://mariadb.org/) ([MySQL](https://www.mysql.com/) alternative)

    * [MailCrab](https://github.com/tweedegolf/mailcrab) (SMTP testing server)

    * [ssh-agent](https://www.ssh.com/ssh/agent) used for sharing your SSH key without adding it to your project Docker containers.

* Choose you preferred file sharing

    * docker volume export

    * Performance optimized file sharing based on [docker-sync](http://docker-sync.io/) 

    * [Mutagen](https://mutagen.io/)



**Note:** dde is currently under heavy development and we don't offer any backward compatibility. However we use it at [whatwedo](https://www.whatwedo.ch/) on daily bases and it's safe to use it in your development environment.



## Requirements



* macOS, Linux or Windows (WSL 2)

* [Docker 17.09.0+](https://docs.docker.com/)

* [docker-compose 1.22+](https://docs.docker.com/compose/)

* [docker-sync 0.5+](http://docker-sync.io/) 

* [Mutagen v0.10.0+](https://mutagen.io/)

* [Bash](https://www.gnu.org/software/bash/)

* [openssl](https://www.openssl.org/)

* No other services listening localhost on:

    * Port 53

    * Port 80

    * Port 443

    * Port 3306



# Architecture



![Architecture](assets/img/architecture.svg)



## Installation



```

cd ~

git clone https://github.com/whatwedo/dde.git

~/dde/dde.sh system:dde:install

~/dde/dde.sh system:up

```



`system:dde:install` modifies your .profile files based on your shell:



* autocompletion 

* aliases 



dde can now be used in a new shell, enjoy!



### Aliases

```

# if you're using bash

echo "alias dde='~/dde/dde.sh'" >> ~/.bash_profile



# if you're using zsh

echo "alias dde='~/dde/dde.sh'" >> ~/.zshrc

```



### Autocompletion



add `eval $(~/dde/dde.sh --autocomplete)`  to  `~/.zshrc` or `~/.bash_profile`  



### Additional OS specific installation steps



#### macOS



Forward requests for `.test`-domains to the local DNS resolver:



```

sudo mkdir -p /etc/resolver

echo -e "nameserver 127.0.0.1" | sudo tee /etc/resolver/test

```



Trust the newly generated Root-CA for the self-signed certificates



```

sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/dde/data/reverseproxy/etc/nginx/certs/ca.pem

```



To ensure DNS functionality in Docker on macOS X:



In the Docker GUI: Go to `"Settings" → "Resources" → "Network"`, then turn off "Use kernel networking for UDP".

In the configuration file: Set `kernelForUDP` to `false` in `~/Library/Group Containers/group.com.docker/settings.json`.



#### Linux



Set your DNS to `127.0.0.1` with fallbacks of your choice.



Trust the newly generated Root-CA found here:



```

~/dde/data/reverseproxy/etc/nginx/certs/ca.pem

```



##### Linux



1. **Convert PEM to CRT and Add Globally:**



   ```bash

   openssl x509 -outform der -in ~/dde/data/reverseproxy/etc/nginx/certs/ca.pem -out ~/dde/data/reverseproxy/etc/nginx/certs/ca.crt

   sudo cp ~/dde/data/reverseproxy/etc/nginx/certs/ca.crt /usr/local/share/ca-certificates/

   sudo update-ca-certificates

   ```



#### Windows (WSL 2)



Set your DNS to `127.0.0.1` with fallbacks of your choice.



1. In WSL-Terminal, adjust the file `/etc/wsl.conf` to



```

[network]

generateResolvConf = false

```



2. Remove the file / link `resolv.conf` and close the WSL-Terminal



3. To restart WSL, run in powershell (admin):



```

wsl --shutdown

Get-Service LxssManager | Restart-Service

```



4. Open a WSL-Terminal and create a file `/etc/resolv.conf` with the following content:



```

nameserver 127.0.0.1

nameserver 1.1.1.1

```



5. Restart WSL again (step #3)



## File-Sync



File-sync can be done by mutagen, docker-sync or by native docker volumes. 



### Mutagen



put the `mutagen.yml` file in the project root directory see [mutagen-example](example/with-mutagen).



In the `docker-compose.yml` the volume is not exposed.



### Docker-Sync



put the `docker-sync.yml` file in the project root directory see [docker-sync-example](example/with-dockersync).



In the `docker-compose.yml` the volume is not exposed.



### native Docker



define in your `docker-compose.yml` or `docker-compose.override.yml` the exposed volumes.



### override the sync Settings



If you have project where the file-sync done by mutagen or docker-sync. you are able to override

the setting in a `docker-compose.override.yml` file.



copy sample `docker-compose.override.yml`:



```

$ dde project:docker-override

```



this command copies [docker-compose.override.yml](example/docker-compose.override.yml) to your project directory.



edit the file. With the custom-tag  `x-dde-sync` you can now choose you preferred syncing mode. 



valid values are `docker-sync`, `mutagen` or `volume`



if you use `volume` you must expose the volume in the `docker-compose.override.yml` file. 



## Tip and Tricks



### Configuring a Custom Shell using `.dde.yml`



To configure a custom shell in whatwedo/dde using the `.dde.yml` configuration file, specify your preferred shell with the container.shell key. This setting instructs whatwedo/dde to use the specified shell within the container. For instance, to utilize zsh as the container shell, your configuration would appear as follows:



```yml

version: "1"



container:

  shell: zsh

```



And in your docker-compose.yml file, add the corresponding environment variable:



```yml

environment:

  - DDE_CONTAINER_SHELL: ${DDE_CONTAINER_SHELL}

```



This ensures that the custom shell setting is effectively utilized within the container.



To specify a default shell in the Dockerfile, include the ARG directive for customization. For example:



```dockerfile

ARG DDE_CONTAINER_SHELL

```



### Configuring DNS Forwarding in DDE



The environment variables `DDE_DNS_FORWARD_1` and `DDE_DNS_FORWARD_2` allow for setting custom DNS servers in the Docker Development Environment (DDE). This is useful when local Internet DNS servers are to be used.



#### Instructions



1. Set `DDE_DNS_FORWARD_1` and `DDE_DNS_FORWARD_2` to the IP addresses of your preferred DNS servers.



   Example:



   ```bash

   export DDE_DNS_FORWARD_1=192.168.1.100

   export DDE_DNS_FORWARD_2=192.168.1.101

   ```



#### Adding `DDE_DNS_FORWARD_1` and `DDE_DNS_FORWARD_2` to `bashrc`



For a more permanent solution, you can add the `DDE_DNS_FORWARD_1` and `DDE_DNS_FORWARD_2` variables to your `bashrc` file. This ensures that these variables are automatically set every time a new shell session is started.



To do this, append the `export` commands to your `~/.bashrc` file:



1. Open your `~/.bashrc` file in a text editor, for example, you can use `nano`:



   ```bash

   nano ~/.bashrc

   ```



Add the following lines at the end of the file:



    ```bash

    export DDE_DNS_FORWARD_1=

    export DDE_DNS_FORWARD_2=

    ```

Replace  and  with the IP addresses of your preferred DNS servers.



Save and close the file.



To apply the changes immediately, source your ~/.bashrc file:



```bash

source ~/.bashrc

```



Now, `DDE_DNS_FORWARD_1` and `DDE_DNS_FORWARD_2` will be set automatically in each new shell session.



### Fix Permission



Services such as nginx in Docker containers normally runs with the `root` user. With the `dde exec` command

you login into the container with the user `dde`. If services writes files, ex. `var/cache`, `root` is the owner. 

On the host files also the `root` is the owner. In this case you normally not able to delete or change the files 

created by the services.



the [command](commands/project/fix-permissions.sh) `project:fix-permissions` resolve this issue by `chown dde:dde` 

in the container and `chown {yourLocalUser}:{yourLocalGroup}` in the local host.



### OPEN_URL & DDE_BROWSER



Add `OPEN_URL` in the `environment` array of your `docker-compose.yml`.



On the `project:up` or `project:open` command the website(s) will be opened in your standard browser.



```yaml

services:

    web:

        ...

        environment:

            - VIRTUAL_HOST=cloud.project.test

            - OPEN_URL=http://cloud.project.test/

    storage:

        ...

        environment:

            - VIRTUAL_HOST=minio.project.test

            - OPEN_URL=http://minio.project.test:9000/



```



Set the environment variable `DDE_BROWSER` if you what to start a specific browser.



`command/local.sh`

```bash

DDE_BROWSER=/usr/bin/firefox

```



## Additional Services



If you need additional central Services. eg. PostgresSQL just add then in `docker-compose.override.yml` in the 

dde-directory.



Available Services



- [PostgresSQL](services/postgres/docker-compose.override.yml)

- MySql

- Redis

- ....



## Usage



```

$ dde help

```



### Project configuration



Due to the early stage of this project there is no full documentation available. We created a [example](example) project with all required and optional configuration. Please checkout the [example](example) directory.



## Including custom command



you can include custom commands by adding them in the `commands/local/` directory. 

Custom commands must be prefixed with the `local:` namespace.



### Anatomy of a command



`commands\local\my_command.sh`

```bash

## inline help for local:my_command

#

# more help for the command

#

# this will be displayed with the --help argument on the command

#

# e.g dde local:command --help

#



function local:my_command() {

    echo 'execute local:my_command'

    _localCommand_someInternalFunction arg1

}



function _localCommand_someInternalFunction() {

    echo "do something with ${1}"

}

```



* script must be located in the `command` directory

* Help text for the commands

  * `:` will be replaced by `/` for locating the help script

  * the first line beginning with `##` is the help text displayed be the help command

  * all following lines beginning with `#` will displayed in the command help 

* you can add as many functions as you want in the script

* to avoid conflicts prefix internal functions 

* functions and can also be defined in the `command/local.sh` file

* the `command/local.sh` file is the last loaded source, so you are able to overwrite system variables

and functions there



`command/local.sh`

```bash

function _local_someGlobalHelperFunction() {

    echo "a global helper function"

}



# overwrite a variable

SYNC_MODE=volume



# overwrite a function/command

function project:env() {

    echo "my custom project env"

}



```



## Known solutions

* **failed to remove network dde**  

If you get this error it means your project `docker-compose.yml` is wrongly configured.

Be sure to mark the `dde` network as external, like in our examples:

```yml

networks:

    default:

        name: "dde"

        external: true # <-- important

```



## Known problems



* Files of filesystem mapped with docker-sync will get group id `0`.



## Bugs and Issues



If you have any problems with this image, feel free to open a new issue in our issue tracker https://github.com/whatwedo/dde/issues



## License



This project is under the MIT license. See the complete license in the repository: [LICENSE](LICENSE)