https://github.com/bbkane/shovel_ansible
Install shovel on a Linux machine!
https://github.com/bbkane/shovel_ansible
Last synced: about 1 year ago
JSON representation
Install shovel on a Linux machine!
- Host: GitHub
- URL: https://github.com/bbkane/shovel_ansible
- Owner: bbkane
- Created: 2023-09-30T04:50:29.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-05-05T15:38:40.000Z (about 2 years ago)
- Last Synced: 2025-04-26T22:57:45.935Z (about 1 year ago)
- Language: Jinja
- Size: 38.1 KB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# shovel_ansible
Install [shovel](https://github.com/bbkane/shovel) + [OpenObserve](https://openobserve.ai/) as systemd services, on a local dev VM or production VM.
# Local Dev VM install
## Start a local Lima VM
Mostly from [Lima: a nice way to run Linux VMs on Mac](https://jvns.ca/blog/2023/07/10/lima--a-nice-way-to-run-linux-vms-on-mac/)
Install Lima:
```bash
brew install lima
```
Useful Lima commands:
```bash
limactl start default
lima # open shell
limactl factory-reset default # to test playbook from scratch
limactl stop
```
## Install locally trusted TLS certificates
Uses [`mkcert`](https://github.com/FiloSottile/mkcert) - Note the warning!
> **Warning**: the rootCA-key.pem file that mkcert automatically generates gives complete power to intercept secure requests from your machine. Do not share it.
```bash
brew install mkcert nss
mkcert -install
mkcert localhost
```
This generates `./localhost-key.pem` and `localhost.pem`, signed by the `rootCA` cert generated by `mkcert`.
## Install shovel + OpenObserve on the Lima VM
Note: if using an arm64 Mac, Lima VMs are aarch64 (another name for the same architecture), so get linux arm64/aarch64 versions of the releases.
Obtain a shovel release from [GitHub](https://github.com/bbkane/shovel/releases) or by building with GoReleasor
Get an [OpenObserve release](https://github.com/openobserve/openobserve/releases).
Update [./lima_vars.yaml](./lima_vars.yaml) with paths to the OpenObserve and shovel tar.gz files.
Export environmental variables for Ansible to read
```bash
export SHOVEL_SERVE_OPENOBSERVE_PASS='...';
export SHOVEL_SERVE_OPENOBSERVE_USER='...';
export ZO_ROOT_USER_EMAIL='...';
export ZO_ROOT_USER_PASSWORD='...';
```
I use [`envelope`](https://github.com/bbkane/envelope) to manage environment variables, so I use the following commands to set up the environment:
```bash
envelope env create
envelope env var create --name ZO_ROOT_USER_EMAIL --value root@example.com
envelope env var create --name ZO_ROOT_USER_PASSWORD --value ZO_ROOT_USER_PASSWORD
# Re-use the root user password
envelope env ref create --name SHOVEL_SERVE_OPENOBSERVE_USER --ref-env-name "$PWD" --ref-var-name ZO_ROOT_USER_EMAIL
envelope env ref create --name SHOVEL_SERVE_OPENOBSERVE_PASS --ref-env-name "$PWD" --ref-var-name ZO_ROOT_USER_PASSWORD
export-env "$PWD"
```
Run the Ansible playbooks:
```bash
ansible-playbook \
-i 'lima-default,' \
--ssh-extra-args "-F $HOME/.lima/default/ssh.config" \
--extra-vars "@lima_vars.yaml" openobserve.ansible.yaml shovel.ansible.yaml
```
## Open sites
Open OpenObserve at: http://localhost:5080/web/traces?period=15m&query=&org_identifier=default
Open shovel at: https://localhost:8080/?count=1&nameservers=dns3.p09.nsone.net%3A53&protocol=udp&qnames=linkedin.com+www.linkedin.com&rtypes=A&subnetMap=&subnets=
## Import dashboards
There isn't an API for this yet, so export by using the download icon when viewing a dashboard and import from the main menu.
# Production VM
Obtain releases and export envvars as above.
Create `prod_vars.yaml`. Use `./lima_vars.yaml` for reference
NOTE: ensure the `-i` flag for a single host ends with a comma (`,`)
```bash
ansible-playbook \
--ask-become-pass \
-i ',' \
--extra-vars "@prod_vars.yaml" \
openobserve.ansible.yaml shovel.ansible.yaml
```
# Debugging
```bash
# Check if the service is running
sudo systemctl status shovel
# Check logs
sudo journalctl -u shovel
# Analyze security to tighten systemd capabilities
sudo systemd-analyze security /etc/systemd/system/openobserve.service
```