Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/logzio/logzio-agent-manifest
Contains all the scripts needed to ship logs, metrics and traces to Logz.io for supported datasources using the Logz.io agent.
https://github.com/logzio/logzio-agent-manifest
integration
Last synced: 11 days ago
JSON representation
Contains all the scripts needed to ship logs, metrics and traces to Logz.io for supported datasources using the Logz.io agent.
- Host: GitHub
- URL: https://github.com/logzio/logzio-agent-manifest
- Owner: logzio
- License: apache-2.0
- Created: 2022-03-21T12:43:49.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-09-17T10:20:04.000Z (2 months ago)
- Last Synced: 2024-09-18T11:09:44.703Z (2 months ago)
- Topics: integration
- Language: Shell
- Homepage:
- Size: 1.08 MB
- Stars: 8
- Watchers: 5
- Forks: 9
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Logz.io Agent Manifest
- [Architecture](#architecture)
- [Logic](#logic)
- [Agent Script Arguments](#agent-script-arguments)
- [Running The Agent Script](#running-the-agent-script)
- [Linux](#linux)
- [MacOS](#macos)
- [Windows](#windows)
- [Troubleshooting Tools](#troubleshooting-tools)
- [Script Arguments](#script-arguments)
- [Linux](#linux-1)
- [MacOS](#macos-1)
- [Windows](#windows-1)
- [Test and Build locally](#test-and-build-locally)
This repo contains all the scripts needed to ship logs, metrics and traces to Logz.io for supported datasources using the Logz.io agent.
Supports the following OS:
- Linux
- MacOS
- Windows
Supports the following datasources:
- AKS (Kubernetes)
- EKS (Kubernetes)
- GKE (Kubernetes)
## Architecture
Each datasource is represented as a path of 3 directories from the root of this repo (dir1/dir2/dir3) (for example: Kubernetes/AKS/Kubernetes).
This is the structure of directories in each path (dir1/dir2/dir3):
- prerequisites
- linux
- mac
- windows
- telemetry
- installer
- linux
- mac
- windows
- logs
- linux
- mac
- windows
- metrics
- linux
- mac
- windows
- traces
- linux
- mac
- windows
Each leaf directory contains main script (tasks to run using the functions from the functions script) and functions script (functions only).
* Also can contain other helper files
scripts directory contains the agent (entry point script. uses the functions from the functions script), functions (functions only) and utils functions (functions for all scripts in this repo) scripts for each OS, which are the entry point for all the scripts in this repo.
## Logic
The agent script will get the agent application JSON from the agent. According the JSON file, it will get a path of 3 directories (dir1/dir2/dir3) that exists in this repo (from the root). This path represents a datasource.
Using that path, it will get the prerequisites and the telemetry (installer, logs, metrics, traces) scripts.
The agent script will run the prerequisites and then the installer.
The installer script will get logs, metrics, traces scripts depending on the application JSON, and will run each of them.
* Through the whole process, the scripts will get files from this repo according the application JSON from the agent.
* Each main script contains tasks to execute. Each task runs in a separate process/job, and only if finished successfully will continue to the next task. If a task was not finished successfully, an error message will be shown and the agent script will exit.
* Some tasks depends on the application JSON (for example: metrics parameters).
* A success message will be shown at the end of a successful run.
* All scripts downloaded from this repo and temp files will be saved in Logz.io temp directory, which will be deleted at the end of the running.
* A log file will be created at the beginning of the running in the directory where the agent script was running from (except in Windows, which will be created under `Documents` directory).
## Agent Script Arguments
Each agent script has the following flags:
| Flag | Description |
| --- | --- |
| `--url=<>` | The Logz.io app URL (for example: https://app.logz.io) |
| `--id=` | The Logz.io agent ID. You will get it from the agent in Logz.io application |
| `debug=<>` | Runs the script with a local application JSON (for tests only) |
* You can run the agent script with `--help` to show the script usage.
## Running The Agent Script
### Linux
```Bash
bash <(curl -sSL https://github.com/logzio/logzio-agent-manifest/releases/download/latest/agent_linux.bash) --url=LOGZIO_APP_URL --id=AGENT_ID
```
### MacOS
```Bash
bash <(curl -sSL https://github.com/logzio/logzio-agent-manifest/releases/download/latest/agent_mac.bash) --url=LOGZIO_APP_URL --id=AGENT_ID
```
### Windows
* Must run from `Windows PowerShell` (NOT Windows PowerShell (x86) or Windows PowerShell ISE).
* Temp directory and the log file will be created under `Documents` directory.
```PowerShell
powershell { iex “& { $(irm https://github.com/logzio/logzio-agent-manifest/releases/download/latest/agent_windows.ps1) } --url=LOGZIO_APP_URL --id=AGENT_ID” }
```
## Troubleshooting Tools
under each OS in scripts directory, there is troubleshooting-tools directory which contains troubleshooting scripts (mainly for Support team).
* A log file will be created at the beginning of the running in the directory where the script was running from (except in Windows, which will be created under `Documents` directory).
### Script Arguments
| Flag | Description |
| --- | --- |
| `--path=LOGZIO_REPO_DATASOURCE_PATH` | The path of a datasource in this repo (dir1/dir2/dir3 for example: Kubernetes/AKS/Kubernetes) |
* You can run the script with `--help` to show the script usage.
### Linux
```Bash
bash <(curl -sSL https://github.com/logzio/logzio-agent-manifest/releases/download/latest/run_prerequisites_linux.bash) --path=LOGZIO_REPO_DATASOURCE_PATH
```
### MacOS
```Bash
bash <(curl -sSL https://github.com/logzio/logzio-agent-manifest/releases/download/latest/run_prerequisites_mac.bash) --path=LOGZIO_REPO_DATASOURCE_PATH
```
### Windows
* Must run from `Windows PowerShell` (NOT Windows PowerShell (x86) or Windows PowerShell ISE).
* Temp directory and the log file will be created under `Documents` directory.
```PowerShell
powershell { iex “& { $(irm https://github.com/logzio/logzio-agent-manifest/releases/download/latest/run_prerequisites_windows.ps1) } --path=LOGZIO_REPO_DATASOURCE_PATH” }
```
## Test and Build locally
If you're contributing code to this repository, please refer to the [Contribution Guidelines](CONTRIBUTING.md) for more details on the process.
Before submitting a Pull Request (PR), ensure that you test your changes locally by following these steps:
1. **Generate binary files:**
Run the following command in the project root folder to generate the necessary binary files in the `assets` folder.
```shell
make
```
2. **Create an Agent:**
Log in to your Logz.io account and create an Agent using the relevant integration that you want to test. This will provide you with a command similar to the following:
```shell
sudo mkdir -p /opt/logzio-agent; sudo chown -R $USER: /opt/logzio-agent; mkdir -p /tmp/logzio; curl -fsSL 'https://github.com/logzio/logzio-agent-manifest/releases/latest/download/agent_mac.tar.gz' -o /tmp/logzio/agent_mac.tar.gz; tar -zxf /tmp/logzio/agent_mac.tar.gz --directory /tmp/logzio; bash /tmp/logzio/agent.bash --url=https://app.logz.io --id=<>
```
3. **Update the command:**
Replace the `curl -fsSL` command with `cp <> /tmp/logzio/` where `<>` is the binary generated at step [1]. Example:
```shell
sudo mkdir -p /opt/logzio-agent; sudo chown -R $USER: /opt/logzio-agent; mkdir -p /tmp/logzio; cp <> /tmp/logzio/; tar -zxf /tmp/logzio/agent_mac.tar.gz --directory /tmp/logzio; bash /tmp/logzio/agent.bash --url=https://app.logz.io --id=<>
```
4. **Test your changes:**
Run the modified command locally to test your changes and ensure everything works as expected.