{"id":24491465,"url":"https://github.com/vxcontrol/soldr","last_synced_at":"2025-04-14T00:21:43.884Z","repository":{"id":63610555,"uuid":"565905250","full_name":"vxcontrol/soldr","owner":"vxcontrol","description":"System of Orchestration, Lifecycle control, Detection and Response","archived":false,"fork":false,"pushed_at":"2023-12-25T07:30:17.000Z","size":55887,"stargazers_count":58,"open_issues_count":11,"forks_count":10,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-27T14:21:47.262Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vxcontrol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"security/vconf/hardening/.gitkeep","support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-11-14T15:16:16.000Z","updated_at":"2025-01-10T15:35:34.000Z","dependencies_parsed_at":"2023-12-23T18:45:18.445Z","dependency_job_id":null,"html_url":"https://github.com/vxcontrol/soldr","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vxcontrol%2Fsoldr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vxcontrol%2Fsoldr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vxcontrol%2Fsoldr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vxcontrol%2Fsoldr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vxcontrol","download_url":"https://codeload.github.com/vxcontrol/soldr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248800047,"owners_count":21163404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-21T18:18:14.237Z","updated_at":"2025-04-14T00:21:43.839Z","avatar_url":"https://github.com/vxcontrol.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SOLDR\n\nSOLDR is an Endpoint Detection and Response system which consists of centralised management part with extensive Web UI and Agents being installed on endpoint systems. SOLDR allows you not only to configure security policies but also write your own modules and make detection of the comprehensive security events as well as do almost instant response on the security alarms.\n\n## Repository structure\n\n- `build` - storage of all built artifacts\n- `cmd` - services entry points\n- `db` - database migrations\n- `internal` - services implementation\n- `scripts` - supplemental scripts\n- `security` - security configuration\n- `web` - web interface source\n\n## Local run\n\n### Prerequisites\n\n- [docker-ce](https://docs.docker.com/engine/installation)\n- [docker-compose](https://docs.docker.com/compose)\n- [protobuf](https://github.com/protocolbuffers/protobuf/releases)\n- [go](https://go.dev/dl/)\n- `protoc-gen-go`\n    - install with `go install google.golang.org/protobuf/cmd/protoc-gen-go@latest`\n- [node](https://github.com/nodesource/distributions/blob/master/README.md), recommended node 16 LTS\n- `python2`\n- `mysql` client\n- [minio client](https://docs.min.io/minio/baremetal/reference/minio-mc.html)\n    - Note: better install as `mcli` due to [conflict](https://github.com/minio/mc/blob/RELEASE.2018-01-18T21-18-56Z/CONFLICT.md)\n\n#### Example: Ubuntu 22.04\n\n```bash\nsudo apt update\nsudo apt install build-essential ca-certificates openssl software-properties-common curl git mysql-client jq\n\n# golang 1.19\nsudo add-apt-repository ppa:longsleep/golang-backports\nsudo apt-get install golang-go\n\n# node 16 LTS\ncurl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -\nsudo apt-get install -y nodejs\n\n# yarn\ncurl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg \u003e/dev/null\necho \"deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main\" | sudo tee /etc/apt/sources.list.d/yarn.list\nsudo apt-get update \u0026\u0026 sudo apt-get install yarn\n\n# docker\ncurl -fsSL https://get.docker.com -o get-docker.sh\nsudo sh get-docker.sh\nusermod -aG docker \"$USER\"\nnewgrp docker\n```\n\n### Configure environment\n\nSetup an environment configuration:\n\n```bash\ncp .env.template .env\n```\n\n### Run whole project by docker compose\n\nDocker images stored in [vxcontrol](https://hub.docker.com/u/vxcontrol) docker hub account.\nThis scenario requires `.env` and `docker-compose.yml` files.\n\n#### Download and update images\n\n```bash\ndocker compose pull\n```\n\n#### Run services\n\n```bash\ndocker compose up -d\n```\n\n#### Run observability stack components\n\n```bash\ndocker compose --profile obs up -d\n```\n\n#### Stop services\n\n```bash\ndocker compose down\n```\n\n#### Remove services and collected data\n\n```bash\ndocker compose down -v\n```\n\n### First run without compose\n\nSOLDR require prepared DB schema and additional data stored in S3 for proper initialization. Pull dependencies and start MySQL and Minio servers for simplicity using docker.\n\n```bash\ndocker compose pull\ndocker compose up -d mysql minio\n```\n\nWait until MySQL and Minio starts (approx. 30 seconds for the first run), and seed required data:\n\n```bash\nmake db-init\nmake s3-init\n```\n\nGenerate certificates and other crypto materials:\n\n**Attention**: Regeneration of crypto configuration will lead to communication breakage for already stored modules and running agents after rebuild the server components.\n\n```bash\nmake generate-all\n```\n\nBuild services and a Web UI:\n\n```bash\nmake build-all\n```\n\nIn case of some issues you can use build with prepared environment in docker image:\n\n```bash\nmake build-backend-vxbuild\n```\n\nUpload agent binary into binary storage and register it:\n\n```\nmake s3-upload-vxagent\n```\n\nConfigure web UI proxy:\n\n```bash\nmake setup-web-proxy\n```\n\nStart a web UI:\n\n```bash\nmake run-web\n```\n\nAn API Server:\n\n```bash\nmake run-api\n```\n\nAn Agent Server:\n\n```bash\nmake run-server\n```\n\nAn Agent instance:\n\n```bash\nmake run-agent\n```\n\nAnd finally, upload SOLDR modules to local storage:\n\n```bash\ndocker compose up -d modules\n```\n\n### First login and validation of the setup\n\nOpen [`http://localhost`](http://localhost) in a browser. Follow the login process and use default credentials:  `admin` / `admin`.\n\nClick on `Modules` tab in the top menu bar, then open module creation wizard using `Create a module` button.\nName your first module and select `Basic` template, rest leave as is.\n\nClick on `Create` button, which will lead you to a Module editor. Now, when a module is created, let's assign it to a policy.\n\nOpen `Groups of agents` tab in the top menu bar, then click `Create a group` button and give some name to a new group.\n\nTo assign an Agent to a new group, open `Agents` tab in the top menu, select `All agents` filter group, then click on an agent name in the list and on the `Move to group` button in the top right corner of the interface. Finally, select the group that you have just created and finish the process using button `Move`.\n\nNow we need to create a policy that will contain our new module and assign it to that group. To create a policy, open `Policies` tab in the top menu bar, then click on `Create a policy` button, name your first policy and proceed with `Create` button.\nUsing `Link to groups` button in the top right corner of the interface, create a link between a group and a policy - click on a link icon in the list right near the name of the group that you want to link.\n\nLet's then add the first created module to the policy - Being in the `Policices` tab, you should see `Modules` section with `Available to be added` group being selected by default. Click on `Install` button to the right of the module name in the list.\n\nAs a final step, let's validate that the module has been installed on the Agent according to assigned policy and that this module can successfully communicate with other parts of the system. Open `Agents` tab, then get into a module by clicking on its name. In the `Modules` section you will find one module being installed on an agent and information that this module is configured by a policy. Using the gear-looking button to the right of the module name to get into the interactive interface of the module.\nThen click on `Send data` button and check the log right below - it should contain following log lines which tells that test data was sent to the agent part of the module and as a result some data was returned:\n```\n    2:04:20 PM.152 SEND DATA: {\"type\":\"hs_browser\",\"data\":\"test test test\"}\n    2:04:20 PM.166 RECV DATA: {\"data\":\"pong\",\"type\":\"hs_server\"}\n```\n\nCongratulations! The setup of the SOLDR project is done, it is fully functioning and ready for you to dig into the wilderness of the Endpoint Detection and Response!\n\n### Next steps\n\nCheck the configuration in the `.env` file and in `web/proxy.conf.json` to understand the service's configuration, so you can start them using your preferred installation topology.\n\nFor the regular start-up of the services, just start an API server, an Agent server and a Web UI.\n\n### Run in debugger in Goland\n\n- Install [EnvFile](https://github.com/ashald/EnvFile) plugin, which will add ability to include file in Run configuration.\n- Open `Edit configuratons` -\u003e `Go build`, select `Run Kind` - `Package`. Add package path, for example `soldr/cmd/api`\n- Open EnvFile tab, enable it and add `.env` file to file paths.\n- Run\n\n### Run in debugger in VSCode\n\nUse `.vscode/launch.json`\n\nLaunch on of the available debug tasks:\n- launch vxapi\n- launch vxserver\n- launch vxagent\n- launch web ui\n\n### Observability stack\n\nObservability stack collect metrics, traces, logs from SOLDR components. Stack consist of:\n- `Grafana` - querying and visualizing observability data\n- `VictoriaMetrics` - datastore for server and SOLDR components metrics\n- `node-exporter` and `elasticsearch-exporter` - scraping metrics\n- `Jaeger` - storing and querying traces\n- `Elasticsearch` - datastore for `Jaeger`\n- `OpenTelemetry collector` - single entry point to receive, process and export all observability data \n\n\u003e For more information about collector, visit [`https://opentelemetry.io/docs/collector`](https://opentelemetry.io/docs/collector).\n\nRun observability stack components and then open `Grafana` in a browser [`https://localhost:3000`](https://localhost:3000). Default credentials: `admin/admin`. After default password for `admin` user changed, you can check provisioned SOLDR dashboards by click `Dashboards` icon on menu bar. On this SOLDR dashboards you can view server, agents and modules resource utilization, events statistics etc. \nFor checking traces, you need to click `Explore` icon on menu bar, then choose `Jaeger` data source from dropdown in the top left. Now you can query and filter traces by `Service Name`, `Operation Name`, `Tags`, `duration` and `Time ranges`.\n\n\u003e Full observability stack requires more resources compare to clean SOLDR.\n\n### Clean up the project\n\nRemove all build files and other security keys:\n\n**Attention**: Removing of crypto configuration and keys will lead to communication breakage for already stored modules and running agents after rebuild the server components.\n\n```bash\nmake clean-all\n```\n\nOnly artefacts from build directory (golang services binaries):\n\n```bash\nmake clean-build\n```\n\nOnly artefacts from web directory (node-modules and result of building frontend):\n\n```bash\nmake clean-web\n```\n\nDangerous step to remove security files (use it in edge case):\n\n```bash\nmake clean-sec\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvxcontrol%2Fsoldr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvxcontrol%2Fsoldr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvxcontrol%2Fsoldr/lists"}