https://github.com/redhatinsights/patchman-engine
The purpose of this repository is to store source code for System Patch Manager application
https://github.com/redhatinsights/patchman-engine
hacktoberfest insights rhel
Last synced: 6 months ago
JSON representation
The purpose of this repository is to store source code for System Patch Manager application
- Host: GitHub
- URL: https://github.com/redhatinsights/patchman-engine
- Owner: RedHatInsights
- License: gpl-3.0
- Created: 2019-10-31T13:42:31.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2025-04-12T05:27:27.000Z (6 months ago)
- Last Synced: 2025-04-12T06:25:56.895Z (6 months ago)
- Topics: hacktoberfest, insights, rhel
- Language: Go
- Homepage:
- Size: 5.42 MB
- Stars: 13
- Watchers: 11
- Forks: 33
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/RedHatInsights/patchman-engine/actions/workflows/unittests.yml)
[](https://github.com/RedHatInsights/patchman-engine/actions/workflows/open_api_spec.yml)
[](https://codecov.io/gh/RedHatInsights/patchman-engine)
# patchman-engine
System Patch Manager is one of the applications for [console.redhat.com](https://console.redhat.com). This application allows users to display and manage available patches for their registered systems. This code repo stores sources for the backend part of the application which provides the REST API to the frontend.
## Table of content
- [Architecture](docs/md/architecture.md)
- [Database](docs/md/database.md)
- [Development environment](#development-environment)
- [Local running](#local-running)
- [Local app requests](#local-app-requests)
- [Tests running](#tests-running)
- [OpenAPI docs](#openapi-docs)
- [Control by private API](#control-by-private-api)
- [VMaaS](#vmaas)
- [Monitoring](#monitoring)
- [Profiling] (#profiling)
## Development environment
### Running locally
Uses `podman-compose` to deploy the individual project components and supporting containers, which simulate the CMSfR platform and database:
~~~bash
podman-compose up --build # Build images if needed and start containers
podman-compose down # Stop and remove containers
~~~
Use `--profile monitoring` to run local `prometheus` and `grafana`, for example
~~~bash
podman-compose --profile monitoring up
~~~
Grafana is accessible at `http://localhost:3000`, Prometheus at `http://localhost:9090`
### Local app requests
When podman compose is running, you can test the app using dev shell scripts:
~~~bash
cd dev/scripts
./systems_list.sh # show systems
./advisories_list.sh # show advisories
./platform_sync.sh # trigger vmaas_sync to sync (using vmaas mock)
./platform_upload.sh # simulate archive upload to trigger listener and evaluator_upload
~~~
### Running in host OS
Run single component in host OS, rest in podman-compose:
~~~bash
podman-compose stop evaluator_upload # stop single component running using podman-compose
export $(xargs < conf/local.env)
./scripts/entrypoint.sh evaluator # (or listener, or manager) run component in host OS
~~~
### Running tests
We cover a large part of the application functionality with tests; this requires also running a test database and mocked services. This is all encapsulated into the configuration runable using podman-compose command. It also includes static code analysis, database migration tests and dockerfiles checking. It's also used when checking pull requests for the repo.
~~~bash
podman-compose -f docker-compose.test.yml up --build --abort-on-container-exit
~~~
### Run single test
After running all test suit, testing platform components are still running (kafka, platform, db). This is especially useful when fixing some test or adding a new one. You need to have golang installed.
~~~bash
podman-compose -f docker-compose.test.yml up --build --no-start # build images
podman-compose -f docker-compose.test.yml start db platform kafka # start containers
. scripts/export_local_env.sh # setup needed env variables for tests
go test -count=1 -v ./evaluator -run TestEvaluate # run "TestEvaluate" test from "evaluator" component
~~~
### Run tests in VS Code
Prerequisite is to have [Go Extension](https://marketplace.visualstudio.com/items?itemName=golang.Go)
installed.
To set it up copy the example settings from `.vscode/settings.example.json`:
```
cp .vscode/settings.example.json .vscode/settings.json
```
### Access to dev/test database
When a podman compose (either dev or test one) is running, then the database can
be access directly by executing
```
podman exec -it db psql -d patchman -U admin
```
or locally using `psql` with:
```
export $(cat conf/local.env conf/database_admin.env | xargs ) 2>/dev/null; ./dev/scripts/psql.sh
```
### OpenAPI docs
Our REST API is documented using OpenAPI v3. On a local instance it can be accessed on .
To update/regenerate OpenAPI sources run:
~~~bash
go get -u github.com/swaggo/swag/cmd/swag # download binary to generate, do it first time only
./scripts/generate_docs.sh
~~~
## Control by private API
There is a private API accessible only from inside of `vmaas_sync` container. It allows running component routines manually. In local environment it can be tested like this:
~~~bash
podman exec -it patchman-engine_vmaas_sync_1 ./sync.sh # trigger advisories syncing event.
podman exec -it patchman-engine_vmaas_sync_1 ./re-calc.sh # trigger systems recalculation event.
podman exec -it patchman-engine_vmaas_sync_1 ./caches-check.sh # trigger account caches checking.
~~~
## VMaaS
This project uses [VMaaS](https://github.com/RedHatInsights/vmaas) for retrieving information about advisories, and resolving which advisories can be applied to which systems. For local development this repo contains VMaaS service mock as a part of platform mock allowing independent running of the service using podman-compose.
## Monitoring
Each application component (except for the database) exposes metrics for [Prometheus](https://prometheus.io/)
on `/metrics` endpoint (see [docker-compose.yml](docker-compose.yml) for ports). Runtime logs can be sent to Amazon
CloudWatch if configuration environment variables are set (see [awscloudwatch.go](base/utils/awscloudwatch.go)).
## Kafka control
Your can control and inspect def Kafka instance using:
~~~bash
docker-compose exec kafka bash # enter kafka component and run inside:
/usr/bin/kafka-topics --list --bootstrap-server=kafka:9092 # show created topics
# list all messages send to a topic
/usr/bin/kafka-console-consumer --bootstrap-server=kafka:9092 --topic platform.inventory.events --from-beginning
# send debugging message to a topic
echo '{"id":"00000000-0000-0000-0000-000000000002"}' | /usr/bin/kafka-console-producer --broker-list kafka:9092 --topic patchman.evaluator.upload
~~~
## Run SonarQube code analysis
~~~bash
export SONAR_HOST_URL=https://sonar-server
export SONAR_LOGIN=paste-your-generated-token
export SONAR_CERT_URL=https://secret-url-to/ca.crt # optional
podman-compose -f dev/sonar/docker-compose.yml up --build
~~~
## Update Grafana config map
Copy Grafana board json config to the temporary file, e.g. `grafana.json` and run:
~~~bash
./scripts/grafana-json-to-yaml.sh grafana.json > ./dashboards/app-sre/grafana-dashboard-insights-patchman-engine-general.configmap.yaml
~~~
## Profiling
App can be profiled using [/net/http/pprof](https://pkg.go.dev/net/http/pprof). Profiler is exposed on app's private port.
### Local development
- set `ENABLE_PROFILE=true` in the `cont/common.env`
- `docker-compose up --build`
- `go tool pprof http://localhost:{port}/debug/pprof/{heap|profile|block|mutex}`
available ports:
- 9000 - manager
- 9002 - listener
- 9003 - evaluator-upload
- 9004 - evaluator-recalc
### Admin API
- set `ENABLE_PROFILE_{container_name}=true` in the ClowdApp
- download the profile file using internal api `/api/patch/admin/pprof/{manager|listener|evaluator_upload|evaluator_recalc}/{heap|profile|block|mutex|trace}`
- `go tool pprof `
## Deps backup
[patchman-engine-deps](https://github.com/RedHatInsights/patchman-engine-deps)