https://github.com/lblod/app-organization-portal
OrganisatiePortaal backend
https://github.com/lblod/app-organization-portal
Last synced: 3 months ago
JSON representation
OrganisatiePortaal backend
- Host: GitHub
- URL: https://github.com/lblod/app-organization-portal
- Owner: lblod
- License: mit
- Created: 2021-03-16T13:31:03.000Z (over 4 years ago)
- Default Branch: development
- Last Pushed: 2024-04-15T14:18:00.000Z (about 1 year ago)
- Last Synced: 2024-04-16T06:36:57.825Z (about 1 year ago)
- Language: JavaScript
- Homepage:
- Size: 3.07 MB
- Stars: 1
- Watchers: 9
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Organization portal
Backend for the organization portal application, based on the mu.semte.ch microservices stack.
## How to
### Boot up the system in DEV environment
cd /path/to/mu-project
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up
You can shut down using `docker-compose stop` and remove everything using `docker-compose rm`.
#### Setup consumers
The application depends on external data. You'll have to perform extra steps if you want this data to be available.
These are multiple data sets, we'll try to be exhaustive on how to ingest these. Even though the procedures are similar.
#### Setup mandatarissen sync
Note: this is assuming you have never ran this before.
```
drc down;
```
Update `docker-compose.override.yml` to:
```
mandatarissen-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_LANDING_ZONE_DATABASE: "triplestore"
DCR_REMAPPING_DATABASE: "triplestore"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Then:
```
drc up -d migrations
drc up -d db mandatarissen-consumer
# Wait until success of the previous step
```
Then, update `docker-compose.override.yml` to:
```
mandatarissen-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Finally, put the stack back up and reindex elasticsearch
```
drc up -d
sh scripts/reset-elastic.sh
```
#### Setup leidinggevenden sync (http://data.lblod.info/vocabularies/leidinggevenden/Functionaris)
Note: this is assuming you have never ran this before.
```
drc down;
```
Update `docker-compose.override.yml` to:
```
leidinggevenden-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_LANDING_ZONE_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
DCR_REMAPPING_DATABASE: "triplestore" # for the initial sync, we go directly to virtuoso
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Then:
```
drc up -d migrations
drc up -d db leidinggevenden-consumer
# Wait until success of the previous step
```
Then, update `docker-compose.override.yml` to:
```
leidinggevenden-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # choose the correct endpoint
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Finally, put the stack back up and reindex elasticsearch
```
drc up -d
sh scripts/reset-elastic.sh
```
#### Setup worship mandatees sync
Note: this is assuming you have never ran this before.
```
drc down;
```
Update `docker-compose.override.yml` to:
```
worship-services-main-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "triplestore"
DCR_REMAPPING_DATABASE: "triplestore"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
worship-services-private-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "triplestore"
DCR_REMAPPING_DATABASE: "triplestore"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Then:
```
drc up -d migrations
drc up -d db worship-services-main-info-consumer worship-services-private-info-consumer
# Wait until success of the previous step
```
Then, update `docker-compose.override.yml` to:
```
worship-services-main-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
worship-services-private-info-consumer:
environment:
DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be" # or another endpoint
DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive/login" # or another endpoint
DCR_SECRET_KEY: "key-of-the-producer"
DCR_LANDING_ZONE_DATABASE: "db"
DCR_REMAPPING_DATABASE: "db"
DCR_DISABLE_DELTA_INGEST: "false"
DCR_DISABLE_INITIAL_SYNC: "false"
```
Finally, put the stack back up and reindex elasticsearch
```
drc up -d
sh scripts/reset-elastic.sh
```
### Setting up the delta-producers related services
To make sure the app can share data, producers need to be set up. There is an intial sync, that is potentially very expensive, and must be started manually
#### producers administrative-units
1. make sure the app is up and running, the migrations have run
2. in docker-compose.override.yml, make sure the following configuration is provided:
```
delta-producer-pub-graph-maintainer-administrative-units:
environment:
START_INITIAL_SYNC: 'true'
```
3. `drc up -d delta-producer-pub-graph-maintainer-administrative-units`
4. You can follow the status of the job, ideally through the dashboard, but this hasn't been setup yet. The following query should also give results:
```
PREFIX task:
PREFIX adms:
PREFIX dct:
SELECT DISTINCT ?job ?operation ?created ?modified ?status WHERE {
GRAPH ?g {
?job a ;
task:operation ?operation;
adms:status ?status;
dct:created ?created;
dct:modified ?modified.
}
FILTER( ?operation IN (
,
)
)
}
```
## Configuring the dashboard
### Accessing the dashboard from your local machine
Since we use dispatcher v2, which dispatches on hostname, we'll have to update `/etc/hosts`.
Add an entry similar to the following. Ensure the first part of the domain starts with `dashboard`.:
```
127.0.0.1 dashboard.localhost
```
### Creating new users
For now, we use specific logins for the dashboard users. Each environment has its own passwords.
To add a user, make sure to have [mu-cli](https://github.com/mu-semtech/mu-cli) installed first.
Then in `docker-compose.override.yml`
```
dashboard-login:
environment:
MU_APPLICATION_SALT: 'a_random_string_with_sufficient_entropy_hence_not_this_one'
```
You can now generate a user by running `mu script project-scripts generate-dashboard-login` and following the steps.
Restart `migrations` and it should insert the new user into the database.