{"id":20631213,"url":"https://github.com/solidlabresearch/solid-stream-aggregator-demo","last_synced_at":"2025-03-08T17:57:18.856Z","repository":{"id":106974630,"uuid":"605981461","full_name":"SolidLabResearch/solid-stream-aggregator-demo","owner":"SolidLabResearch","description":"Demo of the Solid Stream Aggregator for the SolidLab Challenge 84","archived":false,"fork":false,"pushed_at":"2023-08-04T11:50:59.000Z","size":1831,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-17T07:09:36.104Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SolidLabResearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-24T10:25:40.000Z","updated_at":"2023-07-14T10:13:52.000Z","dependencies_parsed_at":"2024-11-16T14:11:36.391Z","dependency_job_id":"bd5b7b45-aed0-4368-885e-98e07109dddb","html_url":"https://github.com/SolidLabResearch/solid-stream-aggregator-demo","commit_stats":null,"previous_names":["solidlabresearch/solid-stream-aggregator-demo"],"tags_count":0,"template":false,"template_full_name":"argahsuknesib/TS-Template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fsolid-stream-aggregator-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fsolid-stream-aggregator-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fsolid-stream-aggregator-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fsolid-stream-aggregator-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SolidLabResearch","download_url":"https://codeload.github.com/SolidLabResearch/solid-stream-aggregator-demo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242588427,"owners_count":20154203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T14:11:34.290Z","updated_at":"2025-03-08T17:57:18.829Z","avatar_url":"https://github.com/SolidLabResearch.png","language":"TypeScript","readme":"# Solid Stream Aggregator Demonstration\n\nThis is a demonstration repository for the [Solid Stream Aggregator](https://github.com/argahsuknesib/solid-stream-aggregator) for the completion of the\nSolidLab Research's Challenge [84](https://github.com/solidLabResearch/challenges/issues/84).\n\n### Pre-requisites\n\n**NOTE** : The community solid server only works with the LTS releases of NodeJS.\n\n1. Clone this repository.\n\n   ```bash\n   git clone https://github.com/argahsuknesib/ssa-demo/\n   ```\n\n2. Install the dependencies from the root of the repository.\n\n   ```bash\n   npm install\n   ```\n\n3. Install the Community Solid Server.\n   **NOTE**: Install the Community Solid Server globally instead of locally. \n   Installing the Community Solid Server will cause component.js issues in the `node_modules` folder.\n\n   ```bash\n   npm install -g @solid/community-server\n   ```\n\n## Setup\n\n### Setting up the Solid pods with the data.\n\nFor the demonstration, we use the [DAHCC dataset](https://dahcc.idlab.ugent.be/dataset.html) \nwhich is an anonymized dataset of patients who lived in UGent's HomeLab.\nEach patient owns a Solid pod to themselves. \nWe use four patients for the demonstration.\nThe `/scripts/data/` folder contains the data to spin up the Solid pod for each patient with \nthe [DAHCC dataset](https://dahcc.idlab.ugent.be/dataset.html)\n\nTo spin up the Solid pods, run the following command from the root of the repository.\n\n ```bash\n npm run start-solid-server\n ```\n\nThis will generate the Solid pod for the patients at http://localhost:3000/. \nFor example, the Solid pod for the patient 1 will be at http://localhost:3000/dataset_participant1/.\n\nThe `/data/` folder contains the dataset for the patients.\n\n### Loading up the data in the Solid pods\n\nAfter the Solid pods are set up, \nwe would like to load some test data to aggregate over. \nThe dataset we are using is the [DAHCC dataset](https://dahcc.idlab.ugent.be/dataset.html). \nFor simplicity, we have mapped a portion of the DAHCC dataset to N3 files\nin [this repository](https://github.com/argahsuknesib/dahcc-heartrate).\n\n1. To get the dataset clone this repository with\n\n   ```bash\n   git clone https://github.com/argahsuknesib/dahcc-heartrate.git\n   ```\n\n2. To load up the data, \n   we use the [LDES in Solid Observations Replay repository](https://github.com/SolidLabResearch/LDES-in-SOLID-Semantic-Observations-Replay).\n   Clone the repository with\n\n   ```bash\n   git clone https://github.com/SolidLabResearch/LDES-in-SOLID-Semantic-Observations-Replay\n   ```\n3. Go to the `engine` folder and install the dependencies with\n\n   ```bash\n   cd engine \u0026\u0026 npm install\n   ```\n\n4. Edit the `src/config/replay_properties.json` file by adding the location of the folder in the \"datasetFolders\" field \n   where you cloned the `dahcc-heartrate` repository. \n   Add the location of the Solid pod's `data` folder in the \"lilURL\" field.\n\n   For example,\n   ```\n   \"lilURL\" : \"http://localhost:3000/dataset_participant1/data/\"\n   ```\n\n5. Start the engine with\n\n   ```bash\n   npm start\n   ```\n\n   The output should look similar to the following:\n\n   ```shell\n   \u003e challenge-16---replay---backend---typescript@1.0.0 start\n  \n   \u003e tsc \u0026\u0026 node dist/app.js`\n  \n   {port: [Getter], loglevel: [Getter], logname: [Getter], datasetFolders: [Getter], credentialsFileName: [Getter], lilURL: [Getter], treePath: [Getter], chunkSize: [Getter], bucketSize: [Getter], targetResourceSize: [Getter], default: {port: '3001', loglevel: 'info', logname: 'WEB API', datasetFolders: C:\\\\nextcloud\\\\development\\\\challenge16-replay\\\\main\\\\Challenge 16 - Replay - Backend - Typescript\\\\data', credentialsFileName: null, lilURL: 'http://localhost:3000/test/', treePath: 'https://saref.etsi.org/core/hasTimestamp', chunkSize: 10, bucketSize: 10, targetResourceSize: 1024}}\n  \n   2022-12-08T14:58:54.612Z [WEB API] info: Express is listening at http://localhost:3001\n   ```\n\n### Starting the webapp to load\n\nThe webapp is a simple vue app that allows you to load the data into the Solid pods. \n\n1. To start the webapp, run the following command from the root of the repository.\n\n   ```bash\n   cd webapp \u0026\u0026 npm install \u0026\u0026 npm run dev\n   ```\n\n2. Open the webapp, select a dataset to be loaded. Click on the `Load selected dataset` button.\n\n3. Click on `Get Observation Subjects` button to get the observation subjects from the dataset.\n\n4. Click on the `Sort observation subjects` button.\n5. Click on the `submit next observation`  button 3 times, till you see the replayer count as 3.\n   Now the pod will have 3 observations.\n\n6. Click on the `Submit remaining observations` button to submit the rest. \n   The data will be loaded up in the Solid pod.\n\n### Aggregation over the Solid pods\n\nOpen a new instance of the terminal.\n\n1. Build the project by running the following command from the root of the repository.\n\n   ```bash\n   npm run build\n   ```\n2. Install the Community Solid Server locally by running the following command from the root of the repository.\n\n   ```bash\n   npm install --save-dev @solid/community-server\n   ```\n\n3. To start the Solid stream aggregator, run the following command from the root of the repository.\n\n   ```bash\n   npm run start aggregation\n   ```\n\n   The Solid stream aggregator starts an HTTP express server at http://localhost:8080/.\n   The server exposes the `/averageHRPatient1` endpoint for the demonstration.\n   When this doing a GET request to this endpoint the following SPARQL query is executed\n\n   ```sparql\n   PREFIX saref: \u003chttps://saref.etsi.org/core/\u003e\n   PREFIX dahccsensors: \u003chttps://dahcc.idlab.ugent.be/Homelab/SensorsAndActuators/\u003e\n   PREFIX : \u003chttps://rsp.js/\u003e\n   REGISTER RStream \u003coutput\u003e AS\n   SELECT (AVG(?o) AS ?averageHR1)\n   FROM NAMED WINDOW :w1 ON STREAM \u003chttp://localhost:3000/dataset_participant1/data/\u003e [RANGE 10 STEP 2]\n        WHERE{\n              WINDOW :w1 { ?s saref:hasValue ?o .\n                          ?s saref:relatesToProperty dahccsensors:wearable.bvp .}\n              }\n   ```\n\n4. Once the Solid stream aggregator is running, \n   request the endpoint to instantiate the query to start the aggregation.\n\n   ```bash\n   curl http://localhost:8080/averageHRPatient1\n   ```\n\n   The aggregation has started, and the results are being written to the\n   http://localhost:3000/aggregation_pod/\n\n**NOTE**: The writing of the aggregation stream to the aggregation pod might take some time due to the \nNaive Algorithm of rebalancing and can slow down the Solid pod.\n\n### Reading the aggregation results\n\nThe aggregation results are written to a LDP container the aggregation pod at http://localhost:3000/aggregation_pod/data/ . \nEach aggregation event is an LDP resource.\nThe aggregated result is then visualised by the \nSolidLab Challenge [85](https://github.com/solidLabResearch/challenges/issues/85).\n\n## License\n\nThis code is copyrighted by [Ghent University - imec](https://www.ugent.be/ea/idlab/en) and \nreleased under the [MIT Licence](./LICENCE)\n\n## Contact\n\nFor any questions, please contact [Kush](mailto:kushagrasingh.bisen@ugent.be).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolidlabresearch%2Fsolid-stream-aggregator-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsolidlabresearch%2Fsolid-stream-aggregator-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolidlabresearch%2Fsolid-stream-aggregator-demo/lists"}