{"id":24432202,"url":"https://github.com/bayer-group/colid-setup","last_synced_at":"2026-02-25T07:45:53.718Z","repository":{"id":44462067,"uuid":"277841608","full_name":"Bayer-Group/COLID-Setup","owner":"Bayer-Group","description":"The setup repository is part of the Corporate Linked  Data Catalog - short: COLID - application. It helps setting up a local environment based on Docker Compose.","archived":false,"fork":false,"pushed_at":"2024-12-17T08:40:02.000Z","size":30801,"stargazers_count":16,"open_issues_count":2,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2026-01-12T16:57:41.807Z","etag":null,"topics":["cloud-native","colid","data-catalog","data-catalogue","elasticsearch","fair","fair-data","findable","linked-data","rdf","shacl","triplestore"],"latest_commit_sha":null,"homepage":"https://bayer-group.github.io/COLID-Documentation/","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Bayer-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-07T14:38:34.000Z","updated_at":"2025-10-06T09:07:28.000Z","dependencies_parsed_at":"2024-01-14T14:29:20.691Z","dependency_job_id":"7a55b0ea-bc95-453a-8d63-2835ab64b3d7","html_url":"https://github.com/Bayer-Group/COLID-Setup","commit_stats":{"total_commits":63,"total_committers":6,"mean_commits":10.5,"dds":0.4126984126984127,"last_synced_commit":"1f5ea454335425aa67ad23f413a1da576d4b1511"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Bayer-Group/COLID-Setup","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bayer-Group%2FCOLID-Setup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bayer-Group%2FCOLID-Setup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bayer-Group%2FCOLID-Setup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bayer-Group%2FCOLID-Setup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Bayer-Group","download_url":"https://codeload.github.com/Bayer-Group/COLID-Setup/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bayer-Group%2FCOLID-Setup/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29814415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T05:36:42.804Z","status":"ssl_error","status_checked_at":"2026-02-25T05:36:31.934Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-native","colid","data-catalog","data-catalogue","elasticsearch","fair","fair-data","findable","linked-data","rdf","shacl","triplestore"],"created_at":"2025-01-20T15:35:33.627Z","updated_at":"2026-02-25T07:45:53.670Z","avatar_url":"https://github.com/Bayer-Group.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Easy setup of the whole environment\n\nThe Setup is part of the Corporate Linked Data Catalog - short: COLID - application.\n[Here](https://bayer-group.github.io/COLID-Documentation/#/?id=introduction) you can find an introduction to the application.\nA description of all its functions is [here](https://bayer-group.github.io/COLID-Documentation/#/functional-specification).\n\nThe complete guide can be found at the following [link](https://bayer-group.github.io/COLID-Documentation/#/).\n\nThis repository helps settings up a local environment based on Docker Compose.\n\n## Installation instructions\n\n1. Install Docker Desktop for Windows from [Docker Hub](https://hub.docker.com/editions/community/docker-ce-desktop-windows/) (latest test with Docker Desktop 4.21.1)\n2. Clone this repository locally\n    ```console\n    git clone --recursive [URL to this Git repo]\n    ```\n3. Pull all changes in all submodules\n    ```console\n    git pull --recurse-submodules\n    ```\n4. Create a file `.env` in parallel to the file `docker-compose.yml` and insert the following variables (example values are shown):\n    ```\n    MESSAGEQUEUE_COOKIE=SWQOKODSQALRPCLNMEQG\n    MESSAGEQUEUE_USERNAME=guest\n    MESSAGEQUEUE_PASSWORD=guest\n    GRAPHDATABASE_USERNAME=admin\n    GRAPHDATABASE_PASSWORD=admin\n    RELATIONAL_DATABASE_ROOT_PASSWORD=dbadminpass\n    RELATIONAL_DATABASE_USERNAME=dbuser\n    RELATIONAL_DATABASE_PASSWORD=dbpass\n    MINIO_ACCESS_KEY=minio\n    MINIO_SECRET_KEY=minio123\n    MINIO_BUCKET_NAME=colid-files\n    SMTP_USERNAME=any\n    SMTP_PASSWORD=any\n    ```\n5. Run `docker-compose up` to download and build all Docker images and startup the environment\n6. Wait for docker-compose to start up\n7. Open the COLID Data Marketplace frontend (see URL below). Go to the profile menu in the upper right corner and click on \"Administration\". Open the Metadata Graph Configuration sub-menu page and click the \"Start reindex\" button in the upper right corner.\n\n\n### Known problems\n\n- While building the frontend the following error could occur. In the Dockerfiles of the frontend applications node is used with an increased heap size while building the applications `node --max_old_space_size=8000`. Try to increase this, if the error occurs.\n    ```\n    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory\n    ```\n- After starting the application a second time, the fuseki database could throw exceptions. Delete the Docker container of the fuseki database with `docker container rm fuseki`. ATTENTION: This will remove all your created data and reload the database with the initial data.\n\n- fuseki-loader/loader.sh could contain `Carriage Return` characters, remove them.\n\n## Application URLs\n\n| Component                                                | URL for Docker environment      | URL for local environment       | Username | Password |\n| -------------------------------------------------------- | ------------------------------- | ------------------------------- |--------- | -------- |\n| COLID frontend                                           | http://localhost:4200/          | http://localhost:4201/          | -        | -        |\n| Data Marketplace frontend                                | http://localhost:4300/          | http://localhost:4301/          | -        | -        |\n| COLID API Swagger documentation                          | http://localhost:51770/swagger  | http://localhost:51771/swagger  | -        | -        |\n| COLID Indexing Crawler Service API Swagger documentation | http://localhost:51780/swagger  | http://localhost:51781/swagger  | -        | -        |\n| COLID Search Service API Swagger documentation           | http://localhost:51800/swagger  | http://localhost:51801/swagger  | -        | -        |\n| COLID AppData Service API Swagger documentation          | http://localhost:51810/swagger  | http://localhost:51811/swagger  | -        | -        |\n| COLID Scheduler Service Hangfire                         | http://localhost:51820/hangfire | http://localhost:51821/hangfire | -        | -        |\n| COLID Reporting Service API Swagger documentation        | http://localhost:51910/swagger  | http://localhost:51911/swagger  | -        | -        |\n| Apache Jena Fuseki Database Webinterface                 | http://localhost:3030/          | -                               | admin    | admin    |\n| RabbitMQ Webinterface                                    | http://localhost:15672/         | -                               | guest    | guest    |\n| KGE-Editor-Frontend                                      | http://localhost:4400/          | http://localhost:4400/          | -        | -        |\n| KGE-Web-Service                                          | http://localhost:8080/          | http://localhost:8080/          | -        | -        |\n| Resource Relationship Manager-Service                    | http://localhost:51830/         | http://localhost:51831/         | -        | -        |\n| Resource Relationship Manager-Frontend                   | http://localhost:7000/          | http://localhost:7000/          | -        | -        |\n| COLID API Carrot2 Service                                | http://localhost:4305/          | http://localhost:4305/          | -        | -        |\n| Opensearch Dashboard                                     | http://localhost:5601/          | -                               | admin    | admin    |\n| Minio Browser                                            | http://localhost:9001/          | -                               | minio    | minio123 |\n## Quick Tips\n\nSome quick tips and advices to work faster.\n\n### Docker\n\nTo purge all unused or dangling images, containers, volumes, and networks run the following command:\n```console\ndocker system prune -a\n```\n\nTo remove all containers:\n```console\ndocker container rm $(docker container ls -aq)\n```\n\n### Opensearch \u0026 Kibana\n\n- After starting the first time, some indices and aliases need to be created\n- Open http://localhost:5601, go to the Dev Tools in the left panel, enter and run the following commands\n    ```\n    PUT dmp-resource-1970-01-01_00.00.00\n\n    PUT dmp-metadata-1970-01-01_00.00.00\n    {\n        \"mappings\": {\n            \"enabled\": false \n        }\n    }\n\n    POST /_aliases\n    {\n        \"actions\" : [\n            { \"add\" : { \"index\" : \"dmp-resource-1970-01-01_00.00.00\", \"aliases\" : [\"dmp-search-resource\", \"dmp-update-resource\"] } },\n            { \"add\" : { \"index\" : \"dmp-metadata-1970-01-01_00.00.00\", \"aliases\" : [\"dmp-search-metadata\", \"dmp-update-metadata\"] } }\n        ]\n    }\n    ```\n## App-wide customization for URL Domain (Optional)\nOn the Semantic Web, URIs identify not just Web documents, but also real-world objects like people and cars, and even abstract ideas and non-existing things like a mythical unicorn. We call these real-world objects or things.\nCOLID uses the native *bayer.com* as default domain in each of its URI as the project was conceived for Bayer Ag. For example - https://pid.bayer.com/kos/19050/hasLabel\n\nHowever you can also configure the custom domain in the URI if needed. In order to do that before building the docker containers, all the triples in the triplestore as well as the references to the URIs should be updated to use the custom domain. \nMultiple files references across the projects need to be changed from *bayer.com* to any custom specific domain - https://pid.orange.com/kos/19050/hasLabel\nDetails are mentioned below.\u003cbr\u003e\n| **File** \t| **Project** \t| **Variable** \t| **Comments** \t|\n|---\t|---\t|---\t|---\t|\n| [loader.sh](https://github.com/Bayer-Group/COLID-Setup/blob/master/fuseki-staging/loader.sh) \t| fuseki-staging \t| baseUrl \t| change baseUrl (example.com) as per your need in the shellscript before uploading triples \t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-AppData-Service/blob/54499e78b79a8e3e73155e5cc06f1d84b6970d1a/src/COLID.AppDataService.WebApi/appsettings.json) \t| AppData Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-Indexing-Crawler-Service/blob/974fc06f644c2526377252ee1d34430afe51dbaa/COLID.IndexingCrawlerService.WebApi/appsettings.json) \t| Indexing Crawler Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-Registration-Service/blob/3d33924b836b1d96453deeb622414513a3eaf664/src/COLID.RegistrationService.WebApi/appsettings.json) \t| Registration Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-Reporting-Service/blob/f21d73f6b6c28b2d5762fa196141c237e267c6f7/src/COLID.ReportingService.WebApi/appsettings.json) \t| Reporting Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-Search-Service/blob/4403a9c442f9d0b7d5db895513fd3286d4b63a6e/COLID.SearchService.WebApi/appsettings.json) \t| Search Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-Scheduler-Service/blob/1ccff502deff6925a9a90a26966de41597311496/src/COLID.Scheduler.Web/appsettings.json) \t| Scheduler Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [appsettings.json](https://github.com/Bayer-Group/COLID-ResourceRelationshipManager-Backend/blob/a8d590c0f87aeb9179619165ad3c9d634285f566/src/COLID.ResourceRelationshipManager/appsettings.json) \t| Resource Relationship Manager Backend Service \t| ServiceUrl,\u003cbr\u003e HttpServiceUrl \t| change both variables as per your custom domain. \u003cbr\u003e \"ServiceUrl\": \"https://pid.example.com/\",\u003cbr\u003e \"HttpServiceUrl\": \"http://pid.example.com/\"\t|\n| [environment.ts](https://github.com/Bayer-Group/COLID-Editor-Frontend/blob/2ffad4bf96d8dc683ddc05ebebdc0ab4b1bf0b13/src/environments/environment.ts), [environment.docker.ts](https://github.com/Bayer-Group/COLID-Editor-Frontend/blob/2ffad4bf96d8dc683ddc05ebebdc0ab4b1bf0b13/src/environments/environment.docker.ts) \t| Editor Frontend | baseUrl, PidUriTemplate.baseUrl\t| change baseUrl (example.com) in both sections as per your custom domain \t|\n| [environment.ts](https://github.com/Bayer-Group/COLID-Data-Marketplace-Frontend/blob/f2ee9f3a66c13a7063b1fc78e7592d77b6314c61/src/environments/environment.ts), [environment.docker.ts](https://github.com/Bayer-Group/COLID-Data-Marketplace-Frontend/blob/f2ee9f3a66c13a7063b1fc78e7592d77b6314c61/src/environments/environment.docker.ts) \t| Data Marketplace Frontend | baseUrl\t| change baseUrl (example.com) as per your custom domain \t|\n| [environment.ts](https://github.com/Bayer-Group/COLID-ResourceRelationshipManager-Frontend/blob/master/projects/frontend/src/environments/environment.ts), [environment.docker.ts](https://github.com/Bayer-Group/COLID-ResourceRelationshipManager-Frontend/blob/master/projects/frontend/src/environments/environment.docker.ts) \t| Resource Relationship Manager Frontend | baseUrl\t| change baseUrl (example.com) as per your custom domain \t|\n\n### COLID: Carrot2 clustering service\nCarrot2 clustering service is an opensource for clustering text. It can automatically discover groups of related documents and label them with short key terms or phrases. Please publish few resources in your local COLID Setup and then you can view the clusters in the Data Marketplace.\nRefer link below for more details\n\n### Minio and S3\nThe repository contains a local S3 bucket image for minio. If you want to use certain features such as \nexporting and importing excel. Please follow below steps\n - Make sure minio image is running\n - Browse to http://localhost:9001\n - Create Bucket 'colid-files'\n - Now you can use the Export and Import functionalities in Data Marketplace\n\n### Links\n\n- [Git Submodules](https://www.vogella.com/tutorials/GitSubmodules/article.html)\n- [Markdown Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet)\n- [Apache Jena Fuseki tdbloader example](https://www.csee.umbc.edu/courses/graduate/691/spring14/01/examples/jena/README.txt)\n- [wait-for-it Script](https://github.com/vishnubob/wait-for-it)\n- [Carrot2](https://github.com/carrot2/carrot2)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbayer-group%2Fcolid-setup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbayer-group%2Fcolid-setup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbayer-group%2Fcolid-setup/lists"}