{"id":15017393,"url":"https://github.com/dzhw/metadatamanagement","last_synced_at":"2025-10-13T21:42:23.984Z","repository":{"id":1950026,"uuid":"39431147","full_name":"dzhw/metadatamanagement","owner":"dzhw","description":"Metadatamanagement (MDM) - Data Search for Higher Education Research and Science Studies","archived":false,"fork":false,"pushed_at":"2025-09-17T09:01:06.000Z","size":304465,"stargazers_count":25,"open_issues_count":302,"forks_count":7,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-09-17T10:31:53.623Z","etag":null,"topics":["angular-material","angularjs","aws-fargate-application","docker-compose","elasticsearch","github-actions","grunt","jasmine","junit","karma","maven","mongodb","readthedocs","robotframework","saucelabs","snyk-io","spring-boot","spring-data-mongodb","spring-security","terraform"],"latest_commit_sha":null,"homepage":"https://metadata.fdz.dzhw.eu","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dzhw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2015-07-21T07:35:40.000Z","updated_at":"2025-06-10T12:01:57.000Z","dependencies_parsed_at":"2024-09-12T01:42:53.477Z","dependency_job_id":"19eefae9-e58a-465d-8cbe-d125d512372d","html_url":"https://github.com/dzhw/metadatamanagement","commit_stats":{"total_commits":8379,"total_committers":38,"mean_commits":220.5,"dds":0.6647571309225444,"last_synced_commit":"520372a7a0bc0cbb9701668c6b2fac23eed35005"},"previous_names":[],"tags_count":127,"template":false,"template_full_name":null,"purl":"pkg:github/dzhw/metadatamanagement","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzhw%2Fmetadatamanagement","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzhw%2Fmetadatamanagement/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzhw%2Fmetadatamanagement/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzhw%2Fmetadatamanagement/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dzhw","download_url":"https://codeload.github.com/dzhw/metadatamanagement/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzhw%2Fmetadatamanagement/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279017052,"owners_count":26085951,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["angular-material","angularjs","aws-fargate-application","docker-compose","elasticsearch","github-actions","grunt","jasmine","junit","karma","maven","mongodb","readthedocs","robotframework","saucelabs","snyk-io","spring-boot","spring-data-mongodb","spring-security","terraform"],"created_at":"2024-09-24T19:50:24.001Z","updated_at":"2025-10-13T21:42:23.951Z","avatar_url":"https://github.com/dzhw.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://github.com/dzhw/metadatamanagement/workflows/Build%20and%20Deploy/badge.svg)](https://github.com/dzhw/metadatamanagement/actions) [![Documentation Status](https://readthedocs.org/projects/metadatamanagement/badge/?version=latest)](https://metadatamanagement.readthedocs.io/de/latest/)\n[![Known Backend Vulnerabilities](https://snyk.io/test/github/dzhw/metadatamanagement/badge.svg?targetFile=pom.xml)](https://snyk.io/test/github/dzhw/metadatamanagement?targetFile=pom.xml\n)[![Known Frontend Vulnerabilities](https://snyk.io/test/github/dzhw/metadatamanagement/badge.svg?targetFile=package.json)](https://snyk.io/test/github/dzhw/metadatamanagement?targetFile=package.json\n)[![codecov](https://codecov.io/gh/dzhw/metadatamanagement/branch/development/graph/badge.svg)](https://codecov.io/gh/dzhw/metadatamanagement)[![Mergify Status](https://gh.mergify.io/badges/dzhw/metadatamanagement.png?style=cut)](https://mergify.io)\n[![DOI](https://zenodo.org/badge/39431147.svg)](https://zenodo.org/badge/latestdoi/39431147)\n\n[![Sauce Test Status](https://app.saucelabs.com/browser-matrix/andydaniel.svg?cachebuster=1234)](https://app.saucelabs.com/u/andydaniel)\n# Metadatamanagement (MDM)\n\nThe MDM holds the metadata of the data packages which are available in our Research Data Center [FDZ](https://fdz.dzhw.eu). It enables researchers to browse our data packages before signing a contract for using the data. \n\n# Developing the MDM system\n\nPlease checkout the development branch before starting to code and create a new branch starting with your username followed by the backlog items issue number you will be working on:\n\n    git checkout development\n    git checkout -b rreitmann/issue1234\n\nBefore you can build this project, you must install and configure the following dependencies on your machine:\n\n1.  Java: You need to install java 15 sdk on your system. On Ubuntu you should use [SDKMAN!][] (`sdk install java 15.0.2.hs-adpt`)\n2.  Maven: You need to install maven 3.6.1 or above on your system. On Ubuntu you should use [SDKMAN!][] (`sdk install maven`)\n3.  [Node.js][]: Node.js 16 and npm (coming with node.js) are required as well. On Ubuntu you should install node using [NVM][] (`nvm install v16`)\n\nOn Windows, `patch.exe` has to exist in the PATH. It is distributed as part of git bash, or can be downloaded manually from [GnuWin32][].\n\n## Running on your local machine\nMake sure that you have read-write-access on the **data** directory (in your project directory) for Elasticsearch and MongoDB. Specifically **Mac** users need to run the following command to create all data directories before bringing up the containers for the first time:\n```bash\nmkdir -p data/elasticsearch/data data/mongodb/db data/mongodb/logs\n```\nOtherwise your Docker Host will attempt to change permissions on the directories and fail.\n\nUse `docker-compose up` to create all containers initially. MongoDB and Elasticsearch will be listening on their default ports. MailDev will show all locally sent email on 8081 and the identity-provider can be setup on port 8082. Any time after that use either `docker-compose up` or `docker-compose start`.\n\nIn case elasticsearch does not start successfully, you might need to increase its memory limit\n`mem_limit: 512m`, e.g. to `1024` (this change required removing and re-building the container).\n\nYou can get a MongoDB dump and restore it locally:\n```\n$ wget https://metadatamanagement-public.s3.eu-central-1.amazonaws.com/20220926_metadatamanagement_e2e.zip\n$ unzip 20220926_metadatamanagement_e2e.zip\n$ mv dump/metadatamanagement data/mongodb/db/\n$ docker exec -it mongodb bash\nmongo$ cd /data/mongodb/db\nmongo$ mongorestore ./metadatamanagement --db=metadatamanagement\nmongo$ exit\nrm -r dump\n```\n\nYou will need to setup your `~/.m2/settings.xml` so that maven can download a dependency from Github:\n\n```xml\n \u003csettings xmlns=\"http://maven.apache.org/SETTINGS/1.0.0\"\n    xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n    xsi:schemaLocation=\"http://maven.apache.org/SETTINGS/1.0.0\n                        http://maven.apache.org/xsd/settings-1.0.0.xsd\"\u003e\n    \u003cservers\u003e\n      \u003cserver\u003e\n        \u003cid\u003egithub\u003c/id\u003e\n        \u003cusername\u003e${GITHUB_USERNAME}\u003c/username\u003e\n        \u003cpassword\u003e${GITHUB_TOKEN}\u003c/password\u003e\n      \u003c/server\u003e\n    \u003c/servers\u003e\n  \u003c/settings\u003e\n```\n\nRun `mvn` **first** to start the Spring backend and to **make sure** the frontend Angular constants module has been generated by the Maven Plugin. Run `npm --prefix mdm-frontend start` to start the Angular Frontend.\n```shell\nmvn clean install -f maven-plugin/pom.xml\nmvn spring-boot:run\n```\n\nIn order for all external services to work on your local machine, you need to set the following environment variables\nwhen starting the Spring Boot application:\n* `DARA_ENDPOINT` (regular endpoint for registering projects)\n* `DARA_USERNAME`\n* `DARA_PASSWORD`\n* `DARA_PID_ENDPOINT` (endpoint for registering variables)\n* `DARA_PID_USERNAME`\n* `DARA_PID_PASSWORD`\n\nStarting the application from the command line would look like this:\n```sh\nDARA_ENDPOINT=\"https://dara.service/projects/\" \\\nDARA_USERNAME=\"bob\" \\\nDARA_PASSWORD=\"secret\" \\\nDARA_PID_ENDPOINT=\"https://dara.service/variables/\" \\\nDARA_PID_USERNAME=\"alice\" \\\nDARA_PID_PASSWORD=\"pid-secret\" \\\n    mvn spring-boot:run\n```\nUse `sensitive-variables.tf` to fill in the **correct credentials**.\n\nIf you run the backend on your machine for the first time, or you have restored a\nmongodb dump, then you need to setup/reindex the elasticsearch indices. Therefore, login as admin to the application,\ngo to `Administration` on the left, navigate to `External Services` and then\nclick the red button `Reindex` for the Elasticsearch service. Reindexing can take up to 1 hour.\n\nIf you want to build a docker image for the metadatamanagement server app you can run\n\n    mvn deploy\n\nThis image can be run with all its dependent containers by\n\n    docker-compose -f docker-compose.yml -f docker-compose-app.yml up -d --build\n\n## Building for the dev environment\n\nOur CI pipleline will do some automatic checks and tests and it will optimize the metadatamanagement client for the dev environment. So before pushing to Github in order to be sure you won't fail the build you should run:\n\n    mvn -Pdev clean verify\n\nThis will concatenate and minify CSS and JavaScript files using grunt. It will also modify the `index.html` so it references\nthese new files.\n\nWe test our project continuously with the Robot Framework. Test Developers can get further info [here](https://github.com/dzhw/metadatamanagement/wiki/Robot-Framework).\n\n\n## Tweet \n\n#### Authentication\nWhen an analysis package or data package is released with version \u003e=1.0.0, the user can optionally post a message about \nthe release on X (formerly Twitter).\n\nTo set this up, you need to have an X Developer Account (Free Access Level) and your projects' api \ncredentials `consumer key` and `consumer secret`. Be aware that the current [Free Access Level](https://developer.twitter.com/en/docs/twitter-api)\nis limited to 50 tweets/24h; 1,500 tweets/month; 1 environment; 1 project.\n\nMake your credentials `consumer key` and `consumer secret` accessible by the `application.yml` of the current stage through\n`sensitive_variables.tf` just like other highly sensitive data. \n\n```shell\n[application.yml]\n...\ntweet:\n  consumerkey: ${vcap.services.tweet.credentials.consumerkey}\n  consumersecret: ${vcap.services.tweet.credentials.consumersecret}\n  oauthtoken: ${vcap.services.tweet.credentials.oauthtoken}\n  oauthtokensecret: ${vcap.services.tweet.credentials.oauthtokensecret}\n  ...\n```\n\nCreate your `oauthtoken` and `oauthtokensecret` by following the three steps of the Postman\nTwitter examples: [Twitter OAuth 1.0a flow test](https://www.postman.com/twitter/workspace/twitter-s-public-workspace/request/9956214-5bd6ebb1-9d79-4456-a9a6-22ead4a41625).\n1. step `oauth/request_token`: \n\nExecute the request with your consumer key and consumer secret from your Developer Account. \nAn `OAUTH_TOKEN_FROM_STEP1` and `OAUTH_TOKEN_SECRET_FROM_STEP1` will be returned.\n\n2. step `oauth/authorize`: \n\nVisit `https://api.twitter.com/oauth/authorize?oauth_token={OAUTH_TOKEN_FROM_STEP1}\u0026oauth_token_secret={OAUTH_TOKEN_SECRET_FROM_STEP1}\u0026oauth_callback_confirmed=true` with `OAUTH_TOKEN_FROM_STEP1` and `OAUTH_TOKEN_SECRET_FROM_STEP1` from the first step,\nand authenticate your app.\n\nAfter being redirected to X, open the network, and copy the values for `oauth_token` as `OAUTH_TOKEN_FROM_STEP2`\nand `oauth_token` as `OAUTH_VERIFIER_FROM_STEP2` from this GET request\n```shell\nGET 'http://twitter.com/?oauth_token={OAUTH_TOKEN_FROM_STEP2}\u0026oauth_verifier={OAUTH_VERIFIER_FROM_STEP2}`'\n```\n\n3. step `oauth/access_token`:\n\nInsert the `OAUTH_TOKEN_FROM_STEP2` and `OAUTH_VERIFIER_FROM_STEP2` from step 2 into the third \nrequest (If you are using Postman like\nthe linked Twitter example, select `No Auth` instead of `OAuth 1.0`).\n```shell\nPOST 'https://api.twitter.com/?oauth_token={OAUTH_TOKEN_FROM_STEP2}\u0026oauth_verifier={OAUTH_VERIFIER_FROM_STEP2}' \n```\n\nAdd the returned values for `oauth_token` and `oauth_token_secret` from step 3 to the `sensitive_variables.tf`.\n\nFor further details also see [Authentication OAuth FAQ](https://developer.twitter.com/en/docs/authentication/faq).\n\n# Big Thanks \n\nCross-browser Testing Platform and Open Source :heart: Provided by [Sauce Labs][saucelabs]\n\nContinuous Integration Platform provided by [Github Actions][GithubActions]\n\n[saucelabs]: https://saucelabs.com\n[Node.js]: https://nodejs.org/\n[Grunt]: http://gruntjs.com/\n[NVM]: https://github.com/creationix/nvm\n[SDKMAN!]: http://sdkman.io/install.html\n[GithubActions]: https://github.com/dzhw/metadatamanagement/actions\n[GnuWin32]: http://gnuwin32.sourceforge.net/packages/patch.htm\n\n[![forthebadge](http://forthebadge.com/images/badges/built-by-developers.svg)](http://forthebadge.com)  [![forthebadge](https://forthebadge.com/images/badges/built-with-science.svg)](https://forthebadge.com)\n [![forthebadge](https://forthebadge.com/images/badges/60-percent-of-the-time-works-every-time.svg)](https://forthebadge.com) [![forthebadge](http://forthebadge.com/images/badges/uses-badges.svg)](http://forthebadge.com) [![forthebadge](https://forthebadge.com/images/badges/makes-people-smile.svg)](https://forthebadge.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdzhw%2Fmetadatamanagement","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdzhw%2Fmetadatamanagement","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdzhw%2Fmetadatamanagement/lists"}