{"id":22777272,"url":"https://github.com/omicsdi/ddi-web-service","last_synced_at":"2025-10-07T22:47:01.904Z","repository":{"id":30046759,"uuid":"33595920","full_name":"OmicsDI/ddi-web-service","owner":"OmicsDI","description":"Web service of the DDI Project","archived":false,"fork":false,"pushed_at":"2023-03-29T12:16:52.000Z","size":29946,"stargazers_count":0,"open_issues_count":10,"forks_count":2,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-10-07T22:47:01.411Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://wwwdev.ebi.ac.uk/Tools/ddi/ws/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OmicsDI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-04-08T08:58:17.000Z","updated_at":"2020-04-11T12:09:08.000Z","dependencies_parsed_at":"2024-12-11T19:13:47.600Z","dependency_job_id":"d1eebf54-c4fd-4d3c-ad2e-da249102d80f","html_url":"https://github.com/OmicsDI/ddi-web-service","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OmicsDI/ddi-web-service","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmicsDI%2Fddi-web-service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmicsDI%2Fddi-web-service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmicsDI%2Fddi-web-service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmicsDI%2Fddi-web-service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OmicsDI","download_url":"https://codeload.github.com/OmicsDI/ddi-web-service/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmicsDI%2Fddi-web-service/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278861032,"owners_count":26058632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-11T19:13:38.593Z","updated_at":"2025-10-07T22:47:01.871Z","avatar_url":"https://github.com/OmicsDI.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"Data Discovery Index web-service\n=========================\n\nA RESTful web service providing access to the European Bioinformatics Institute Data Discovery Index.\n\nIn-dev prototype deployed here: (soon). \n\n#### End points\n\nDetailed and functional documentation will be available thanks to [Swagger](https://github.com/swagger-api) once the\nweb-service is deployed. As a summary, these are the available end-points for the web service:\n\n* `stast`: statistics about the data in the DDI project. \n* `datatset`: information about datasets, such as search, terms frequency etc.\n\n\n\nCurrently kubernetes setup is in dev branch \n\nCI/CD process for Omicsdi Docker deployement on kubernetes :- \n\n\nCI/CD process is followed for OmicsDI development using docker on kubernetes.\n\nIn OmicsDI, we separated our web services into multiple different services, including api web service, profile service, frontend web service, server side rendering web service, etc. \n\nWe also have multiple different environments for production as well as testing. \n\nSo, to make easier to apply new changes as well as separate the development and deployment, we have implemented our Omicsdi continuous delivery service. \n\nSource code of this service is located at https://github.com/OmicsDI/ddi-continuous-delivery. \n\n\nThe flow of this service is depicted as below:\n\n![Image of Omicsdi CI/CD](https://github.com/OmicsDI/ddi-web-service/blob/master/OmicsDI-CD.jpg)\n\nAs you can see in the flow chart above, every time the user merges a pull request from one branch to another, a web hook will be triggered to call our Continuous Delivery web service. \n\nThe web service will validate the request and then will submit a build job into our job queue (Redis). \n\nA CD worker will then pick up the job, clone/checkout the source code from Github with the branch described from the job information, build a Docker image based on the Dockerfile in the source code root, upload the generated image into Dockerhub and then finally, redeploy the container. \n\nThat is an overview of our CD, now we're going deep into each of those steps to see how it works and how to configure it.\n\nContinuous Delivery Workers\n\nOnce the build job has been submitted into our Job queue, then worker will pick up the job. \n\nDepending on the job metadata, it will clone/checkout the git repository with the branch that is the target branch in the pull request. \n\nAfter checked out the target branch, this worker will then build docker image based on the Dockerfile that located in the git source root using the docker agent of the host Kubernetes itself (https://github.com/OmicsDI/ddi-continuous-delivery/blob/master/k8s/ddi.continuous.delivery.worker.yaml#L51). \n\nThe build tag name will be using the format of omicsdi/\u003cproject name\u003e:omicsdi.\u003cbranch name\u003e.\u003cbuild id\u003e.\n\nAfter the docker image is built, the generated docker image will be uploaded into Dockerhub using our omicsdi deployment account, which described in Kubernetes secret as well. \n\nOnce deployed, it moves to the last phase of Container Redeployment.\n\nContainer Redeployment\n\nNow, depending on the project, you may or may not need to redeploy your changes. \n\nDataflow for instance, don't need to apply changes straight away because the next time it spin up a container, it will automatically pull new images from Dockerhub.\n\nBut for web service like ddi-web-service, we have to tell Kubernetes to redeploy the container. \n\nSo, If your project need to redeploy, define the k8s folder with the Kubernetes manifest file name's format as follow: \u003cname\u003e.\u003cenvironment\u003e.yaml (example: https://github.com/OmicsDI/ddi-web-service/tree/dev/k8s)\n\nIf your project don't need to redeploy, do not define the k8s folder\n\nNow, if you want to redeploy the container, you have to change the image tag described in the Kubernetes manifest, otherwise, \n\nKubernetes will not recognise that something has changed and will not redeploy the container (https://github.com/kubernetes/kubernetes/issues/33664). \n\nSo, in order to workaround this issue, we have defined the default tag name, which is omicsdi.dev.01. \n\nIt acts as a placeholder and will be replaced each time the container is being redeployed (https://github.com/OmicsDI/ddi-web-service/blob/dev/k8s/ddi.webservice.dev.yaml#L19). \n\nDeploying our Continuous Delivery services stack\n\nNormally, the CD will work with all namespaces in our Kubernetes cluster, so, I would prefer to deploy it into the default's namespace, for better organising our services. \n\nThe following steps will help you to deploy our CD into your Kubernetes cluster:\n\n1.  Clone the Continuous Delivery project\n\nKubernetes\n```\ngit clone git@github.com:OmicsDI/ddi-continuous-delivery.git\ncd ddi-continuous-delivery\n```\n\n2. Build the Docker images (Skip if you haven't changed anything in the code)\n\nKubernetes\n```\ndocker build -t omicsdi/ddi-continuous-delivery:latest .\ndocker build -t omicsdi/ddi-continuous-delivery-worker:latest -f Dockerfile.worker .\n```\n\n3. Update your DNS\n\nKubernetes\n```\nFile: ddi.continuous.delivery.ingress.yaml\nReplace: host: cd.omicsdi.org -\u003e host: \u003cyour host name or IP\u003e\n```\n  \n4. Deploy the services:\n\nKubernetes\n```\nkubectl apply -f k8s/ddi.redis.yaml\nkubectl apply -f k8s\n```\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomicsdi%2Fddi-web-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fomicsdi%2Fddi-web-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomicsdi%2Fddi-web-service/lists"}