https://github.com/veupathdb/vdi-service
VEuPathDB Dataset Installer
https://github.com/veupathdb/vdi-service
vdi
Last synced: 28 days ago
JSON representation
VEuPathDB Dataset Installer
- Host: GitHub
- URL: https://github.com/veupathdb/vdi-service
- Owner: VEuPathDB
- License: apache-2.0
- Created: 2022-12-08T18:56:05.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2026-01-23T19:18:50.000Z (5 months ago)
- Last Synced: 2026-01-24T09:10:53.861Z (5 months ago)
- Topics: vdi
- Language: Kotlin
- Homepage:
- Size: 18.8 MB
- Stars: 0
- Watchers: 18
- Forks: 0
- Open Issues: 31
-
Metadata Files:
- Readme: readme.adoc
- License: license
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
= VEuPathDB Dataset Installer Service (VDI)
:source-highlighter: highlightjs
:toc:
:toclevels: 3
:docs-root: https://veupathdb.github.io/vdi-service
:latest-docs: {docs-root}/latest
:confluence: https://veupathdb.atlassian.net/wiki/spaces
ifdef::env-github[]
:tip-caption: :bulb:
:note-caption: :information_source:
:important-caption: :heavy_exclamation_mark:
:caution-caption: :fire:
:warning-caption: :warning:
endif::[]
ifndef::env-github[]
:icons: font
endif::[]
== Links
=== Compose Stack & Configuration
[unstyled]
* https://github.com/VEuPathDB/vdi-compose-stack
=== Documentation & Schema
Service configuration and REST API documentation in addition to JSON schema
files are generated on commit and tag, then published to GitHub pages.
link:https://veupathdb.github.io/vdi-service/[*Published Documentation Index*]
== Operation
* link:docs/logging.md[Logging]
== Development
=== Repository Structure
.Component Source
* link:project/conventions/[Gradle Build Conventions]
* link:project/common/[Shared Components]
** link:project/common/config/src/main/json/schema[Config File JSON Schema]
* link:project/core/[Core VDI Service Components]
** link:project/core/module/rest-service[REST Service]
** link:project/core/module/lane[Dataset Processing Lanes]
** link:project/core/module/daemon[Background Processes]
** link:project/core/lib/db/application[App DB Queries]
** link:project/core/lib/dataset/reconciler[Reconciler Core]
* link:project/plugin-server/[VDI Plugin Handler Server]
* link:stack-db/definition[Internal Cache DB DDL]
=== Building VDI Components
==== Docker Images
.VDI Core Server Image
[source, shell]
----
make build-image
----
.Internal Cache DB Image
[source, shell]
----
make build-db
----
==== Source Compilation
.Regenerate RAML-Based Java Code
[source, shell]
----
make raml-gen
----
.Compile VDI Core Server
[source, shell]
----
make build-core-server
----
.Compile Plugin Server
[source, shell]
----
make build-plugin-server
----
=== Common Tasks
==== Release a New Version
The core service and plugin server are versioned together, however they are
presently built differently.
===== Plugin Server
When pushing up a new git tag, a
link:.github/workflows/plugin-server-release.yml[GitHub workflow] will
automatically be triggered to build and release the plugin server. That
workflow will then open pull requests on each of the individual plugin
repositories to update their copies of the server to the newest version.
===== Core Server
The core server docker image build and push to DockerHub is handled in
https://ci.apidb.org/job/vdi-service/view/tags/[Jenkins].
==== Update Dependencies
Dependency version updates are automatically suggested once per week by
https://github.com/dependabot[GitHub's Dependabot], however if manual updates
are needed, the JVM dependencies for the full stack are declared in the
link:gradle/libs.versions.toml[dependency catalog]. +
(ref https://docs.gradle.org/current/userguide/version_catalogs.html[Gradle Version Catalogs])
==== Update Cache DB Schema
===== Relevant Locations
* link:stack-db/definition[Cache DB Schema Definitions]
* link:stack-db/migrations/src/main/sql/db-migrations[Version Migration SQL]
* link:project/core/lib/db/internal[Database JVM Code (Queries & Wrappers)]
===== Making a Schema Change
1. Make the changes to the stack-db schema definition.
2. Add a new migration directory containing migration scripts to update existing
database instances in the stack-db migrations.
3. Update the impacted queries, database managers, and POJO types in the
`core.lib.db.internal` subproject.
===== Migrations: Additional Explanation
Physically, a migration is a directory full of queries used to apply the changes
necessary for a target database schema version. The directories are named after
the version that they will migrate the database schema to, e.g. a directory
named `012` would be expected to migrate the database schema to version 12. The
directories will be checked in ascending numeric order, only being used if the
database schema version is less than the version indicated by the directory
name.
[IMPORTANT]
VDI does not update the recorded db schema version itself. The migration is
expected to perform that update, preferably as its final step/statement.
The migration scripts are baked into the service jar as resources.
On startup, VDI will test the stack-internal cache-db to fetch the current db
schema version. VDI will then iterate through the migration directories,
executing the SQL scripts contained in any directory whose numeric name is
greater than the current schema version.
[CAUTION]
VDI will not notice or apply migration scripts that have been altered in schema
versions less than or equal to the current db schema version.
== License
Copyright 2020 VEuPathDB
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at link:http://www.apache.org/licenses/LICENSE-2.0[].
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.