https://github.com/veupathdb/web-monorepo
A monorepo that contains all frontend code for VEuPathDB websites
https://github.com/veupathdb/web-monorepo
Last synced: about 2 months ago
JSON representation
A monorepo that contains all frontend code for VEuPathDB websites
- Host: GitHub
- URL: https://github.com/veupathdb/web-monorepo
- Owner: VEuPathDB
- License: apache-2.0
- Created: 2023-03-22T17:39:45.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2026-01-31T18:26:06.000Z (5 months ago)
- Last Synced: 2026-02-01T06:39:16.043Z (5 months ago)
- Language: TypeScript
- Size: 541 MB
- Stars: 3
- Watchers: 15
- Forks: 0
- Open Issues: 286
-
Metadata Files:
- Readme: README.adoc
- License: LICENSE
Awesome Lists containing this project
README
ifdef::env-github[]
:note-caption: :information_source:
endif::[]
= web-monorepo
These are instructions to work with the VEuPathDB UI either by executing a single workspace’s yarn script -using the nx task runner- (eg the mblast UI or the EDA UI), or by accessing the complete UI by starting a local site. In both cases a webpack server will be started and a new browser tab will be opened. The backend will be accessed via a remote site stated in your monorepo (packages/sites/*website) .env file.
:toc:
== Prerequisites
This project requires a system that can handle **Node.js 18.12+** (most Linux distributions from ~2019 onwards, macOS 10.15+, Windows 10+) and on which you can install https://volta.sh/[Volta] for version management (see https://docs.volta.sh/guide/#installing-volta for system requirements)
Then the installation process detailed below will be able to bootstrap the following:
- **Node.js 24+** (specified in package.json for this project)
- **Corepack** (a Node.js tool that automatically manages package manager versions)
- **Yarn 4.12.0** (which itself requires Node.js 18.12+)
== Setup
=== Setup with Volta (Recommended)
https://volta.sh/[Volta] is recommended for Node version management. It will automatically use Node 24 when working in this repository.
To set up Volta with Corepack:
[source, shell]
----
# Install volta if you haven't already
curl https://get.volta.sh | bash
----
Open a new terminal for Volta to take effect
[source, shell]
----
volta install node
volta install yarn
volta install corepack
corepack enable --install-directory ~/.volta/bin
----
This configuration allows Volta to manage Node versions while Corepack manages Yarn versions based on the `packageManager` field in `package.json`.
=== Install yarn dependencies
From anywhere in the repo:
[source, shell]
----
yarn
----
*Note:* This repository uses immutable installs for security. The `yarn.lock` file must be committed to git and cannot be modified by `yarn install`. Commands like `yarn add ` and `yarn remove ` work normally and will update the lockfile as expected. This prevents accidental mass upgrades (e.g., deleting and regenerating yarn.lock would upgrade many packages to newer versions). All dependency upgrades should be intentional and reviewed.
=== Set up git hooks
A pre-commit hook will run `prettier` to normalise formatting of files touched by the commit. This irons out inter-developer IDE formatting differences and is essential to avoid confusion and uninformative formatting-only diffs in the history.
Run this once after cloning to enable pre-commit formatting checks:
[source, shell]
----
npx husky install
----
NOTE: `yarn install` only runs the `postinstall` script in `package.json` when the dependency tree changes, so new clones won't get the hooks automatically. **You must run `npx husky install` manually once per clone.**
== Development on Cedar Server
Follow the instructions above in a regular terminal (not a dev site environment after sourcing `etc/setenv`) to set up Volta, Node, Yarn and Corepack.
To work smoothly in a dev site environment (any time after sourcing `etc/setenv`), you need to address a PATH configuration issue that can shadow Volta's yarn with an older system version.
=== The Issue
Intranet dev site configurations use `etc/setenv` scripts that prepend legacy paths to PATH. This causes the system's old yarn to take precedence over Volta's managed version, breaking the build.
=== The Fix
Choose one of these approaches:
*Option 1: Using a helper shell function in regular workflow (Recommended)*
Add this function to your shell profile (`~/.bashrc`, `~/.zshrc`, etc.):
[source, bash]
----
# Prepend directory to PATH, moving it to front if already present elsewhere
path_prepend() {
case "$PATH" in "$1":*|"$1"|"") PATH="$1"; return ;; esac # already first or empty
PATH=":$PATH:" # add sentinels for matching anywhere
PATH="${PATH//:$1:/:}" # remove exact match (colon-bounded)
PATH="$1$PATH" # prepend (leading colon already exists)
PATH="${PATH%:}" # strip trailing colon
}
----
Then update your standard workflow to include one additional step after sourcing `etc/setenv`:
[source, shell]
----
# 1. Start terminal session
# 2. cd /var/www/PlasmoDB/plasmo.bmaccallum
# 3. source etc/setenv
# 4. Restore Volta to front of PATH:
path_prepend ~/.volta/bin
----
This avoids modifying shared files and is easy to remember.
*Option 2: Edit etc/setenv directly*
Edit your dev site's `etc/setenv` file (typically `$BASE_GUS/etc/setenv`) to append instead of prepend:
[source, bash]
----
# Change from:
export PATH=$GUS_HOME/bin:$PROJECT_HOME/install/bin:$PATH
# To:
export PATH=$PATH:$GUS_HOME/bin:$PROJECT_HOME/install/bin
----
This ensures Volta's yarn (from `~/.volta/bin`) remains first in PATH.
=== Package Manager Verification
*IMPORTANT:* This project requires Yarn. Using `npm install` will fail with dependency resolution errors because this monorepo uses Yarn-specific features (workspace protocol, etc.) that npm doesn't support.
Corepack enforces the correct Yarn version automatically via the `packageManager` field in `package.json`. If you see errors about wrong Yarn version, refer to the setup instructions above.
== Overview
This repository is a "monorepo", using the https://nx.dev[nx] build system and https://yarnpkg.com/[yarn@4] dependency manager. The source code is divided into one of three types of packages: "config", "lib", and "site".
- "config" packages (`./packages/configs`) include various build and development configurations and tools.
- "lib" packages (`./packages/libs`) include standalone library and webapp source code.
- "site" packages (`./packages/sites`) include complete website source code.
Many "lib" and "site" packages include development tools, such as development servers, test scripts, etc.
Many commands require a reference to the package name. All package names are currently prefixed with `@veupathdb/`. For example, the eda package name is `@veupathdb/eda`.
The repository is currently configured as a "package based repository". This is subject to change, in the future (see https://nx.dev/concepts/integrated-vs-package-based to read about the differences between package based and integrated repos).
== Common Tasks
The following tasks are common performed by developers. This serves as a reference guide, and is not exhaustive in any way. If you feel something is missing, create an issue, or open a pull request.
_All commands are expected to be run in the repository's root directory, unless otherwise specified._
=== Start a local dev site
"Site" projects are stored in the `packages/sites` directory. Each one corresponds to a "cohort". For example, `packages/sites/genomics-site`
contains the code used to build the client code for a genomics website.
Each cohort contains a `.env.sample` file. Copy this to a sibling `.env` file. You may need to modify some values, such as usernames,
passwords, etc. You can also specify a website to use for various services. Typically a deployed QA site will suffice, but you can also run a "local backend" via various methods. (TODO: link to relevants docs/repos).
Once you have created a `.env` file, you can run the local dev site with the command:
[source, shell]
----
yarn nx start
----
For example, if you want to run a local clinepi site, you would run the command:
[source, shell]
----
yarn nx start @veupathdb/clinepi-site
----
Once the website has been compiled, the dev server will output some build statistics and automatically open a browser tab. You can kill the local dev server with `CTRL-C` in the terminal where you started it.
=== Modifying code (and updating a running local dev site)
When a local dev site is running, it will detect when build dependencies are updated and reload the active webpage.
__Note: the following refers to code tracked by the monorepo, and not third-party dependencies from npm.__
There are two types of souce code that can be updated:
1. Source code within the package being served.
2. Source code within a dependent package.
Changes to code within the package being served will be detected automatically. The local dev service should reompile the affected module and reload the website without intervention.
Changes to code within a dependent workspace package will require a build command for the local dev server to detect the change.
For example, if you change code in `packages/libs/eda`, you will need to run this command in a new terminal window, from the repository root:
[source, shell]
----
yarn workspace @veupathdb/eda build-npm-modules
----
Once this command completed, the running dev server will see the updated build artifacts, recompile the website, and reload the webpage. There are some cases where this might not work as expected, such as if the recompile step fails. In those cases, you may need to restart the local dev server. You can monitor the progress of the recompilation step in the terminal where you started the dev server.
=== View workspace dependency graph
From the root `package.json`:
[source, shell]
----
yarn nx graph
----
=== Execute a workspace's yarn script using the `nx` task runner
From the root `package.json`:
[source, shell]
----
yarn nx run :
----
For example, you can start the MultiBLAST dev server by running
[source, shell]
----
yarn nx run @veupathdb/multi-blast:start
----
=== Rebuild dependencies when running a development server
When running a development server (such as `yarn nx start @veupathdb/eda` or `yarn nx start @veupathdb/clinepi-site`),
use the following command to rebuild changes made to dependencies, and to have the dev site reload with the changes:
[source, shell]
----
cd packages/libs/
yarn build-npm-modules
----
**Note: You may need to manually reload your website to see the changes the first time.**
_Using the equivalent `nx` command (`yarn nx build-npm-modules @veupathdb/`) has proven inadequate in this scenario._
== Notes on individual packages
=== EDA dev server
Directory: `packages/libs/eda`
You will need to configure the server with a `packages/libs/eda/.env.local` file that sets various environment variables.
For more documentation see the link:packages/libs/eda/README.md[package README] and link:packages/libs/eda/.env.local.sample.localservices[this sample file].
=== VEuPathDB sites
Directory: `packages/sites/{site name}-site`
Copy the `packages/sites/{site name}-site/.env.sample` file to `packages/sites/{site name}-site/.env` and configure the new file with passwords and the desired backend for the site.
Run `yarn` to update dependencies if necessary.
Run the command `yarn nx start @veupathdb/{site name}-site`. For example, to run the ortho site use `yarn nx start @veupathdb/ortho-site`.
== IDE hints
=== emacs tide
If it is showing errors for tsx imports (especially in `eda`) and
`tide-verify-setup` mentions tsserver version 3.x then it is time to
upgrade emacs tide (to, at time of writing 4.5.4):
[source]
----
M-x package-reinstall tide
----
== Client Bundle Server 🐉
**Warning. There be dragons! This section has not been maintained for a long time.**
The Client Bundle Server is a Docker image based on NGINX that is used to serve
VEuPathDB client code over HTTP.
As the client code comes in 2 flavors (bundles), legacy and modern, this NGINX
server has an internal path rewrite based on the requesting browser's user agent
string to the appropriate client bundle component on request.
This means using a modern browser, requesting the file
`genomics/site-client.bundle.js` will cause the server to actually return
`modern/genomics/site-client.bundle.js` whereas requesting that same file from
an older or unsupported browser (such as CURL or Postman) the server will return
`legacy/genomics/site-client.bundle.js`.
=== Browsers
Whether a browser is considered modern or legacy is dependent on the version of
the browser compared to a RegEx constructed by the
link:https://github.com/browserslist/browserslist-useragent-regexp[browserslist-useragent-regexp]
library using the input query constructed in the
link:packages/configs/browserslist-config[browserslist-config] package of
this repo. (See link:packages/configs/browserslist-config/index.js[index.js]
for the raw queries)
=== Docker Image
The docker image is based on NGINX-Perl and includes NodeJS for executing a
script based on
link:https://github.com/browserslist/browserslist-useragent-regexp[browserslist-useragent-regexp]
that determines which path a specified file should be served from.
The image build is multi-staged with the first stage compiling primary contents
of this repository, and the second stage setting up NGINX and the secondary JS
script included in the link:docker/[docker] directory
(link:docker/makeSupportedBrowsersScript.js[makeSupportedBrowsersScript.js]).
=== Paths
Content is served from the following paths from the root path used to reach a
running instance of the built Docker image:
[source]
----
{URL}/clinepi/{target-file}
{URL}/genomics/{target-file}
{URL}/mbio/{target-file}
{URL}/ortho/{target-file}
----
These paths correspond to the following container internal paths:
[source]
----
/var/www/legacy/clinepi/{target-file}
/var/www/modern/clinepi/{target-file}
/var/www/legacy/genomics/{target-file}
/var/www/modern/genomics/{target-file}
/var/www/legacy/mbio/{target-file}
/var/www/modern/mbio/{target-file}
/var/www/legacy/ortho/{target-file}
/var/www/modern/ortho/{target-file}
----
=== Testing
The Docker image may be tested locally by performing the following steps from
the link:docker/[docker/] subdirectory:
. Build and Start the image:
+
[source, shell]
----
make docker-build
make docker-run
----
. Using your favorite HTTP request making tool such as Postman, CURL, or a web
browser, make a request to
http://localhost/genomics/site-client.bundle.js.LICENSE.txt . If the service is
working you should receive a LICENSE text file's contents as the response with
a 200 status code. If it is not working you will receive a 403 or 404 error.