https://github.com/gchq/bailo
Managing the lifecycle of machine learning to support scalability, impact, collaboration, compliance and sharing.
https://github.com/gchq/bailo
ml mlops nodejs
Last synced: 5 months ago
JSON representation
Managing the lifecycle of machine learning to support scalability, impact, collaboration, compliance and sharing.
- Host: GitHub
- URL: https://github.com/gchq/bailo
- Owner: gchq
- License: apache-2.0
- Created: 2021-02-05T13:21:21.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2026-02-04T17:14:36.000Z (5 months ago)
- Last Synced: 2026-02-05T00:25:55.134Z (5 months ago)
- Topics: ml, mlops, nodejs
- Language: TypeScript
- Homepage:
- Size: 171 MB
- Stars: 92
- Watchers: 5
- Forks: 18
- Open Issues: 29
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
[![Contributors][contributors-shield]][contributors-url] [![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url] [![Issues][issues-shield]][issues-url]
[![License][license-shield]][license-url] [![Contributor Covenant][code-of-conduct-shield]][code-of-conduct-url]
Making it easy to compliantly manage the machine learning lifecycle
Explore the docs »
Report a Bug
·
Request a Feature
## About The Project
[![Product Screen Shot][product-screenshot]](https://github.com/gchq/Bailo)
Bailo helps you manage the lifecycle of machine learning to support scalability, impact, collaboration, compliance and
sharing.
## Getting Started
### Running Locally with Docker
#### Requirements
- Node v24
- Docker / Docker Compose
#### Development Build
To run in development mode (modified files on your host machine will be reloaded into the running application):
```bash
git clone https://github.com/gchq/Bailo.git && cd Bailo
npm install
npm run certs
# This builds all the Bailo images, rerun it when you update dependencies.
docker compose build --parallel
# Then run the development instance of Bailo.
docker compose up -d
```
On first run, it may take a while (up to 30 seconds) to start up. It needs to build several hundred TypeScript modules.
These are cached however, so future starts only require a few seconds. You should access the site via
[localhost:8080](http://localhost:8080).
The registry requires a JWKS file for the token authentication with the backend application. For development, a JWKS
file is generated by running `npm run certs`. For production, the script `generateJWKS.ts` can be used to generate a
JWKS file for the public key referenced in the backend application configuration.
To stop the running services:
```bash
docker compose down
```
#### Production Build
The above `docker compose` commands can, alternatively, be run with `-f compose.prod.yml` to have a more static version
of the application running which mimics a production environment. Volumes are not dynamically mounted, the build time is
slower as the frontend is pre-compiled, and environment variables & users are more production ready. This build is also
more reliable for our cypress tests:
```bash
# Build and run the productionised instance of Bailo.
docker compose -f compose.yaml -f compose.prod.yaml --env-file prod.env up --force-recreate --build -d
# Stop the service and destroy the volumes
docker compose -f compose.yaml -f compose.prod.yaml down -v
```
The above `docker compose` approaches are preferred, however we also provide a monolithic `Dockerfile.standalone`. To
run in standalone mode, not development mode (http://localhost:8080). Not for production use:
```bash
docker build -t "bailo:standalone" -f ./Dockerfile.standalone .
docker run --name bailo -p 8080:8080 -d bailo:standalone
```
### Kubernetes Deployment
We recommend using our [helm charts](https://github.com/gchq/Bailo/blob/main/infrastructure/helm/README.md) to deploy
Bailo to Kubernetes.
We expect the administrator to provide their own forms of authentication. By default all users authenticate as 'user'.
### Testing a New Deployment
You can test out your new deployment using the example models which can be found in `frontend/cypress/fixtures`
[`minimal_binary.zip`](frontend/cypress/fixtures/minimal_binary.zip) and
[`minimal_code.zip`](frontend/cypress/fixtures/minimal_code.zip). There are also example forms in the `scripts` folder
[`minimal_model_schema.json`](backend/src/scripts/example_schemas/minimal_model_schema.json),
[`minimal_data_card_schema.json`](backend/src/scripts/example_schemas/minimal_data_card_schema.json) and
[`minimal_access_request_schema.json`](backend/src/scripts/example_schemas/minimal_access_request_schema.json).
## Logical Project Flow (Overview)
### Components
| | |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| NGINX | [![nginx][NGINX]](https://nginx.org/) |
| UI | [![TypeScript][Typescript]](https://www.typescriptlang.org/) [![Next.js][Next.js]](https://nextjs.org/) [![React][React]](https://react.dev/) [![Cypress][Cypress]](https://www.cypress.io/) |
| API | [![TypeScript][Typescript]](https://www.typescriptlang.org/) [![Next.js][Next.js]](https://nextjs.org/) [![Express.js][Express]](https://expressjs.com/) [![Vitest][Vitest]](#) |
| Docker Registry | [![Docker][Docker]](https://www.docker.com/) |
| Mongo | [![MongoDB][Mongo]](https://www.mongodb.com/) |
| S3 | [![MinIO][Minio]](https://www.min.io/) |
1. A user accesses a URL. We use [NextJS routing](https://nextjs.org/docs/routing/introduction) to point it to a file in
`frontend/pages`. `[xxx].tsx` files accept any route, `xxx.tsx` files allow only that specific route.
2. Data is loaded using [SWR](https://swr.vercel.app/). Data loaders are stored in `frontend/actions`. Each one exposes
variables to specify if it is loading, errored, data, etc.
3. Requests to the backend get routed through [express](https://expressjs.com/) within `backend/src/routes.ts`. Each
route is an array with all items being middleware except the last, which is the handler (`[...middleware, handler]`).
4. Routes interact with the database via `mongoose`, which stores models in `backend/src/models`.
## Known Issues
- _Issue: Sometimes Docker struggles when you add a new dependency._
Fix: Run `docker compose down --rmi all`
followed by `docker compose up --build`.
- _Issue: Sometimes SWR fails to install its own binary and the project will refuse to start up (development only)_
Fix: Run `npm uninstall next && npm install next`. Some users report still having issues. If so, run:
`rm -rf node_modules && rm -rf package-lock.json && npm cache clean -f && npm i`.
- _Issue: Unable to authenticate to the Docker registry / compile binaries._
Fix: Make sure that your
authentication proxy is setup to allow the 'Authorisation' header. Make sure that your application is able to access
the Docker registry internally as it will not provide user authentication.
- _Issue: `npm run build` fails in the `frontend` directory with `Error: EACCES: permission denied .`_
Fix: Run
`sudo rm -rf .next` in the `frontend` directory. This typically happens because `docker compose` has modified the
mounted volume owner.
## Usage
See [our user documentation](https://gchq.github.io/Bailo/docs)
## Contributing
If you'd like to make a contribution then the details for doing all of that are in
[CONTRIBUTING.md](https://github.com/gchq/Bailo/blob/main/CONTRIBUTING.md).
## License
Bailo is released under the Apache 2.0 Licence and is covered by Crown Copyright. See [LICENSE.txt][license-url] for
more information.
## Acknowledgments
- [Othneils's README Template](https://github.com/othneildrew/Best-README-Template)
- [Stroom's Code Contribution Guidelines](https://github.com/gchq/stroom/blob/master/CONTRIBUTING.md)
[contributors-shield]: https://img.shields.io/github/contributors/gchq/bailo.svg?style=for-the-badge
[contributors-url]: https://github.com/gchq/Bailo/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/gchq/bailo.svg?style=for-the-badge
[forks-url]: https://github.com/gchq/Bailo/network/members
[stars-shield]: https://img.shields.io/github/stars/gchq/bailo.svg?style=for-the-badge
[stars-url]: https://github.com/gchq/Bailo/stargazers
[issues-shield]: https://img.shields.io/github/issues/gchq/bailo.svg?style=for-the-badge
[issues-url]: https://github.com/gchq/Bailo/issues
[license-shield]: https://img.shields.io/github/license/gchq/bailo.svg?style=for-the-badge
[license-url]: https://github.com/gchq/Bailo/blob/main/LICENSE.txt
[code-of-conduct-shield]: https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg?style=for-the-badge
[code-of-conduct-url]: https://github.com/gchq/Bailo/blob/main/CODE_OF_CONDUCT.md
[product-screenshot]: frontend/public/images/bailo-marketplace.png
[Next.js]: https://img.shields.io/badge/Next.js-black?logo=next.js&logoColor=white
[React]: https://img.shields.io/badge/React-%2320232a.svg?logo=react&logoColor=%2361DAFB
[Express]: https://img.shields.io/badge/Express.js-%23404d59.svg?logo=express&logoColor=%2361DAFB
[Docker]: https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=fff
[Mongo]: https://img.shields.io/badge/MongoDB-%234ea94b.svg?logo=mongodb&logoColor=white
[Minio]: https://img.shields.io/badge/MinIO-C72E49?logo=minio&logoColor=fff
[NGINX]: https://img.shields.io/badge/nginx-009639?logo=nginx&logoColor=fff
[Vitest]: https://img.shields.io/badge/Vitest-6E9F18?logo=vitest&logoColor=fff
[Typescript]: https://img.shields.io/badge/TypeScript-3178C6?logo=typescript&logoColor=fff
[Cypress]: https://img.shields.io/badge/Cypress-69D3A7?logo=cypress&logoColor=fff
[MUI]: https://img.shields.io/badge/Material%20UI-007FFF?style=for-the-badge&logo=mui&logoColor=white