{"id":24568863,"url":"https://github.com/aeon0/mlpipe-manager","last_synced_at":"2025-07-29T16:38:36.315Z","repository":{"id":41792267,"uuid":"188952304","full_name":"aeon0/MLPipe-Manager","owner":"aeon0","description":"NodeJS server backend to distribute \u0026 manage Keras/Tensorflow trainings","archived":false,"fork":false,"pushed_at":"2022-12-30T17:44:06.000Z","size":1344,"stargazers_count":0,"open_issues_count":13,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-17T05:31:53.619Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aeon0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-28T04:12:37.000Z","updated_at":"2021-10-28T05:48:32.000Z","dependencies_parsed_at":"2023-01-31T12:31:38.284Z","dependency_job_id":null,"html_url":"https://github.com/aeon0/MLPipe-Manager","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aeon0/MLPipe-Manager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aeon0%2FMLPipe-Manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aeon0%2FMLPipe-Manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aeon0%2FMLPipe-Manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aeon0%2FMLPipe-Manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aeon0","download_url":"https://codeload.github.com/aeon0/MLPipe-Manager/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aeon0%2FMLPipe-Manager/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265483886,"owners_count":23774290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-23T14:55:35.794Z","updated_at":"2025-07-16T05:13:36.318Z","avatar_url":"https://github.com/aeon0.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"mlpipe_icon.png\" width=\"500\"\u003e\n\n# MLPipe Manager\n\nNodeJS server backend to train \u0026 manage Keras/Tensorflow trainings.\n\u003c/br\u003e\nThe MLPipe system also provides a lightwight python package to send your Keras/Tensorflow trainings to the backend, [MLPipe-Trainer](https://github.com/j-o-d-o/MLPipe-Trainer). As well as a GUI to easly interact with the Manager, [MLPipe-Viewer](https://github.com/j-o-d-o/MLPipe-Viewer).\n\n## The Features\n\n- Manage trainings of Machine Learning models (Tensorflow/Keras)\n- Visualize training results\n- Train on remote machines (e.g. AWS instances)\n  - Automize uploading training code and training data\n  - Get results automatically back via the MLPipe-Manager API endpoints\n  - (AWS) close spot instances once training is done\n- Create and manage keys to automatically connect to remote servers\n- User Managment with Developer and Admin roles\n\n## Getting started\n\n- Currently tested with LTS Node v10.16.3\n- Spin up your server, connect to it and checkout the MLPipe-Manager repository\n- If you have not yet, install MongoDB. Docu [here](https://github.com/j-o-d-o/MLPipe-Manager/docs/install_mongodb.md)\n- Make a copy of the .env.template file, rename it to .env and adjust the fields: `MONGODB_URI, MONGODB_URI_TEST, USER_TOKEN`\n\n### Setup AWS credentials (optinal)\nNote: This is optional in case you want to terminate AWS spot requests automatically after training is finished\u003c/br\u003e\u003c/br\u003e\nCreate the file `~/.aws/credentials` (Linux/Mac) or `C:\\Users\\USERNAME\\.aws\\credentials` (Windows):\n```ini\n[default]\naws_access_key_id = your_access_key\naws_secret_access_key = your_secret_key\n```\nFind/create this key with: \u003c/br\u003e\nlog in to aws console -\u003e click on your user name -\u003e My Security Credentials -\u003e Access keys\u003c/br\u003e\nIn the .env file you can specify the region to be used e.g. `AWS_REGION=eu-west-3`.\n\n### Install packages and start server\n```bash\n# Install packages\n\u003e\u003e npm install\n# Build the typescript project (typescript needs to be installed: \u003e\u003e npm install typescript -g)\n\u003e\u003e tsc\n# Start with NODE_ENV=prod and node\n\u003e\u003e npm start\n```\nTo run it with [pm2](http://pm2.keymetrics.io/), run `\u003e\u003e pm2 --name MLPipe-Manager start npm -- start`\u003c/br\u003e\nNote, in case you are not using any webserver and want to access the api through http://ip:port, dont forget to open the port in case you use any firewall. Alternatively, server the API via a webserver and domain.\n\n### Create admin user\nCurrently the creation of new users is only possible by admins. As there is no MLPipe user in the beginning, there is a script to create a first admin user:\n```bash\n\u003e\u003e cd scripts\n# positional arguments: [0]: user name, [1]: user email, [2]: password\n\u003e\u003e node create_admin_user.js UserName my@email.com my_password\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faeon0%2Fmlpipe-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faeon0%2Fmlpipe-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faeon0%2Fmlpipe-manager/lists"}