{"id":26481488,"url":"https://github.com/infuseai/primehub-job","last_synced_at":"2025-03-20T03:20:53.802Z","repository":{"id":48015021,"uuid":"281642380","full_name":"InfuseAI/primehub-job","owner":"InfuseAI","description":null,"archived":false,"fork":false,"pushed_at":"2021-08-24T05:08:34.000Z","size":166,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-05-22T05:03:00.363Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InfuseAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-22T10:05:55.000Z","updated_at":"2022-11-14T07:44:11.000Z","dependencies_parsed_at":"2022-08-12T16:40:53.225Z","dependency_job_id":null,"html_url":"https://github.com/InfuseAI/primehub-job","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InfuseAI%2Fprimehub-job","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InfuseAI%2Fprimehub-job/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InfuseAI%2Fprimehub-job/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InfuseAI%2Fprimehub-job/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InfuseAI","download_url":"https://codeload.github.com/InfuseAI/primehub-job/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244542283,"owners_count":20469294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-20T03:20:53.010Z","updated_at":"2025-03-20T03:20:53.796Z","avatar_url":"https://github.com/InfuseAI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This is still in the Alpha stage.\n\n## PrimeHub Job\n\nSubmit PrimeHub jobs easier\n\n### Install\n\n```\npip install primehub_job\n```\n\n### How to Use\n\n1. PrimeHub version \u003e= 2.8\n2. Set the API_TOKEN environment variable\n3. Make sure your group have group volume\n\n``` python\nfrom primehub_job import submit_phjob, get_phjob_logs, get_phjob_result, wait_and_get_phjob_result\n\n# You can write @submit_job() to use default settings.\n@submit_phjob(name='submitJob', image='datascience-notebook', instance_type='cpu')\ndef test(a, b='b'):\n    print(a)\n    print(b)\n    return a, b\n\n# Submit a job and wait the result\nwait_and_get_phjob_result(test('aaa', 'ccc'))\n\n# Submit a job\njob_id = test('aaa', 'ccc')\n# Get the running logs\nprint(get_phjob_logs(job_id))\n# Get the job result\nget_phjob_result(job_id)\n\n```\n\n### Restrictions\n\n1. Don't use global variables in functions\n2. If you want to return a model, please use the framework's saver to save model, and return the saved model path to load the model. Here is the example:\n```python\n# tensorflow\ndef export_model(model, export_path):\n    if os.path.isdir(export_path):\n        print('Cleaning up\\n')\n        shutil.rmtree(export_path)\n\n    model.save(export_path)\n    return export_path\n\n@submit_phjob()\ndef training(epochs=2, dropout=0.2):\n    mnist = tf.keras.datasets.mnist\n\n    (x_train, y_train),(x_test, y_test) = mnist.load_data()\n    x_train, x_test = x_train / 255.0, x_test / 255.0\n\n    model = tf.keras.models.Sequential([\n      tf.keras.layers.Flatten(input_shape=(28, 28)),\n      tf.keras.layers.Dense(512, activation=tf.nn.relu),\n      tf.keras.layers.Dropout(dropout),\n      tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n    ])\n\n    model.compile(optimizer='adam',\n                  loss='sparse_categorical_crossentropy',\n                  metrics=['accuracy'])\n\n    model.fit(x_train, y_train, epochs=epochs)\n    model.evaluate(x_test, y_test)\n\n    return export_model(model, '1')\n\njob_id = training()\n\n# load trained model\nmodel = tf.keras.models.load_model(get_phjob_folder_path(job_id) + '/' + wait_and_get_phjob_result(job_id))\n``` \n\n### Run the Tests\n\n```\npython -m pytest\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfuseai%2Fprimehub-job","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finfuseai%2Fprimehub-job","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfuseai%2Fprimehub-job/lists"}