{"id":18892984,"url":"https://github.com/dongjunlee/hb-base","last_synced_at":"2025-04-15T00:13:57.326Z","repository":{"id":236588793,"uuid":"115315402","full_name":"DongjunLee/hb-base","owner":"DongjunLee","description":"Project structure of Deep Learning experiments","archived":false,"fork":false,"pushed_at":"2018-01-20T06:45:26.000Z","size":31,"stargazers_count":12,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-15T00:13:41.549Z","etag":null,"topics":["base","deep-learning","experiments","slack-notifications","structure","tensorboard"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DongjunLee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-25T07:10:02.000Z","updated_at":"2023-09-08T17:34:21.000Z","dependencies_parsed_at":"2024-04-27T23:52:28.401Z","dependency_job_id":"d37c0561-f122-4f27-94dc-9f25792ae9d2","html_url":"https://github.com/DongjunLee/hb-base","commit_stats":null,"previous_names":["dongjunlee/hb-base"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DongjunLee%2Fhb-base","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DongjunLee%2Fhb-base/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DongjunLee%2Fhb-base/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DongjunLee%2Fhb-base/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DongjunLee","download_url":"https://codeload.github.com/DongjunLee/hb-base/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248981271,"owners_count":21193147,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["base","deep-learning","experiments","slack-notifications","structure","tensorboard"],"created_at":"2024-11-08T08:06:54.805Z","updated_at":"2025-04-15T00:13:57.306Z","avatar_url":"https://github.com/DongjunLee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/logo.png\" width=250\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\n  \u003ca href=\"https://github.com/hb-research/hb-base\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/DeepLearning-Experiments-brightgreen.svg\" alt=\"Project Introduction\"\u003e\n  \u003c/a\u003e\n  \n  \u003ca href=\"https://github.com/hb-research/hb-base\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Project-Structure-brightgreen.svg\" alt=\"Project Introduction\"\u003e\n  \u003c/a\u003e\n\n\u003c/p\u003e\n\n# hb-base: project structure of Deep Learning experiments\n\n\nhb-base proposes the structure of a deep learning project. Using TensorFlow's higher api a basic structure is provided. If you start a deep learning project from this project, All you need to do is implement the core.\n\n## Why?\n\n- There are many boilerplate codes when creating a new deep learning project.\n- Recommend using **higher APIs** ([Estimator](https://www.tensorflow.org/api_docs/python/tf/estimator/Estimator), [Experiment](https://www.tensorflow.org/api_docs/python/tf/contrib/learn/Experiment), [Dataset](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) and [tf.metrics](https://www.tensorflow.org/api_docs/python/tf/metrics))\n- You can **focus the core** (model's graph).\n- The training or evaluate results are automatically applied to the **TensorBoard**.\n- When terminated learning, you can receive a notificatio with a Slack.\n\n## Requirements\n\n- Python 3.6\n- TensorFlow 1.4\n- [hb-config](https://github.com/hb-research/hb-config) (Singleton Config)\n- requests\n- [Slack Incoming Webhook URL](https://my.slack.com/services/new/incoming-webhook/)\n\n## Project Structure\n\n    .\n    ├── config/                 # Config files (.yml, .json) using with hb-config\n    ├── data/                   # dataset path\n    ├── notebooks/              # Prototyping with numpy or tf.interactivesession\n    ├── scripts/                # download dataset using shell scripts\n    ├── concrete_model/         # concrete model architecture graphs (from input to logits)\n        ├── __init__.py             # Graph logic\n        ├── ...                     # Implements the components or modules\n    ├── data_loader.py          # data_reader, preprocessing, make_batch\n    ├── hook.py                 # training or test hook feature (eg. print_variables, handle training config)\n    ├── main.py                 # define experiment_fn (enable tfdbg)\n    ├── model.py                # define EstimatorSpec      \n    └── utils.py                # slack notification (incoming-webhook)\n\nReference : [hb-config](https://github.com/hb-research/hb-config), [Dataset](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#from_generator), [experiments_fn](https://www.tensorflow.org/api_docs/python/tf/contrib/learn/Experiment), [EstimatorSpec](https://www.tensorflow.org/api_docs/python/tf/estimator/EstimatorSpec), [tfdbg](https://www.tensorflow.org/programmers_guide/debugger)\n\nDirectories below contain dummy data.\n\n- config/\n- data/\n- notebooks/\n- scripts/\n\n## Experiments mode\n\n- `evaluate` : Evaluate on the evaluation data.\n- `extend_train_hooks` :  Extends the hooks for training.\n- `reset_export_strategies` : Resets the export strategies with the new_export_strategies.\n- `run_std_server` : Starts a TensorFlow server and joins the serving thread.\n- `test` : Tests training, evaluating and exporting the estimator for a single step.\n- `train` : Fit the estimator using the training data.\n- `train_and_evaluate` : Interleaves training and evaluation.\n\n\n## Usage Example\n\n\u003e Download as zip then implements concrete model's graph, data_loader and  customizing others.\n\nAfter implements.. Install requirements.\n\n```pip install -r requirements.txt```\n\nThen, Download dataset and pre-processing.\n\n```\nsh scripts/download_dataset.sh\npython data_loader.py --config check-tiny\n```\n\nFinally, start train and evaluate model\n\n```python main.py --config check-tiny --mode train_and_evaluate```\n\n### Tensorboar\n\n```tensorboard --logdir logs```\n\n\n---\n\n## Author\n\n`Maintainer` [Dongjun Lee](https://github.com/DongjunLee) (humanbrain.djlee@gmail.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdongjunlee%2Fhb-base","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdongjunlee%2Fhb-base","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdongjunlee%2Fhb-base/lists"}